DYNR/netbird
2
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-08-05 00:15:26 -04:00
Code Issues 23.5k Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5989] Consolidate connection-mode flags; add p2p-dynamic #11429

New Issue
Open
opened 2026-08-05 01:29:40 -04:00 by saavagebueno · 6 comments
saavagebueno commented 2026-08-05 01:29:40 -04:00
Owner
Copy Link

Originally created by @MichaelUray on GitHub (Apr 25, 2026).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/5989

Edit 2026-05-08: Removed p2p-dynamic-lazy as a separate mode; its two-tier teardown is now part of p2p-dynamic itself, based on implementation feedback. The proposal is now four enum values, not five.

Summary

cc @pappz — would value your input given the engine-side context from #5807 / netbirdio/android-client#152.

This is an RFC-style proposal to replace the two independent peer-connection flags (NB_FORCE_RELAY and NB_ENABLE_EXPERIMENTAL_LAZY_CONN) with a single connection-mode enum that has four explicit values, including one new mode (p2p-dynamic) that combines an activity-triggered relay-first wake-up with a two-tier teardown (ICE first, then relay, then full idle) for bursty mobile/LTE access. Inactivity thresholds become explicit, configurable settings rather than a single env var.

The companion proposal #5990 extends this with per-peer/per-group server-side override of both the mode and the thresholds.

This addresses the same use case as #5589 (mobile default flip) and #4103 (UI/CLI exposure of relay-only mode) with a broader mechanism. If maintainers agree this is the right direction, the original authors of #5589 / #4103 may want to consider whether their issues are still independently needed or can be closed in favor of this work.

Background

The peer-connection state machine in client/internal/peer/conn.go is currently controlled by two independent settings whose effects overlap on the same code path:

Each has its own client-side and server-side toggles, and the precedence between them is asymmetric and undocumented (see conn_mgr.go:48-82). The recently closed android-client#152 (revert ForceRelay default to false on Android) made it visible that the binary force-relay flag is too coarse for mobile defaults: turning it off costs battery on large meshes (eager ICE for unused peers, see #1354, #2138), turning it on prevents same-LAN P2P even when peers are in the same subnet (see #5589).

Proposed solution

Single enum connection-mode with four values:

Mode Behavior Maps to existing
relay-forced Skip ICE entirely; transport is relay only. Relay stays up indefinitely. NB_FORCE_RELAY=true skip-ICE branch in peer/conn.go:188-203
p2p Eager worker_relay + worker_ice in parallel; hot-swap to P2P on success (conn.go:421 redirect packets from relayed conn to WireGuard). Both stay up indefinitely. current default for non-mobile platforms
p2p-lazy No connection at all until WireGuard sees outgoing traffic to the peer; then full worker_relay + worker_ice. After relay-idle-threshold without traffic, the entire connection is torn down. client/internal/lazyconn/ package as-is
p2p-dynamic (new) Activity-triggered relay-first wake-up: WireGuard activity opens the relay path immediately so the user's first packet flows over relay while worker_ice runs in parallel; once ICE succeeds, traffic hot-swaps to P2P. Two-tier teardown: after p2p-idle-threshold without traffic the ICE worker tears down (relay stays warm for fast resume); after a longer relay-idle-threshold without any traffic the relay also tears down, returning the peer to fully idle. Combines p2p-lazy's zero-cost-when-truly-idle property with low-latency-on-first-packet for recently used peers. new

Why this new mode

p2p-dynamic addresses two structural issues that the binary ForceRelay flag cannot resolve:

  • Idle battery + data cost ≈ p2p-lazy when no peers are recently used (no per-peer transport traffic for unused peers), but first-packet latency to recently-used peers ≈ always-connected because the relay path is opened before ICE finishes negotiation.
  • Active-traffic latency ≈ p2p once the upgrade settles (~1s of relay-routed traffic at the start of each active session before the hot-swap; subsequent traffic is direct).
  • Resolves the lazy/eager mismatch issue: today an eager peer keeps waking up a lazy peer because DeactivatePeer is a no-op when the local manager is not in lazy mode (the lazy peer's GO_IDLE signal is silently ignored, so the eager side immediately reconnects).
  • Same-LAN P2P (the original motivation for closed netbirdio/android-client#152 and #5589) works automatically once peers actually start communicating — no relay round-trip via the internet for traffic between same-LAN peers.

The two-tier teardown is the key shape: cost (mobile data + battery) scales with recently active peers rather than total reachable peers, while bursty access (e.g. tap a peer, use for a few minutes, idle, come back five minutes later) feels instant because the warm relay path resumes faster than a cold ICE re-negotiation. Mode resolution stays predictable; thresholds are an orthogonal config concern (next section).

Inactivity thresholds — explicit settings, configurable per scope

Two explicit thresholds replace today's single NB_LAZY_CONN_INACTIVITY_THRESHOLD env var:

Setting Default Applies to Effect on inactivity expiry
p2p-idle-threshold 5 min (proposal) p2p-dynamic ICE worker torn down; relay stays warm
relay-idle-threshold 1 h (proposal) p2p-lazy, p2p-dynamic Relay torn down (and ICE if still up); next packet re-opens everything

Both thresholds are configurable independently and follow the same source hierarchy as the mode itself (covered in the companion proposal): account default → per-group → per-peer override, with explicit client-side override on top. Ship reasonable defaults, let admins / power users tune.

relay-forced and p2p are unaffected by either threshold — those modes are explicitly always-on by design. NB_LAZY_CONN_INACTIVITY_THRESHOLD continues to work as a backwards-compat alias for relay-idle-threshold (see backwards compatibility below).

Phased rollout — no default changes in this proposal

This proposal explicitly does NOT change any default mode for any platform. The new mode ships as an opt-in choice alongside the existing three behaviors (preserved via the backwards-compat mapping below). Once the implementation is in users' hands and field telemetry exists for the new mode's real-world behavior (battery, latency, edge cases), a follow-up discussion can decide whether to make it the new universal default — ideally a single default across all platforms rather than continuing today's mobile-vs-non-mobile split.

This phasing avoids relitigating the default-flip question while the new mechanism is unproven.

Backwards compatibility

Existing knobs continue to work and map to the new enum, with deprecation notices in --help text and docs:

  • NB_FORCE_RELAY=true → connection-mode=relay-forced
  • NB_FORCE_RELAY=false (or unset) + NB_ENABLE_EXPERIMENTAL_LAZY_CONN=true → connection-mode=p2p-lazy
  • --enable-lazy-connection → --connection-mode=p2p-lazy
  • Account-level Settings.LazyConnectionEnabled=true → equivalent to setting account-level connection-mode=p2p-lazy
  • NB_LAZY_CONN_INACTIVITY_THRESHOLD → backwards-compat alias for relay-idle-threshold

No env-var or CLI removal in this change; deprecate in this minor, remove no earlier than next major.

Settings-source precedence (client-side)

Replace the current asymmetric "client-ON locks server out, client cannot opt-out of server-ON" with a single explicit precedence (applies to both the mode and the thresholds):

  1. Client env var (highest — for debug/CI)
  2. Client config (CLI/UI explicit set, including the special value follow-server to clear a local override)
  3. Server-pushed value (default — what the server resolves for this peer)

Each layer is allowed to set any of the four modes (not just enable/disable) and to override either threshold independently, so a power-user can explicitly opt out of an account-wide setting in either direction (today not possible).

Server-side per-peer/per-group resolution that produces the value sent to the client is covered in the companion proposal.

Implementation notes

Most pieces already exist:

  • relay-forced: existing skip-ICE branch in peer/conn.go:188.
  • p2p: existing default code path.
  • p2p-lazy: existing client/internal/lazyconn/ package with one threshold (relay-idle-threshold).
  • p2p-dynamic: new — but reuses lazyconn/activity/ for the activity-detector and lazyconn/inactivity/manager for the tear-downs. The novel pieces are: (a) opening the relay path in parallel with ICE on the activity trigger so the first user packet flows immediately over relay, and (b) adding a second timer to the per-peer inactivity manager so the ICE worker can be torn down independently of the relay path.

Subnet-router peers stay always-on via the existing ExcludePeer mechanism. Rosenpass remains mutually exclusive with p2p-lazy and p2p-dynamic (same constraint as today, conn_mgr.go:66). Mobile clients drop their ad-hoc UI for ForceRelay and adopt a single mode-picker; the existing Android EnvKeyNBLazyConn/EnvKeyNBInactivityThreshold exports in client/android/env_list.go are already in place for the gomobile binding.

Related issues

  • #5589 — same use case (mobile relay default), broader mechanism here. Original author may want to consider closing in favor of this if accepted.
  • #4103 — same use case (UI/CLI exposure of relay-only mode), addressed by the unified mode-picker. Original author may want to consider closing.
  • Closed android-client#152 — addressed by p2p-dynamic which gives same-LAN P2P without the eager-ICE battery cost.
  • Closed netbirdio/ios-client#94 — closed in favor of this proposal.
  • #1354 — long-standing battery usage discussion (origin of ForceRelay=true mobile default).
  • #2138 — iOS battery / usability feedback.
  • Companion proposal: server-side per-peer/per-group connection-mode override #5990.
Originally created by @MichaelUray on GitHub (Apr 25, 2026). Original GitHub issue: https://github.com/netbirdio/netbird/issues/5989 > **Edit 2026-05-08:** Removed `p2p-dynamic-lazy` as a separate mode; its two-tier teardown is now part of `p2p-dynamic` itself, based on implementation feedback. The proposal is now four enum values, not five. ## Summary cc @pappz — would value your input given the engine-side context from #5807 / netbirdio/android-client#152. This is an RFC-style proposal to replace the two independent peer-connection flags (`NB_FORCE_RELAY` and `NB_ENABLE_EXPERIMENTAL_LAZY_CONN`) with a single `connection-mode` enum that has four explicit values, including one new mode (`p2p-dynamic`) that combines an activity-triggered relay-first wake-up with a two-tier teardown (ICE first, then relay, then full idle) for bursty mobile/LTE access. Inactivity thresholds become explicit, configurable settings rather than a single env var. The companion proposal #5990 extends this with per-peer/per-group server-side override of both the mode and the thresholds. This addresses the same use case as #5589 (mobile default flip) and #4103 (UI/CLI exposure of relay-only mode) with a broader mechanism. If maintainers agree this is the right direction, the original authors of #5589 / #4103 may want to consider whether their issues are still independently needed or can be closed in favor of this work. ## Background The peer-connection state machine in `client/internal/peer/conn.go` is currently controlled by two independent settings whose effects overlap on the same code path: - `NB_FORCE_RELAY` / `EnvKeyNBForceRelay` ([peer/env.go](https://github.com/netbirdio/netbird/blob/main/client/internal/peer/env.go)) - `NB_ENABLE_EXPERIMENTAL_LAZY_CONN` / `LazyConnectionEnabled` ([lazyconn/env.go](https://github.com/netbirdio/netbird/blob/main/client/internal/lazyconn/env.go)) Each has its own client-side and server-side toggles, and the precedence between them is asymmetric and undocumented (see [conn_mgr.go:48-82](https://github.com/netbirdio/netbird/blob/main/client/internal/conn_mgr.go#L48-L82)). The recently closed [android-client#152](https://github.com/netbirdio/android-client/pull/152) (revert ForceRelay default to false on Android) made it visible that the binary `force-relay` flag is too coarse for mobile defaults: turning it off costs battery on large meshes (eager ICE for unused peers, see [#1354](https://github.com/netbirdio/netbird/issues/1354), [#2138](https://github.com/netbirdio/netbird/issues/2138)), turning it on prevents same-LAN P2P even when peers are in the same subnet (see [#5589](https://github.com/netbirdio/netbird/issues/5589)). ## Proposed solution Single enum `connection-mode` with four values: | Mode | Behavior | Maps to existing | |---|---|---| | `relay-forced` | Skip ICE entirely; transport is relay only. Relay stays up indefinitely. | `NB_FORCE_RELAY=true` skip-ICE branch in `peer/conn.go:188-203` | | `p2p` | Eager `worker_relay` + `worker_ice` in parallel; hot-swap to P2P on success (`conn.go:421` `redirect packets from relayed conn to WireGuard`). Both stay up indefinitely. | current default for non-mobile platforms | | `p2p-lazy` | No connection at all until WireGuard sees outgoing traffic to the peer; then full `worker_relay` + `worker_ice`. After `relay-idle-threshold` without traffic, the entire connection is torn down. | `client/internal/lazyconn/` package as-is | | `p2p-dynamic` *(new)* | Activity-triggered relay-first wake-up: WireGuard activity opens the relay path immediately so the user's first packet flows over relay while `worker_ice` runs in parallel; once ICE succeeds, traffic hot-swaps to P2P. Two-tier teardown: after `p2p-idle-threshold` without traffic the ICE worker tears down (relay stays warm for fast resume); after a longer `relay-idle-threshold` without any traffic the relay also tears down, returning the peer to fully idle. Combines `p2p-lazy`'s zero-cost-when-truly-idle property with low-latency-on-first-packet for recently used peers. | new | ### Why this new mode `p2p-dynamic` addresses two structural issues that the binary `ForceRelay` flag cannot resolve: - **Idle battery + data cost ≈ `p2p-lazy`** when no peers are recently used (no per-peer transport traffic for unused peers), but **first-packet latency to recently-used peers ≈ always-connected** because the relay path is opened before ICE finishes negotiation. - **Active-traffic latency ≈ `p2p`** once the upgrade settles (~1s of relay-routed traffic at the start of each active session before the hot-swap; subsequent traffic is direct). - **Resolves the lazy/eager mismatch issue:** today an eager peer keeps waking up a lazy peer because [`DeactivatePeer`](https://github.com/netbirdio/netbird/blob/main/client/internal/conn_mgr.go#L237-L244) is a no-op when the local manager is not in lazy mode (the lazy peer's `GO_IDLE` signal is silently ignored, so the eager side immediately reconnects). - **Same-LAN P2P** (the original motivation for closed netbirdio/android-client#152 and #5589) works automatically once peers actually start communicating — no relay round-trip via the internet for traffic between same-LAN peers. The two-tier teardown is the key shape: cost (mobile data + battery) scales with **recently active** peers rather than **total reachable** peers, while bursty access (e.g. tap a peer, use for a few minutes, idle, come back five minutes later) feels instant because the warm relay path resumes faster than a cold ICE re-negotiation. Mode resolution stays predictable; thresholds are an orthogonal config concern (next section). ### Inactivity thresholds — explicit settings, configurable per scope Two explicit thresholds replace today's single `NB_LAZY_CONN_INACTIVITY_THRESHOLD` env var: | Setting | Default | Applies to | Effect on inactivity expiry | |---|---|---|---| | `p2p-idle-threshold` | 5 min (proposal) | `p2p-dynamic` | ICE worker torn down; relay stays warm | | `relay-idle-threshold` | 1 h (proposal) | `p2p-lazy`, `p2p-dynamic` | Relay torn down (and ICE if still up); next packet re-opens everything | Both thresholds are configurable independently and follow the same source hierarchy as the mode itself (covered in the companion proposal): account default → per-group → per-peer override, with explicit client-side override on top. Ship reasonable defaults, let admins / power users tune. `relay-forced` and `p2p` are unaffected by either threshold — those modes are explicitly always-on by design. `NB_LAZY_CONN_INACTIVITY_THRESHOLD` continues to work as a backwards-compat alias for `relay-idle-threshold` (see backwards compatibility below). ### Phased rollout — no default changes in this proposal This proposal explicitly does NOT change any default mode for any platform. The new mode ships as an opt-in choice alongside the existing three behaviors (preserved via the backwards-compat mapping below). Once the implementation is in users' hands and field telemetry exists for the new mode's real-world behavior (battery, latency, edge cases), a follow-up discussion can decide whether to make it the new universal default — ideally a single default across all platforms rather than continuing today's mobile-vs-non-mobile split. This phasing avoids relitigating the default-flip question while the new mechanism is unproven. ### Backwards compatibility Existing knobs continue to work and map to the new enum, with deprecation notices in `--help` text and docs: - `NB_FORCE_RELAY=true` → `connection-mode=relay-forced` - `NB_FORCE_RELAY=false` (or unset) + `NB_ENABLE_EXPERIMENTAL_LAZY_CONN=true` → `connection-mode=p2p-lazy` - `--enable-lazy-connection` → `--connection-mode=p2p-lazy` - Account-level `Settings.LazyConnectionEnabled=true` → equivalent to setting account-level `connection-mode=p2p-lazy` - `NB_LAZY_CONN_INACTIVITY_THRESHOLD` → backwards-compat alias for `relay-idle-threshold` No env-var or CLI removal in this change; deprecate in this minor, remove no earlier than next major. ### Settings-source precedence (client-side) Replace the current asymmetric "client-ON locks server out, client cannot opt-out of server-ON" with a single explicit precedence (applies to both the mode and the thresholds): 1. Client env var (highest — for debug/CI) 2. Client config (CLI/UI explicit set, including the special value `follow-server` to clear a local override) 3. Server-pushed value (default — what the server resolves for this peer) Each layer is allowed to set any of the four modes (not just enable/disable) and to override either threshold independently, so a power-user can explicitly opt **out** of an account-wide setting in either direction (today not possible). Server-side per-peer/per-group resolution that produces the value sent to the client is covered in the companion proposal. ### Implementation notes Most pieces already exist: - **`relay-forced`**: existing skip-ICE branch in `peer/conn.go:188`. - **`p2p`**: existing default code path. - **`p2p-lazy`**: existing `client/internal/lazyconn/` package with one threshold (`relay-idle-threshold`). - **`p2p-dynamic`**: new — but reuses [`lazyconn/activity/`](https://github.com/netbirdio/netbird/tree/main/client/internal/lazyconn/activity) for the activity-detector and [`lazyconn/inactivity/manager`](https://github.com/netbirdio/netbird/tree/main/client/internal/lazyconn/inactivity) for the tear-downs. The novel pieces are: (a) opening the relay path in parallel with ICE on the activity trigger so the first user packet flows immediately over relay, and (b) adding a second timer to the per-peer inactivity manager so the ICE worker can be torn down independently of the relay path. Subnet-router peers stay always-on via the existing `ExcludePeer` mechanism. Rosenpass remains mutually exclusive with `p2p-lazy` and `p2p-dynamic` (same constraint as today, [conn_mgr.go:66](https://github.com/netbirdio/netbird/blob/main/client/internal/conn_mgr.go#L66)). Mobile clients drop their ad-hoc UI for ForceRelay and adopt a single mode-picker; the existing Android `EnvKeyNBLazyConn`/`EnvKeyNBInactivityThreshold` exports in `client/android/env_list.go` are already in place for the gomobile binding. ## Related issues - [#5589](https://github.com/netbirdio/netbird/issues/5589) — same use case (mobile relay default), broader mechanism here. Original author may want to consider closing in favor of this if accepted. - [#4103](https://github.com/netbirdio/netbird/issues/4103) — same use case (UI/CLI exposure of relay-only mode), addressed by the unified mode-picker. Original author may want to consider closing. - Closed [android-client#152](https://github.com/netbirdio/android-client/pull/152) — addressed by `p2p-dynamic` which gives same-LAN P2P without the eager-ICE battery cost. - Closed [netbirdio/ios-client#94](https://github.com/netbirdio/ios-client/pull/94) — closed in favor of this proposal. - [#1354](https://github.com/netbirdio/netbird/issues/1354) — long-standing battery usage discussion (origin of `ForceRelay=true` mobile default). - [#2138](https://github.com/netbirdio/netbird/issues/2138) — iOS battery / usability feedback. - Companion proposal: server-side per-peer/per-group connection-mode override #5990.
saavagebueno commented 2026-08-05 01:29:40 -04:00
Author
Owner
Copy Link

@MichaelUray commented on GitHub (May 1, 2026):

Phase 1 PRs gepostet:

  • Backend: netbirdio/netbird#6047 -- ConnectionMode enum + 3 modes implementiert (relay-forced, p2p, p2p-lazy), backwards-compat-mapping aller alten Flags, neue Source-Precedence (env > config > server-pushed). p2p-dynamic ist proto/DB-mäßig reserviert, daemon-seitig pass-through bis Phase 2.
  • Dashboard: netbirdio/dashboard#627 -- Lazy-Toggle ersetzt durch 2-Wert-Dropdown + conditional relay-timeout input. relay-forced und p2p-dynamic bleiben in Phase 1 admin-only.

Konsolidierung gegenüber dem RFC: Beim Implementation-Brainstorm wurden p2p-dynamic und p2p-dynamic-lazy zu einem einzelnen p2p-dynamic-Mode mit zwei orthogonalen Timeouts (p2p_timeout, relay_timeout, 0 = disabled) zusammengeführt. Das RFC schlug 5 Modes vor, die Implementation hat 4. Begründung: weniger mentaler Overhead, eine einzige Mode-Achse, Verhalten orthogonal über klar benannte Threshold-Werte konfigurierbar. Die zwei Timeouts gelten mode-übergreifend (relay_timeout wirkt in p2p-lazy und p2p-dynamic, p2p_timeout nur in p2p-dynamic). Falls die Maintainer den 5-Mode-Aufbau bevorzugen, kann das in einem follow-up-Commit getrennt werden.

Hardware-tested auf einer produktiven NetBird-Instanz mit 32 connected peers über 12 OpenWrt-Router-Versionen (22.03 bis 25.12), Windows 10/11, Debian 13, Android 12/14, iOS 26.3.1: Cutover ohne Disconnect, zwei Mode-Wechsel (p2p-lazy <-> p2p via API) ohne Disconnect, 8-Min-Monitor zeigt zero peer-count drift. Backwards-compat-Vertrag hält: alte Daemons sehen weiterhin nur den alten lazy_connection_enabled-Boolean, der via toPeerConfig gemapped wird.

Phase 2 (p2p-dynamic daemon-Implementierung -- decoupled worker_relay/worker_ice OnNewOffer-Registrierung, two-tier inactivity manager, DeactivatePeer-no-op-Fix) und Phase 3 (= #5990, per-peer/per-group Resolution) folgen in eigenen PRs.

<!-- gh-comment-id:4359657548 --> @MichaelUray commented on GitHub (May 1, 2026): **Phase 1 PRs gepostet**: - Backend: netbirdio/netbird#6047 -- ConnectionMode enum + 3 modes implementiert (`relay-forced`, `p2p`, `p2p-lazy`), backwards-compat-mapping aller alten Flags, neue Source-Precedence (env > config > server-pushed). `p2p-dynamic` ist proto/DB-mäßig reserviert, daemon-seitig pass-through bis Phase 2. - Dashboard: netbirdio/dashboard#627 -- Lazy-Toggle ersetzt durch 2-Wert-Dropdown + conditional relay-timeout input. `relay-forced` und `p2p-dynamic` bleiben in Phase 1 admin-only. **Konsolidierung gegenüber dem RFC**: Beim Implementation-Brainstorm wurden `p2p-dynamic` und `p2p-dynamic-lazy` zu einem einzelnen `p2p-dynamic`-Mode mit zwei orthogonalen Timeouts (`p2p_timeout`, `relay_timeout`, `0` = disabled) zusammengeführt. Das RFC schlug 5 Modes vor, die Implementation hat 4. Begründung: weniger mentaler Overhead, eine einzige Mode-Achse, Verhalten orthogonal über klar benannte Threshold-Werte konfigurierbar. Die zwei Timeouts gelten mode-übergreifend (`relay_timeout` wirkt in `p2p-lazy` und `p2p-dynamic`, `p2p_timeout` nur in `p2p-dynamic`). Falls die Maintainer den 5-Mode-Aufbau bevorzugen, kann das in einem follow-up-Commit getrennt werden. **Hardware-tested** auf einer produktiven NetBird-Instanz mit 32 connected peers über 12 OpenWrt-Router-Versionen (22.03 bis 25.12), Windows 10/11, Debian 13, Android 12/14, iOS 26.3.1: Cutover ohne Disconnect, zwei Mode-Wechsel (`p2p-lazy` <-> `p2p` via API) ohne Disconnect, 8-Min-Monitor zeigt zero peer-count drift. Backwards-compat-Vertrag hält: alte Daemons sehen weiterhin nur den alten `lazy_connection_enabled`-Boolean, der via `toPeerConfig` gemapped wird. Phase 2 (`p2p-dynamic` daemon-Implementierung -- decoupled `worker_relay`/`worker_ice` `OnNewOffer`-Registrierung, two-tier inactivity manager, `DeactivatePeer`-no-op-Fix) und Phase 3 (= #5990, per-peer/per-group Resolution) folgen in eigenen PRs.
saavagebueno commented 2026-08-05 01:29:40 -04:00
Author
Owner
Copy Link

@MichaelUray commented on GitHub (May 6, 2026):

@mlsmaycon
I think there was no dedicated discussion section available for me on this Github repository when I opened this issues here for a discussion.
Not sure if it makes sense to open a Github discussion with a duplication of this issue here.

<!-- gh-comment-id:4390200641 --> @MichaelUray commented on GitHub (May 6, 2026): @mlsmaycon I think there was no dedicated discussion section available for me on this Github repository when I opened this issues here for a discussion. Not sure if it makes sense to open a Github discussion with a duplication of this issue here.
saavagebueno commented 2026-08-05 01:29:40 -04:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (May 7, 2026):

@MichaelUray Thanks for putting this proposal together — it’s an interesting direction. That said, I think it’s missing some context around how the current modes already behave today. Let me walk through the existing modes and some of the decisions behind them:

  1. Default mode (workstations)
    By default, peers establish a relay connection while simultaneously attempting to create a direct connection. Once a direct connection succeeds, traffic switches over to it, while the relay path remains available as a failover if the ICE/direct connection drops. In this mode, peers exchange health checks at short intervals, which are data-transfer-hungry and consume power too.

  2. Force relay mode
    In this mode, ICE/direct connection attempts are skipped and only relay connections are used. This is currently the default behavior on mobile clients because it helps reduce battery and data usage. Users can disable it at any time directly on the client side. On workstation clients, this mode exists mainly through explicitly defined environment variables and is generally intended for testing/debugging scenarios.

  3. Lazy connection mode
    Lazy connections can be enabled both on the client side (on both peers) and on the management side. In this mode, connections are established only when WireGuard detects actual traffic/activity. Once triggered, peers establish either a relayed or direct P2P connection, which remains active until it has been idle for one hour. After that, the connection is torn down and peers return to waiting for the next activity trigger. During active periods, health checks are still exchanged at short intervals.

We’re currently moving toward enabling lazy connections by default for new accounts. With some improvements already planned around health checks, we may also be able to disable forced relay mode on mobile devices in the coming weeks.

Regarding the proposal itself and the implementation:

While the idea is interesting, I’m concerned it could introduce additional complexity and make configuration synchronization between peers harder to reason about. One thing we could explore instead is reducing the idle timeout in lazy mode from 1 hour down to something like 15–30 minutes.

The PRs around these changes are also becoming fairly large. Ideally, I’d expect changes in this area to follow the existing lazy connection patterns more closely — for example, by adding smaller configuration knobs for relay forcing or idle timing — rather than introducing a broader behavioral shift. Right now, it feels like the implementation is drifting a bit outside the current design context.

For the time being, we’d be very happy to accept a smaller contribution around a dedicated “force relay” flag, since that is already something planned internally. However, accepting a much larger behavioral change would be difficult while we’re focused on stabilization work for the 1.0 release, as it would increase overall risk quite a bit.

<!-- gh-comment-id:4395315827 --> @mlsmaycon commented on GitHub (May 7, 2026): @MichaelUray Thanks for putting this proposal together — it’s an interesting direction. That said, I think it’s missing some context around how the current modes already behave today. Let me walk through the existing modes and some of the decisions behind them: 1. Default mode (workstations) By default, peers establish a relay connection while simultaneously attempting to create a direct connection. Once a direct connection succeeds, traffic switches over to it, while the relay path remains available as a failover if the ICE/direct connection drops. In this mode, peers exchange health checks at short intervals, which are data-transfer-hungry and consume power too. 2. Force relay mode In this mode, ICE/direct connection attempts are skipped and only relay connections are used. This is currently the default behavior on mobile clients because it helps reduce battery and data usage. Users can disable it at any time directly on the client side. On workstation clients, this mode exists mainly through explicitly defined environment variables and is generally intended for testing/debugging scenarios. 3. Lazy connection mode Lazy connections can be enabled both on the client side (on both peers) and on the management side. In this mode, connections are established only when WireGuard detects actual traffic/activity. Once triggered, peers establish either a relayed or direct P2P connection, which remains active until it has been idle for one hour. After that, the connection is torn down and peers return to waiting for the next activity trigger. During active periods, health checks are still exchanged at short intervals. We’re currently moving toward enabling lazy connections by default for new accounts. With some improvements already planned around health checks, we may also be able to disable forced relay mode on mobile devices in the coming weeks. Regarding the proposal itself and the implementation: While the idea is interesting, I’m concerned it could introduce additional complexity and make configuration synchronization between peers harder to reason about. One thing we could explore instead is reducing the idle timeout in lazy mode from 1 hour down to something like 15–30 minutes. The PRs around these changes are also becoming fairly large. Ideally, I’d expect changes in this area to follow the existing lazy connection patterns more closely — for example, by adding smaller configuration knobs for relay forcing or idle timing — rather than introducing a broader behavioral shift. Right now, it feels like the implementation is drifting a bit outside the current design context. For the time being, we’d be very happy to accept a smaller contribution around a dedicated “force relay” flag, since that is already something planned internally. However, accepting a much larger behavioral change would be difficult while we’re focused on stabilization work for the 1.0 release, as it would increase overall risk quite a bit.
saavagebueno commented 2026-08-05 01:29:40 -04:00
Author
Owner
Copy Link

@MichaelUray commented on GitHub (May 7, 2026):

@mlsmaycon Thanks for the context. I understand the concern that this looks like a broader behavioral shift, so let me explain the reasoning.

Why a separate p2p-dynamic mode

The main reason I modeled this as a new opt-in mode rather than changing p2p-lazy is compatibility and operational clarity:

  • p2p keeps its current relay+ICE behavior, unchanged.
  • p2p-lazy keeps its existing single-tier lazy model.
  • Older clients that don't advertise support for p2p_dynamic get capability-gated downgrade to existing p2p-lazy (already wired through SupportedFeatures + LegacyLazyFallback).
  • Only new clients on explicitly configured accounts run the new lifecycle.
  • A single mode label keeps the operational picture simple — each peer is in exactly one mode, which is easier to reason about in logs and debugging than a combination of independent flags whose semantics may overlap or conflict.

This means mixed-version accounts stay predictable: each peer's behavior follows a mode label it understands, with no in-place semantic drift on existing modes.

To address the configuration-synchronization concern specifically: the intention is that the management server resolves one effective mode and the effective timers per peer, and clients expose both the configured and effective values in status/debug output. So when debugging a connection, the question is not "which combination of flags happened to win?", but "what effective mode did this peer receive, and which fallback/capability decision was applied?"

This also fits the direction you mentioned — lazy-by-default for new accounts, and dropping force-relay as the mobile default in the coming weeks.
p2p-dynamic is intended to complement that direction, not compete with it.
A peer in this mode without recent activity uses the same idle/no per-peer connection state as lazy mode; the difference is what happens during the active phase for peers the user is actually using.

What the new lifecycle adds

The motivation isn't "lazy with a shorter timeout" — it's an extra state in the active phase, aimed at bursty mobile/LTE (limited data volume) access patterns:

[Idle, no per-peer transport traffic]
       │
       │ WG activity trigger
       ▼
[Relay-up, ICE in parallel]  ── user traffic can use relay
       │                         while ICE/P2P continues
       │ ICE succeeds
       ▼
[Relay + P2P, full active]  ◄──────┐
       │                           │
       │ p2p_idle_timeout (short)  │ new traffic →
       ▼                           │ fast-path P2P re-attach
[Relay-warm, P2P torn down]  ──────┘
       │   (ICE/STUN/TURN machinery off; only the relay/WG path remains warm)
       │
       │ relay_idle_timeout (long, no activity)
       ▼
[Idle, no per-peer transport traffic again]

The two intended differences from existing lazy:

  1. Relay-first wake-up is explicit, so user traffic can use the relay path while ICE/P2P negotiation continues in parallel.
  2. Two-tier teardown: P2P drops after short inactivity, relay stays warm briefly, full idle after longer inactivity.

For mobile/LTE users with many reachable peers but only a few recently used ones, the goal is that cost (mobile data + battery) scales with recently active peers, not total reachable peers, while still preserving fast first access for recently used peers.

Initial hardware tests in my mixed mobile/LTE-style setup with 20+ accessible peers suggested promising reductions in idle and recently-active peer overhead, while still allowing fast reconnection to recently used peers.

Curious to hear your thoughts about it.

<!-- gh-comment-id:4401310875 --> @MichaelUray commented on GitHub (May 7, 2026): @mlsmaycon Thanks for the context. I understand the concern that this looks like a broader behavioral shift, so let me explain the reasoning. ## Why a separate `p2p-dynamic` mode The main reason I modeled this as a new opt-in mode rather than changing `p2p-lazy` is compatibility and operational clarity: - `p2p` keeps its current relay+ICE behavior, unchanged. - `p2p-lazy` keeps its existing single-tier lazy model. - Older clients that don't advertise support for `p2p_dynamic` get capability-gated downgrade to existing `p2p-lazy` (already wired through `SupportedFeatures` + `LegacyLazyFallback`). - Only new clients on explicitly configured accounts run the new lifecycle. - A single mode label keeps the operational picture simple — each peer is in exactly one mode, which is easier to reason about in logs and debugging than a combination of independent flags whose semantics may overlap or conflict. This means mixed-version accounts stay predictable: each peer's behavior follows a mode label it understands, with no in-place semantic drift on existing modes. To address the configuration-synchronization concern specifically: the intention is that the management server resolves one effective mode and the effective timers per peer, and clients expose both the configured and effective values in status/debug output. So when debugging a connection, the question is not "which combination of flags happened to win?", but "what effective mode did this peer receive, and which fallback/capability decision was applied?" This also fits the direction you mentioned — lazy-by-default for new accounts, and dropping force-relay as the mobile default in the coming weeks. `p2p-dynamic` is intended to complement that direction, not compete with it. A peer in this mode without recent activity uses the same idle/no per-peer connection state as lazy mode; the difference is what happens during the active phase for peers the user is actually using. ## What the new lifecycle adds The motivation isn't "lazy with a shorter timeout" — it's an extra state in the active phase, aimed at bursty mobile/LTE (limited data volume) access patterns: ``` [Idle, no per-peer transport traffic] │ │ WG activity trigger ▼ [Relay-up, ICE in parallel] ── user traffic can use relay │ while ICE/P2P continues │ ICE succeeds ▼ [Relay + P2P, full active] ◄──────┐ │ │ │ p2p_idle_timeout (short) │ new traffic → ▼ │ fast-path P2P re-attach [Relay-warm, P2P torn down] ──────┘ │ (ICE/STUN/TURN machinery off; only the relay/WG path remains warm) │ │ relay_idle_timeout (long, no activity) ▼ [Idle, no per-peer transport traffic again] ``` The two intended differences from existing lazy: 1. **Relay-first wake-up is explicit**, so user traffic can use the relay path while ICE/P2P negotiation continues in parallel. 2. **Two-tier teardown**: P2P drops after short inactivity, relay stays warm briefly, full idle after longer inactivity. For mobile/LTE users with many reachable peers but only a few recently used ones, the goal is that cost (mobile data + battery) scales with recently active peers, not total reachable peers, while still preserving fast first access for recently used peers. Initial hardware tests in my mixed mobile/LTE-style setup with 20+ accessible peers suggested promising reductions in idle and recently-active peer overhead, while still allowing fast reconnection to recently used peers. Curious to hear your thoughts about it.
saavagebueno commented 2026-08-05 01:29:41 -04:00
Author
Owner
Copy Link

@MichaelUray commented on GitHub (May 25, 2026):

@mlsmaycon

did you actually get a chance to look through my further explanations in my comment above?

I realize that reply and the issue body together covered a lot at once, and the timeline now has quite a few linked items between then and now — so it's possible the actual value got buried. Putting the concrete improvements into a short list might help:

  • New active-phase state for p2p-dynamic: relay stays warm briefly after ICE/P2P teardown, enabling fast-path P2P re-attach when a peer becomes active again. This is the key behavioral difference from existing lazy mode and is targeted at mobile/LTE access patterns where users have many reachable peers but only a few recently used ones.
  • Two orthogonal timers (p2p_timeout, relay_timeout), both server-pushed and mode-agnostic — replacing the single 1h hardcoded timeout in existing lazy mode with configurable, tunable per-account settings.
  • Per-peer effective-mode visibility in the client: the daemon exposes both the configured and the server-resolved effective mode per peer in status/debug output, so when a connection misbehaves the question becomes "what effective mode did this peer receive?" rather than "which combination of flags happened to win?". For mixed-version fleets this materially improves debuggability.
  • Legacy-client compatibility is explicit: older clients are capability-gated via SupportedFeatures and downgraded to existing p2p-lazy via LegacyLazyFallback. Existing accounts and clients aren't affected unless they explicitly enroll in the new mode label. No in-place semantic drift on existing modes.
  • Hardware-tested: cutover and live mode-switching tested without disconnects across 32 connected peers spanning 12 OpenWrt versions (22.03 - 25.12), Windows 10/11, Debian 13, Android 12/14, iOS 26.3.1. Mixed Kernel + Userspace WireGuard. Mode toggles (p2p-lazy ↔ p2p via API) did not interrupt active sessions.

The reasoning for modeling this as a new opt-in label instead of changing existing modes is in my 2026-05-07 reply — I won't repeat it here, but the short version is: each peer stays in exactly one labeled mode, capability-gated for backwards compatibility, with a single effective resolution per peer that the client surfaces in status. That keeps mixed-version operation predictable.

Since the 2026-05-07 reply, the work has continued:

  • Phase-3.7i orphan-disconnect: built on top of the foundation stack — addresses peers that never receive an explicit lastActive update and could stay stuck in inactivity tracking.
  • Phase-3.7j GO_IDLE receive-side handling: field-deployed across the same fleet plus Windows 11 and Android 16 builds.
  • One concrete bug surfaced and was fixed: a remote GO_IDLE from a legacy lazy-mode peer (e.g. v0.51.2) could trigger an unintended ICE-detach + guard-retry-exhaustion cycle on newer clients, leaving the tunnel relay-only until the next manual reconnect.
  • Standalone PR offer: if useful, that specific receive-side fix could be a small isolated PR against current main.

Thanks for taking the time.

Michael

<!-- gh-comment-id:4534124708 --> @MichaelUray commented on GitHub (May 25, 2026): @mlsmaycon did you actually get a chance to look through my further explanations in [my comment above](https://github.com/netbirdio/netbird/issues/5989#issuecomment-4401310875)? I realize that reply and the issue body together covered a lot at once, and the timeline now has quite a few linked items between then and now — so it's possible the actual value got buried. Putting the concrete improvements into a short list might help: - **New active-phase state for `p2p-dynamic`**: relay stays warm briefly after ICE/P2P teardown, enabling fast-path P2P re-attach when a peer becomes active again. This is the key behavioral difference from existing lazy mode and is targeted at mobile/LTE access patterns where users have many reachable peers but only a few recently used ones. - **Two orthogonal timers** (`p2p_timeout`, `relay_timeout`), both server-pushed and mode-agnostic — replacing the single 1h hardcoded timeout in existing lazy mode with configurable, tunable per-account settings. - **Per-peer effective-mode visibility in the client**: the daemon exposes both the configured and the server-resolved effective mode per peer in status/debug output, so when a connection misbehaves the question becomes "what effective mode did this peer receive?" rather than "which combination of flags happened to win?". For mixed-version fleets this materially improves debuggability. - **Legacy-client compatibility** is explicit: older clients are capability-gated via `SupportedFeatures` and downgraded to existing `p2p-lazy` via `LegacyLazyFallback`. Existing accounts and clients aren't affected unless they explicitly enroll in the new mode label. No in-place semantic drift on existing modes. - **Hardware-tested**: cutover and live mode-switching tested without disconnects across 32 connected peers spanning 12 OpenWrt versions (22.03 - 25.12), Windows 10/11, Debian 13, Android 12/14, iOS 26.3.1. Mixed Kernel + Userspace WireGuard. Mode toggles (`p2p-lazy` ↔ `p2p` via API) did not interrupt active sessions. The reasoning for modeling this as a new opt-in label instead of changing existing modes is in [my 2026-05-07 reply](https://github.com/netbirdio/netbird/issues/5989#issuecomment-4401310875) — I won't repeat it here, but the short version is: each peer stays in exactly one labeled mode, capability-gated for backwards compatibility, with a single effective resolution per peer that the client surfaces in status. That keeps mixed-version operation predictable. Since the 2026-05-07 reply, the work has continued: - **Phase-3.7i orphan-disconnect**: built on top of the foundation stack — addresses peers that never receive an explicit `lastActive` update and could stay stuck in inactivity tracking. - **Phase-3.7j GO_IDLE receive-side handling**: field-deployed across the same fleet plus Windows 11 and Android 16 builds. - **One concrete bug surfaced and was fixed**: a remote `GO_IDLE` from a legacy lazy-mode peer (e.g. v0.51.2) could trigger an unintended ICE-detach + guard-retry-exhaustion cycle on newer clients, leaving the tunnel relay-only until the next manual reconnect. - **Standalone PR offer**: if useful, that specific receive-side fix could be a small isolated PR against current `main`. Thanks for taking the time. Michael
saavagebueno commented 2026-08-05 01:29:41 -04:00
Author
Owner
Copy Link

@RMTT commented on GitHub (Jun 24, 2026):

Is possible to make connection-mode a policy option? For example, when adding policy from source peer A to destination peer B, use relay-forced.

<!-- gh-comment-id:4791000591 --> @RMTT commented on GitHub (Jun 24, 2026): Is possible to make connection-mode a policy option? For example, when adding policy from source peer A to destination peer B, use `relay-forced`.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2021 Q4
2021 Q4
2022 Q1
2022 Q1
2022 Q1
2022 Q1
2022 Q1
2022 Q1
accessibility
accessibility
accessibility
acl
acl
acl
agent
agent
agent
agent
agent
agent
Android
Android
Android
Android
Android
Android
api
api
api
authentik
authentik
authentik
automation
automation
automation
azure
azure
azure
battery-usage
battery-usage
battery-usage
bug
cache
cache
cache
client
client
client
client-ui
client-ui
client-ui
cloud
cloud
cloud
cloud-only
cloud-only
cloud-only
cloudflare
cloudflare
cloudflare
community
community
community
compatibility
compatibility
compatibility
config-idp
config-idp
config-idp
config-issue
config-issue
config-issue
connection
connection
connection
contribution
contribution
contribution
coturn
coturn
coturn
cross-vpn
cross-vpn
cross-vpn
dashboard
dashboard
dashboard
data-usage
data-usage
data-usage
distribution
distribution
distribution
dns
dns
dns
docker
docker
docker
documentation
documentation
documentation
duplicate
duplicate
duplicate
enhancement
enhancement
event-stream
event-stream
event-stream
feature-request
feature-request
feature-request
freebsd
freebsd
freebsd
getting-started
getting-started
getting-started
go
go
go
good first issue
good first issue
good first issue
gui
gui
gui
help wanted
help wanted
help wanted
home-assistant
home-assistant
home-assistant
idp
idp
idp
inconsistency
inconsistency
inconsistency
integration
integration
integration
integrations
integrations
integrations
ios
ios
ios
ipv6
ipv6
ipv6
jwt
jwt
jwt
k8s
k8s
k8s
keycloak
keycloak
keycloak
linux
linux
linux
login
login
login
macos
macos
macos
management-service
management-service
management-service
Medium
Medium
Medium
missing-docs
missing-docs
missing-docs
mobile
mobile
mobile
moved-internal
moved-internal
moved-internal
needs-review
needs-review
needs-review
netbird-ui
netbird-ui
netbird-ui
networking
networking
networking
new-platform
new-platform
new-platform
nginx
nginx
nginx
notification
notification
notification
okta
okta
okta
openwrt
openwrt
openwrt
P2
P2
P2
packaging
packaging
packaging
peer-management
peer-management
peer-management
peer-management
peer-management
peer-management
peer-management
peer-management
peer-management
performance
performance
performance
postgres
postgres
postgres
posture-checks
posture-checks
posture-checks
psk
psk
psk
pull-request
Mirrored from GitHub Pull Request
 question
question
question
refactor
refactor
refactor
relay
relay
relay
release
release
release
rfc
rfc
rfc
routes
routes
routes
security
security
security
security-improvement
security-improvement
security-improvement
security-related
security-related
security-related
self-hosting
self-hosting
self-hosting
server
server
server
signal
signal
signal
sleep-issue
sleep-issue
sleep-issue
ssh
ssh
ssh
ssl
ssl
ssl
status
status
status
store
store
store
synology
synology
synology
system-compatibility-issue
system-compatibility-issue
system-compatibility-issue
test-suite
test-suite
test-suite
third-party-integration
third-party-integration
third-party-integration
triage
triage
triage
triage
triage
triage
triage-needed
triage-needed
triage-needed
troubleshooting
troubleshooting
troubleshooting
UX
UX
UX
waiting-feedback
waiting-feedback
waiting-feedback
windows
windows
windows
wontfix
wontfix
wontfix
zitadel
zitadel
zitadel
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dynr-servers saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#11429
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?