[PR #5943] [CLOSED] [management] Cached serial check on sync #28792

New Issue

saavagebueno · 2026-08-05T08:06:57-04:00

saavagebueno commented

2026-08-05 08:06:57 -04:00

📋 Pull Request Information

Original PR: https://github.com/netbirdio/netbird/pull/5943
Author: @mlsmaycon
Created: 4/21/2026
Status: ❌ Closed

Base: main ← Head: cached-serial-check-on-sync

📝 Commits (10+)

3f4ef00 [management] Skip full network map on Sync when peer state is unchanged
8430b06 [management] Add Redis-backed kill switch for Sync fast path
5d58000 Merge branch 'main' into cached-serial-check-on-sync
3716838 Remove unused cacheKey helper and testcontainers imports, simplify Redis container setup
48c080b Replace Redis dependency with a generic cache store for fast path flag handling
93391fc generate only current.bin and android_current.bin on ci/cd
3eb1298 Refactor sync fast path tests and fix CI flakiness
46446ac Add detailed timing logs to sync fast path operations
66494d6 Replace Tracef with Debugf for sync fast path logging
dc86c96 Improve timing precision in sync fast path logging

📊 Changes

37 files changed (+2448 additions, -52 deletions)

View changed files

📝 .github/workflows/golang-test-linux.yml (+4 -1)
📝 client/cmd/testutil_test.go (+1 -1)
📝 client/internal/engine_test.go (+1 -1)
📝 client/server/server_test.go (+1 -1)
📝 management/internals/controllers/network_map/controller/controller.go (+11 -2)
📝 management/internals/controllers/network_map/interface.go (+5 -0)
📝 management/internals/controllers/network_map/interface_mock.go (+17 -2)
📝 management/internals/server/boot.go (+3 -1)
➕ management/internals/shared/fastpathcache/invalidator.go (+48 -0)
➕ management/internals/shared/grpc/fast_path_caches.go (+122 -0)
➕ management/internals/shared/grpc/fast_path_flag.go (+131 -0)
➕ management/internals/shared/grpc/fast_path_flag_test.go (+176 -0)
➕ management/internals/shared/grpc/peer_serial_cache.go (+82 -0)
➕ management/internals/shared/grpc/peer_serial_cache_decision_test.go (+116 -0)
➕ management/internals/shared/grpc/peer_serial_cache_test.go (+134 -0)
📝 management/internals/shared/grpc/server.go (+91 -28)
➕ management/internals/shared/grpc/sync_fast_path.go (+562 -0)
➕ management/internals/shared/grpc/sync_fast_path_response_test.go (+163 -0)
📝 management/server/account.go (+16 -0)
📝 management/server/account/manager.go (+3 -0)

...and 17 more files

📄 Description

Describe your changes

[management] Cached serial check on Sync

Summary

Adds a Redis-gated fast path to the management gRPC Sync handler that lets
returning peers skip full NetworkMap computation when their client-side
state already matches the account's current serial. When the fast path
applies, the peer receives a lightweight SyncResponse containing only a
fresh NetbirdConfig (TURN + Relay tokens) — mirroring what
TimeBasedAuthSecretsManager already pushes on token refresh — and relies on
its existing in-memory state.

No proto changes. No client changes. The optimisation is gated behind a
runtime kill switch so it can be rolled out gradually and disabled
cluster-wide without a redeploy.

Why

Every Sync today runs the full network-map computation even when nothing
has changed since the peer last connected. For a large account with many
peers reconnecting (e.g. after a management restart, a flaky network event,
or a routine client reconnect), that is a lot of wasted computation — all to
deliver the same bytes the peer already has.

Observations that made this safe to short-circuit:

Account state mutations bump Network.Serial via IncrementNetworkSerial.
NetworkMap.Serial on the last delivered map is already a monotonic
witness of what the peer has.
Peer metadata changes (hostname, kernel, IP) already hash into
metaHash(peerMeta, realIP) — so a meta change is independently observable
without peeking into the peer object.
TimeBasedAuthSecretsManager already demonstrates that sending
NetbirdConfig-only responses mid-stream is supported by every historical
client (it's how token refreshes work).

Design

Fast-path gate

The fast path runs only when all of the following hold:

Shared peerSerialCache is wired (nil cache disables it).
FastPathFlag.Enabled() reports true (Redis-gated kill switch).
The peer is not Android. Android's GrpcClient.GetNetworkMap errors on
a nil NetworkMap — we bail out to stay safe.
The cache holds an entry for this peer pubkey.
The cached Serial matches network.CurrentSerial().
The cached MetaHash matches the incoming metaHash(peerMeta, realIP).
The cached Serial is non-zero (guard against uninitialised entries).

Any false branch falls through to the existing slow path unchanged.

Response shape

buildFastPathResponse constructs a SyncResponse with only
NetbirdConfig populated — same shape as
TimeBasedAuthSecretsManager.pushNewTURNAndRelayTokens. It includes:

Fresh TURN credentials (when time-based TURN is configured).
Fresh Relay token (when Relay is configured).
ExtraSettings from the settings manager (best-effort; errors log at debug).
Signal config, STUN config, and any integration-extended config via
integrationsConfig.ExtendNetBirdConfig.

It omits NetworkMap, PeerConfig, Checks, and RemotePeers. The client
keeps whatever state it already has and just refreshes its control-plane
credentials.

Between the eligibility check (GetAccountNetwork → serial N) and the
peer's subscription (OnPeerConnected) there is a window where another
writer could commit serial N+1 and broadcast. A peer that subscribes after
the broadcast lands never receives the update for N+1 and ends up at stale
serial N with no NetworkMap.

commitFastPath closes this race:

MarkPeerConnected.
GetPeerByPeerPubKey.
OnPeerConnected → subscribe to the update channel.
Re-fetch GetAccountNetwork and compare to the initial serial.
If the serial advanced (or the re-fetch fails), call
cancelPeerRoutinesWithoutLock to tear down the subscription and return
committed=false so the caller falls through to the slow path — which
delivers the new full map.

Only if the re-check succeeds does runFastPathSync actually send the lean
response.

Cache lifecycle

Written by the server after a successful wire send — either at the
end of the slow-path sendInitialSync (recordPeerSyncEntry) or from the
per-update handler for NetworkMap-typed messages
(recordPeerSyncEntryFromUpdate).
Invalidated on every successful Login (invalidatePeerSyncEntry) so
state changes that happen through login (SSH key rotation, approval, user
binding) always yield a full map on the next Sync.
Expired by TTL (DefaultPeerSerialCacheTTL = 24h) — self-healing if a
Set is dropped, the entry simply times out and the next Sync is a slow path
that re-primes the cache.

Cache writes are best-effort: errors log at debug and do not fail the Sync.
Reads treat any error (including the miss sentinel) as a miss so the slow
path runs.

Runtime kill switch

FastPathFlag wraps an atomic.Bool populated by RunFastPathFlagRoutine,
which polls the shared cache store (same Redis used by the rest of
management, so no extra env var) every minute for key peerSyncFastPath.

Key value (trimmed)	Effect
`1`, `true`, `TRUE`, `True`, `" true "`	fast path enabled
`0`, `false`, `""`, missing key, `yes`, anything else	fast path disabled

Properties:

Fail-closed: any store read error (other than the NotFound miss) flips
the flag to disabled and logs at error level. A transient Redis outage
while enabled rolls back to the slow path for the affected replica until
Redis recovers.
No extra config: reuses BaseServer.CacheStore() — so the same
NB_CACHE_REDIS_ADDRESS that already wires the shared cache wires the
flag. Without Redis, the store falls back to in-process gocache which is
enough for single-replica dev/test.
Default off: a brand-new deployment with no key set stays on the slow
path until an operator sets the key.
Nil-safe: a nil *FastPathFlag (or one constructed by NewFastPathFlag(false))
reports disabled so tests and consumers can opt out entirely.

Ops flow

# Enable cluster-wide
redis-cli -u $NB_CACHE_REDIS_ADDRESS SET peerSyncFastPath 1

# Disable cluster-wide
redis-cli -u $NB_CACHE_REDIS_ADDRESS DEL peerSyncFastPath
# or
redis-cli -u $NB_CACHE_REDIS_ADDRESS SET peerSyncFastPath 0

Propagation latency is ≤ 1 minute per replica (the poll interval).

Scenarios

Scenario	Path	Rationale
Fresh peer, first Sync	slow	No cache entry → miss → slow path primes cache.
Returning peer, same serial, same meta, flag on	fast	All conditions met; lightweight response with fresh tokens.
Returning peer, same serial, same meta, flag off	slow	Kill switch bypasses the whole optimisation.
Returning peer, account serial advanced (another peer registered / policy change / etc.)	slow	`shouldSkipNetworkMap` fails on serial mismatch; full map delivered.
Returning peer, hostname / kernel / realIP changed	slow	`metaHash` mismatch; full map delivered so integrations see the new meta.
Returning peer, serial advances during the fast-path attempt	slow (fallback)	`commitFastPath` re-check detects the advance after subscribing; tears down, returns false, slow path runs.
Android peer (any state)	slow	Android client errors on nil `NetworkMap`; gate unconditionally skips.
Returning peer after `Login` call	slow	`invalidatePeerSyncEntry` removed the entry on Login; cache miss → slow path.
Cache write dropped on the previous Sync	slow	Next Sync misses the cache → slow path → re-primes.
Redis transient read failure while flag was enabled	slow	Flag fails closed to disabled for that replica; recovers on next successful read.
Redis completely unavailable (`NB_CACHE_REDIS_ADDRESS` unset)	slow	Shared store is in-process gocache; single-replica deployments can still toggle by setting the key programmatically in tests.
v0.20 / v0.40 / v0.60 legacy clients	slow (first Sync)	First Sync primes cache. v0.40 reconnects do hit the fast path today (documented tradeoff — v0.40's `GetNetworkMap` call is indistinguishable from a main Sync on the server); will tighten when a proto opt-in lands.

Compatibility

Wire format: no change. SyncResponse with NetbirdConfig set and
NetworkMap nil is already supported by every historical client because
that's what TimeBasedAuthSecretsManager pushes on TURN/Relay token
refresh.
Proto schema: no change. Future PR may add an opt-in flag so legacy
clients can be identified and forced to the slow path on reconnect; the
documented legacy-reconnect tradeoff above has a failing test ready to
tighten when that lands.
Android: explicitly skipped — GrpcClient.GetNetworkMap on Android
errors on a nil NetworkMap and was only fixed upstream in v0.50.0.

Testing

Unit

TestParseFastPathFlag — flag value parsing table.
TestFastPathFlag_{EnabledDefaultsFalse,NilSafeEnabled,SetEnabled} —
flag container behaviour.
TestRunFastPathFlagRoutine_{NilStoreStaysDisabled,ReadsFlagFromStore,MissingKeyKeepsDisabled,DefaultKeyUsedWhenEmpty,FailsClosedOnReadError}
— runtime routine behaviour, including the fail-closed flip via a
flakyStore wrapper that injects transient errors.
TestShouldSkipNetworkMap — 10-row table over the eligibility predicate.
TestPeerSerialCache_{GetSetDelete,GetMissReturnsZero,TTLExpiry,OverwriteUpdatesValue,IsolatedPerKey,Concurrent,Redis}
— cache wrapper behaviour over both gocache and a Redis testcontainer.
TestBuildFastPathResponse_* — response composition across
time-based-TURN, static-TURN, no-relay, and extra-settings-error branches.

Behavioural (in-process gRPC)

TestSyncFastPath_FirstSync_SendsFullMap — first Sync is always slow.
TestSyncFastPath_SecondSync_MatchingSerial_SkipsMap — happy path: lean
response, fresh tokens, no NetworkMap.
TestSyncFastPath_AndroidNeverSkips — Android gate holds even with primed
cache.
TestSyncFastPath_MetaChanged_SendsFullMap — meta hash change forces
slow path.
TestSyncFastPath_LoginInvalidatesCache — post-Login Sync hits the slow
path.
TestSyncFastPath_OtherPeerRegistered_ForcesFullMap — serial advance
caused by another peer's registration forces the slow path.

Legacy wire-format regression

testdata/sync_request_wire/ contains frozen SyncRequest bytes for
v0.20.0, v0.40.0, and v0.60.0. Tests replay each fixture through the
in-process server to assert:

TestSync_WireFixture_LegacyClients_AlwaysReceiveFullMap — first Sync
always delivers a full map.
TestSync_WireFixture_LegacyClient_ReconnectStillGetsFullMap — pins the
current accepted tradeoff where a v0.40 reconnect with a primed cache
does hit the fast path (test must be tightened when proto opt-in lands).
TestSync_WireFixture_AndroidReconnect_NeverSkips — android gate holds
at the wire level.
TestSync_WireFixture_ModernClientReconnect_TakesFastPath — baseline
fast-path hit over the real wire.

current.bin and android_current.bin are regenerated by CI
(go run ./management/server/testdata/sync_request_wire/generate.go) and
.gitignore'd; the three legacy fixtures are checked in and frozen.

Test hygiene

Fixed-sleep waitForPeerDisconnect replaced with a bounded
require.Eventually poll on peer.Status.Connected — no more timing
flakes under race detector.
Windows skip consolidated into a single skipOnWindows(t) helper and
applied uniformly to every fast-path + legacy-wire test.

Rollback

redis-cli -u $NB_CACHE_REDIS_ADDRESS DEL peerSyncFastPath — propagates
within 1 minute, every replica falls back to the slow path.
If a deeper rollback is needed, revert the PR; the code is additive and
its only integration point is the optional peerSerialCache + fastPathFlag
params in NewServer.

Issue ticket number and link

Stack

Checklist

Is it a bug fix
Is a typo/documentation fix
Is a feature enhancement
It is a refactor
Created tests that fail without the change (if possible)

By submitting this pull request, you confirm that you have read and agree to the terms of the Contributor License Agreement.

Documentation

Select exactly one:

I added/updated documentation for this change
Documentation is not needed for this change (explain why)

Docs PR URL (required if "docs added" is checked)

Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

Summary by CodeRabbit

Release Notes

New Features
- Added Sync fast path optimization that improves reconnection performance by delivering lightweight responses when network state remains unchanged.
- Implemented network state caching and feature-flag gating for optimized peer synchronization.
- Maintained full backward compatibility with legacy clients.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/netbirdio/netbird/pull/5943 **Author:** [@mlsmaycon](https://github.com/mlsmaycon) **Created:** 4/21/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `cached-serial-check-on-sync` --- ### 📝 Commits (10+) - [`3f4ef00`](https://github.com/netbirdio/netbird/commit/3f4ef0031b22aa0c742dd2b42209c1d0881750e2) [management] Skip full network map on Sync when peer state is unchanged - [`8430b06`](https://github.com/netbirdio/netbird/commit/8430b06f2a5eee339d4cf5a878ab7335a5f0e0a9) [management] Add Redis-backed kill switch for Sync fast path - [`5d58000`](https://github.com/netbirdio/netbird/commit/5d58000dbd5ee25c2183a6478e18e3d6a597c4c9) Merge branch 'main' into cached-serial-check-on-sync - [`3716838`](https://github.com/netbirdio/netbird/commit/3716838c25b2516bd235b0c80d3789cf5f7a75f7) Remove unused cacheKey helper and testcontainers imports, simplify Redis container setup - [`48c080b`](https://github.com/netbirdio/netbird/commit/48c080b8612fcf9b5aa1418ae73dff574d182f36) Replace Redis dependency with a generic cache store for fast path flag handling - [`93391fc`](https://github.com/netbirdio/netbird/commit/93391fc68fc78c8791e002f0ddb2d7da51a91594) generate only current.bin and android_current.bin on ci/cd - [`3eb1298`](https://github.com/netbirdio/netbird/commit/3eb1298cb4ff3d26233315ebd867ed154c17a4e3) Refactor sync fast path tests and fix CI flakiness - [`46446ac`](https://github.com/netbirdio/netbird/commit/46446acd301aef2a00ec507ee9c5940741afb353) Add detailed timing logs to sync fast path operations - [`66494d6`](https://github.com/netbirdio/netbird/commit/66494d61afbc2b105f18a74750532ec7e59bcf67) Replace Tracef with Debugf for sync fast path logging - [`dc86c96`](https://github.com/netbirdio/netbird/commit/dc86c9655d1d4ab11d9bc2243cc21e77a20eb110) Improve timing precision in sync fast path logging ### 📊 Changes **37 files changed** (+2448 additions, -52 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/golang-test-linux.yml` (+4 -1) 📝 `client/cmd/testutil_test.go` (+1 -1) 📝 `client/internal/engine_test.go` (+1 -1) 📝 `client/server/server_test.go` (+1 -1) 📝 `management/internals/controllers/network_map/controller/controller.go` (+11 -2) 📝 `management/internals/controllers/network_map/interface.go` (+5 -0) 📝 `management/internals/controllers/network_map/interface_mock.go` (+17 -2) 📝 `management/internals/server/boot.go` (+3 -1) ➕ `management/internals/shared/fastpathcache/invalidator.go` (+48 -0) ➕ `management/internals/shared/grpc/fast_path_caches.go` (+122 -0) ➕ `management/internals/shared/grpc/fast_path_flag.go` (+131 -0) ➕ `management/internals/shared/grpc/fast_path_flag_test.go` (+176 -0) ➕ `management/internals/shared/grpc/peer_serial_cache.go` (+82 -0) ➕ `management/internals/shared/grpc/peer_serial_cache_decision_test.go` (+116 -0) ➕ `management/internals/shared/grpc/peer_serial_cache_test.go` (+134 -0) 📝 `management/internals/shared/grpc/server.go` (+91 -28) ➕ `management/internals/shared/grpc/sync_fast_path.go` (+562 -0) ➕ `management/internals/shared/grpc/sync_fast_path_response_test.go` (+163 -0) 📝 `management/server/account.go` (+16 -0) 📝 `management/server/account/manager.go` (+3 -0) _...and 17 more files_ </details> ### 📄 Description ## Describe your changes # [management] Cached serial check on Sync ## Summary Adds a Redis-gated fast path to the management gRPC `Sync` handler that lets returning peers skip full `NetworkMap` computation when their client-side state already matches the account's current serial. When the fast path applies, the peer receives a lightweight `SyncResponse` containing only a fresh `NetbirdConfig` (TURN + Relay tokens) — mirroring what `TimeBasedAuthSecretsManager` already pushes on token refresh — and relies on its existing in-memory state. No proto changes. No client changes. The optimisation is gated behind a runtime kill switch so it can be rolled out gradually and disabled cluster-wide without a redeploy. ## Why Every `Sync` today runs the full network-map computation even when nothing has changed since the peer last connected. For a large account with many peers reconnecting (e.g. after a management restart, a flaky network event, or a routine client reconnect), that is a lot of wasted computation — all to deliver the same bytes the peer already has. Observations that made this safe to short-circuit: - Account state mutations bump `Network.Serial` via `IncrementNetworkSerial`. - `NetworkMap.Serial` on the last delivered map is already a monotonic witness of what the peer has. - Peer metadata changes (hostname, kernel, IP) already hash into `metaHash(peerMeta, realIP)` — so a meta change is independently observable without peeking into the peer object. - `TimeBasedAuthSecretsManager` already demonstrates that sending `NetbirdConfig`-only responses mid-stream is supported by every historical client (it's how token refreshes work). ## Design ### Fast-path gate The fast path runs only when **all** of the following hold: 1. Shared `peerSerialCache` is wired (nil cache disables it). 2. `FastPathFlag.Enabled()` reports true (Redis-gated kill switch). 3. The peer is **not** Android. Android's `GrpcClient.GetNetworkMap` errors on a nil `NetworkMap` — we bail out to stay safe. 4. The cache holds an entry for this peer pubkey. 5. The cached `Serial` matches `network.CurrentSerial()`. 6. The cached `MetaHash` matches the incoming `metaHash(peerMeta, realIP)`. 7. The cached `Serial` is non-zero (guard against uninitialised entries). Any `false` branch falls through to the existing slow path unchanged. ### Response shape `buildFastPathResponse` constructs a `SyncResponse` with only `NetbirdConfig` populated — same shape as `TimeBasedAuthSecretsManager.pushNewTURNAndRelayTokens`. It includes: - Fresh TURN credentials (when time-based TURN is configured). - Fresh Relay token (when Relay is configured). - `ExtraSettings` from the settings manager (best-effort; errors log at debug). - Signal config, STUN config, and any integration-extended config via `integrationsConfig.ExtendNetBirdConfig`. It omits `NetworkMap`, `PeerConfig`, `Checks`, and `RemotePeers`. The client keeps whatever state it already has and just refreshes its control-plane credentials. ### Race handling (re-check after subscribe) Between the eligibility check (`GetAccountNetwork` → serial N) and the peer's subscription (`OnPeerConnected`) there is a window where another writer could commit serial N+1 and broadcast. A peer that subscribes *after* the broadcast lands never receives the update for N+1 and ends up at stale serial N with no `NetworkMap`. `commitFastPath` closes this race: 1. `MarkPeerConnected`. 2. `GetPeerByPeerPubKey`. 3. `OnPeerConnected` → subscribe to the update channel. 4. Re-fetch `GetAccountNetwork` and compare to the initial serial. 5. If the serial advanced (or the re-fetch fails), call `cancelPeerRoutinesWithoutLock` to tear down the subscription and return `committed=false` so the caller falls through to the slow path — which delivers the new full map. Only if the re-check succeeds does `runFastPathSync` actually send the lean response. ### Cache lifecycle - **Written** by the server *after* a successful wire send — either at the end of the slow-path `sendInitialSync` (`recordPeerSyncEntry`) or from the per-update handler for `NetworkMap`-typed messages (`recordPeerSyncEntryFromUpdate`). - **Invalidated** on every successful `Login` (`invalidatePeerSyncEntry`) so state changes that happen through login (SSH key rotation, approval, user binding) always yield a full map on the next Sync. - **Expired** by TTL (`DefaultPeerSerialCacheTTL = 24h`) — self-healing if a Set is dropped, the entry simply times out and the next Sync is a slow path that re-primes the cache. Cache writes are best-effort: errors log at debug and do not fail the Sync. Reads treat any error (including the miss sentinel) as a miss so the slow path runs. ### Runtime kill switch `FastPathFlag` wraps an `atomic.Bool` populated by `RunFastPathFlagRoutine`, which polls the shared cache store (same Redis used by the rest of management, so no extra env var) every minute for key `peerSyncFastPath`. | Key value (trimmed) | Effect | | --- | --- | | `1`, `true`, `TRUE`, `True`, `" true "` | fast path **enabled** | | `0`, `false`, `""`, missing key, `yes`, anything else | fast path **disabled** | Properties: - **Fail-closed**: any store read error (other than the NotFound miss) flips the flag to disabled and logs at error level. A transient Redis outage while enabled rolls back to the slow path for the affected replica until Redis recovers. - **No extra config**: reuses `BaseServer.CacheStore()` — so the same `NB_CACHE_REDIS_ADDRESS` that already wires the shared cache wires the flag. Without Redis, the store falls back to in-process gocache which is enough for single-replica dev/test. - **Default off**: a brand-new deployment with no key set stays on the slow path until an operator sets the key. - **Nil-safe**: a nil `*FastPathFlag` (or one constructed by `NewFastPathFlag(false)`) reports disabled so tests and consumers can opt out entirely. #### Ops flow ```sh # Enable cluster-wide redis-cli -u $NB_CACHE_REDIS_ADDRESS SET peerSyncFastPath 1 # Disable cluster-wide redis-cli -u $NB_CACHE_REDIS_ADDRESS DEL peerSyncFastPath # or redis-cli -u $NB_CACHE_REDIS_ADDRESS SET peerSyncFastPath 0 ``` Propagation latency is ≤ 1 minute per replica (the poll interval). ## Scenarios | Scenario | Path | Rationale | | --- | --- | --- | | Fresh peer, first Sync | slow | No cache entry → miss → slow path primes cache. | | Returning peer, same serial, same meta, flag on | fast | All conditions met; lightweight response with fresh tokens. | | Returning peer, same serial, same meta, flag off | slow | Kill switch bypasses the whole optimisation. | | Returning peer, account serial advanced (another peer registered / policy change / etc.) | slow | `shouldSkipNetworkMap` fails on serial mismatch; full map delivered. | | Returning peer, hostname / kernel / realIP changed | slow | `metaHash` mismatch; full map delivered so integrations see the new meta. | | Returning peer, serial advances *during* the fast-path attempt | slow (fallback) | `commitFastPath` re-check detects the advance after subscribing; tears down, returns false, slow path runs. | | Android peer (any state) | slow | Android client errors on nil `NetworkMap`; gate unconditionally skips. | | Returning peer after `Login` call | slow | `invalidatePeerSyncEntry` removed the entry on Login; cache miss → slow path. | | Cache write dropped on the previous Sync | slow | Next Sync misses the cache → slow path → re-primes. | | Redis transient read failure while flag was enabled | slow | Flag fails closed to disabled for that replica; recovers on next successful read. | | Redis completely unavailable (`NB_CACHE_REDIS_ADDRESS` unset) | slow | Shared store is in-process gocache; single-replica deployments can still toggle by setting the key programmatically in tests. | | v0.20 / v0.40 / v0.60 legacy clients | slow (first Sync) | First Sync primes cache. v0.40 reconnects *do* hit the fast path today (documented tradeoff — v0.40's `GetNetworkMap` call is indistinguishable from a main Sync on the server); will tighten when a proto opt-in lands. | ## Compatibility - **Wire format**: no change. `SyncResponse` with `NetbirdConfig` set and `NetworkMap` nil is already supported by every historical client because that's what `TimeBasedAuthSecretsManager` pushes on TURN/Relay token refresh. - **Proto schema**: no change. Future PR may add an opt-in flag so legacy clients can be identified and forced to the slow path on reconnect; the documented legacy-reconnect tradeoff above has a failing test ready to tighten when that lands. - **Android**: explicitly skipped — `GrpcClient.GetNetworkMap` on Android errors on a nil `NetworkMap` and was only fixed upstream in v0.50.0. ## Testing ### Unit - `TestParseFastPathFlag` — flag value parsing table. - `TestFastPathFlag_{EnabledDefaultsFalse,NilSafeEnabled,SetEnabled}` — flag container behaviour. - `TestRunFastPathFlagRoutine_{NilStoreStaysDisabled,ReadsFlagFromStore,MissingKeyKeepsDisabled,DefaultKeyUsedWhenEmpty,FailsClosedOnReadError}` — runtime routine behaviour, including the fail-closed flip via a `flakyStore` wrapper that injects transient errors. - `TestShouldSkipNetworkMap` — 10-row table over the eligibility predicate. - `TestPeerSerialCache_{GetSetDelete,GetMissReturnsZero,TTLExpiry,OverwriteUpdatesValue,IsolatedPerKey,Concurrent,Redis}` — cache wrapper behaviour over both gocache and a Redis testcontainer. - `TestBuildFastPathResponse_*` — response composition across time-based-TURN, static-TURN, no-relay, and extra-settings-error branches. ### Behavioural (in-process gRPC) - `TestSyncFastPath_FirstSync_SendsFullMap` — first Sync is always slow. - `TestSyncFastPath_SecondSync_MatchingSerial_SkipsMap` — happy path: lean response, fresh tokens, no `NetworkMap`. - `TestSyncFastPath_AndroidNeverSkips` — Android gate holds even with primed cache. - `TestSyncFastPath_MetaChanged_SendsFullMap` — meta hash change forces slow path. - `TestSyncFastPath_LoginInvalidatesCache` — post-Login Sync hits the slow path. - `TestSyncFastPath_OtherPeerRegistered_ForcesFullMap` — serial advance caused by another peer's registration forces the slow path. ### Legacy wire-format regression `testdata/sync_request_wire/` contains frozen `SyncRequest` bytes for v0.20.0, v0.40.0, and v0.60.0. Tests replay each fixture through the in-process server to assert: - `TestSync_WireFixture_LegacyClients_AlwaysReceiveFullMap` — first Sync always delivers a full map. - `TestSync_WireFixture_LegacyClient_ReconnectStillGetsFullMap` — pins the current accepted tradeoff where a v0.40 reconnect with a primed cache *does* hit the fast path (test must be tightened when proto opt-in lands). - `TestSync_WireFixture_AndroidReconnect_NeverSkips` — android gate holds at the wire level. - `TestSync_WireFixture_ModernClientReconnect_TakesFastPath` — baseline fast-path hit over the real wire. `current.bin` and `android_current.bin` are regenerated by CI (`go run ./management/server/testdata/sync_request_wire/generate.go`) and `.gitignore`'d; the three legacy fixtures are checked in and frozen. ### Test hygiene - Fixed-sleep `waitForPeerDisconnect` replaced with a bounded `require.Eventually` poll on `peer.Status.Connected` — no more timing flakes under race detector. - Windows skip consolidated into a single `skipOnWindows(t)` helper and applied uniformly to every fast-path + legacy-wire test. ## Rollback 1. `redis-cli -u $NB_CACHE_REDIS_ADDRESS DEL peerSyncFastPath` — propagates within 1 minute, every replica falls back to the slow path. 2. If a deeper rollback is needed, revert the PR; the code is additive and its only integration point is the optional `peerSerialCache` + `fastPathFlag` params in `NewServer`. ## Issue ticket number and link ## Stack  ### Checklist - [ ] Is it a bug fix - [ ] Is a typo/documentation fix - [x] Is a feature enhancement - [ ] It is a refactor - [ ] Created tests that fail without the change (if possible) > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md). ## Documentation Select exactly one: - [ ] I added/updated documentation for this change - [x] Documentation is **not needed** for this change (explain why) ### Docs PR URL (required if "docs added" is checked) Paste the PR link from https://github.com/netbirdio/docs here: https://github.com/netbirdio/docs/pull/__  ## Summary by CodeRabbit ## Release Notes * **New Features** * Added Sync fast path optimization that improves reconnection performance by delivering lightweight responses when network state remains unchanged. * Implemented network state caching and feature-flag gating for optimized peer synchronization. * Maintained full backward compatibility with legacy clients.  --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

saavagebueno added the pull-request label 2026-08-05 08:06:57 -04:00

Sign in to join this conversation.

Branches Tags

main

claude/agent-network-test-cases-p743vw

android/gui-integration

revert/component-types

feat-post_quantum_ml_kem

ice-stun-wg-demux

dependabot/npm_and_yarn/proxy/web/npm_and_yarn-b39864987c

agent-network-setup-poc

dependabot/go_modules/aws-sdk-8f849ebaed

dependabot/github_actions/actions-a940c7c866

dependabot/go_modules/otel-e34c790afd

dependabot/go_modules/testcontainers-de325c0dd6

dependabot/go_modules/wireguard-dbd6b95108

dependabot/go_modules/pion-5f703e1eca

dependabot/go_modules/gorm-2271c8195b

fix-login-needed-check

dependabot/go_modules/google.golang.org/grpc-1.82.1

fix/ui-gtk3-support

enterprise-traefik-and-migration-fixes

disambiguate_p2p_metrics

revert/component-types-hookup

embedded-vnc

feature/ios-ssh

docs/agent-network-docs-update

dependabot/go_modules/github.com/aws/aws-sdk-go-v2/service/s3-1.106.3

dependabot/go_modules/github.com/pion/stun/v3-3.1.5

fix-ssh-authorized-users-multi-rule

peer-acl-multi-source

reverse-proxy-crowdsec-appsec

reverse-proxy-allow-match-or

client-local-metrics

lazy-conn-per-peer

lazy-conn-rosenpass

dependabot/go_modules/github.com/gopacket/gopacket-1.7.0

dependabot/go_modules/goauthentik.io/api/v3-3.2026050.6

dependabot/go_modules/github.com/pires/go-proxyproto-0.15.0

dependabot/go_modules/github.com/jackc/pgx/v5-5.10.0

dependabot/go_modules/github.com/oapi-codegen/runtime-1.6.0

dependabot/go_modules/github.com/eko/gocache/lib/v4-4.2.4

dependabot/go_modules/github.com/pkg/sftp-1.13.11

dependabot/go_modules/github.com/coreos/go-oidc/v3-3.20.0

ssh-windows-privilege-check

fix/explicit-cors-handling

fix/remove-math-rand

test/gui-memory-leak-fix

fix/ui-status-dispatch

install-script-ui-dependencies

fix/grpc-get-network-map

fix/subscribe-status-coalesce

fix/tray-menu-item-leak

fix/windows-tray-race

feature/changeset

worktree-dns-route-qtype-fallthrough

mdm_integration

mlsmaycon-patch-2

feat/agent-network-ollama

proxy-tunnel-cache-ttl-env

coderabbitai/utg/1e5b0a5

grpc-acl

test/battery-drain

components-impl-drop-indexes-use-xids-no-resource-policy-map

fix/nmap-relevant-groups

e2e-guardrail-blocks-unselected-model

fix/lazyconn-cold-start-allowed-ips

vertex-guardrails-model-access-e2e

add-atomic-cache-ops

refactor/relay-foreign-cache

ci/trigger-release-tests

feature/kimi-3-agent-networks-dns-warmup

feature/dns-lazy-conn-warmup

daemon-ipc-acl

feature/ui-translation-key-parity-check

refactor/relay-foreign-cache-tests

fix/lazyconn-idle-keep-wg-peer

dmitri-propagate-auth-grant-types-on-combined

0.74.7-branch

diagnose-empty-vs-corrupt-state

windows-sleep-detector

fix/cli-up-wait-for-daemon

rp_key_persistency

feature/native-grpc

0.74.6-branch

0.74.6-branch-sync

0.74.5-branch-sync

0.74.4-branch

fix/remove-stale-peers-removal

fix/remove-stale-proxy-logic

fix/nsis-preserve-autostart-on-upgrade

refactor/peer-event-bus

dependabot/go_modules/goauthentik.io/api/v3-3.2026050.3

dependabot/go_modules/github.com/pkg/sftp-1.13.10

components-impl-drop-indexes

fix-reset-aggregation-window-flake

dependabot/go_modules/github.com/pion/dtls/v3-3.1.5

0.74.x

fix/relay-states-lock

update-process-pkg

update-gopsutil-v4

fix/relay_states_lock

increase-sysinfo-timeout

dependabot/go_modules/github.com/Azure/go-ntlmssp-0.1.1

dependabot/go_modules/github.com/eko/gocache/store/redis/v4-4.2.6

dependabot/go_modules/github.com/eko/gocache/lib/v4-4.2.3

fix/fail-to-create-upnp-port-mapping-on-opnsense-firewall

0.74.3-branch

fix/routeselector-atomic-exit-node

netmap_progressive_alignment

nmap/components-impl

dependabot/go_modules/github.com/jackc/pgx/v5-5.9.2

dependabot/go_modules/github.com/oapi-codegen/runtime-1.4.2

dependabot/go_modules/github.com/gopacket/gopacket-1.6.1

dependabot/go_modules/github.com/coreos/go-oidc/v3-3.19.0

dependabot/go_modules/github.com/pires/go-proxyproto-0.12.0

fix/signal-watchdog-sync-stop

docs/agent-network

test/affected-logic

fix/revert-ice-filter

refactor/simplify-affected-peers

pascal-filter-policies-by-direction

claude/lock-contention-peer-connect-g8t6au

dmitri-filter-policies-by-direction

refactor/migrate-profiles-to-go

profile-bindings-ios

fix/skip-restart-unchanged-route

fix/mgmt-cache-async-resolve

refactor/wails-update-105

client_lifetime_serialization_refactor

fix/browser-ssh-2

fix/ipv6-and-netstack-accept-loop

fix/browser-ssh

profile-id-name-test

refactor/mgmt-bootstrap

feat/getting-started-unified-wizard

socket-grpc-permissions

fix/mysql-index-migration

windows-dns-firewall

tests/enable-race-on-tests

ui-refactor-gtk3

feature/affected-peers-grpc

profile-id

lazyconn-first-packet-fix-v2

claude/focused-gates-VMTgb

ui-tray-linux-leftclick

fix/ctx-enrichment

daemon-owner

feature/android-client-ssh

worktree-accept-ra-forwarding

nmap/combined-deploy

task/align_protobuff_toolset

feature/session-extend

add-json-yaml-flags

refactor/ephemeral-cleanup

claude/webtransport-relay-wasm-mUjY9

claude/vnc-udp-feasibility-6KB1U

fix/wgport-config

e2e-windows-dns-combined

fix/login-cmd-root-flags

feat/reseller-openapi-spec

github-issue-resolver

add-steamos-support

fix-darwin-uninstaller

flutter-test

ci/freebsd-pkg-bootstrap

cached-serial-check-on-sync

fix-mgmt-cache-bypass-overlay

revert-easyjson-5938

revert-ice-5820

revert-firewalld-5928

refactor/permissions-manager

revert-dns-5935-systemd-resolved

revert-dns-5935-5945

revert-dns-5945-mgmt-cache

feature/log-most-busy-peers

prototype/ui-wails

coderabbitai/utg/8ae8f20

feature/use-peer-fqdn-on-https

release/0.68.3

add-slack-channel

claude/rdp-token-passthrough-eNcqW

transparent-proxy

fix/macos-stale-route-eexist

crowdsec-selfhosted

fix/remove-otel-units

entire/checkpoints/v1

fix/getting-started

feat/static-connectors-combined-server

feature/use-local-keys-embedded

feature/fleetdm

set-env-only-if-not-fork

feature/expose-has-channel

fix/connection-status-race

fix/filter-cgnat-cni-ice-candidates

feature/check-cert-locker-before-acme

test/proxy-fixes

test/proxy-mtu

prototype/ui-tauri

test/proxy-speed

fix-reused-ports

feat/migrate-to-embedded-idp

feature/add-serial-to-proxy-merged

deploy/proxy-serial

test/connection

feature/disable-legacy-port

feature/flag-to-disable-legacy-port

test/perftest

fix/http-redirect

poc-token-command

dn-reverse-proxy

prototype/reverse-proxy-rename

prototype/reverse-proxy-logs-pagination

feature/client-metrics

prototype/reverse-proxy-clusters

debug-dns-route

fix/win-dns-batch

add-extra-route-logs

job-stream-notify-disconnection-eof

deploy/secrets-manager

trigger-proxy-update

bug/update-ios-client-code-build-tags

sync-client-netmap-serial

log/conn-disconn

nmap/compaction-deploy

ci-win-test

feature/disk-encryption-check

wasm-debug

swap-dns-prio

fix/dex-config

feature/migrate-auto-groups-to-table

nmap/compaction

dex-nocgo-stub

feature/exclude-terraform-from-rate-limiting

test-freebsd

retries-refactor

coderabbitai/docstrings/b7e98ac

feat/integrate-zitadel

bug/ios-hanging-reconection

zitadel-idp

feat/network-map-serial

refactor/get-account-no-users

feat/auto-upgrade

feature/report-high-pat-id

feature/temporary-access-for-resource

fix/nmap-fwrules

dont-restart-dns

prototype/ui

update-gomobile

go-dns-for-ice

wasm-ldflags

test-ldflags

wasmbuild-test

feature/networks-s2s

vk/compare-nmaps

dbg/bothmaps

reorder-dns-shutdown

fix/relay-reconnection-race

fix/nmap-exitnodes

vk/debug/nmap-both

move-licensed-code

feat/better-daemon-connection-lost-message

feat/auto-update-2

test/timings

refactor/getaccount-raw

tests/nmap-getaccount

refactor/nmap

refactor/nmap-limit-buffer

feature/detect-mac-wakeup

feature/extract-modules

quick-setings

feat/sync-limiter

feature/store-cache-impl

fix-install-version

feature/store-metrics

feature/metrics-on-store

feature/use-gorm-cache

loadtest-signal

unsymmetrical-squash

refactor/reducate-signaling

test/update-reduce

feature/store-cache

feature/remote-debug

cli-ws-proxy-backend-addr

feat/mgmt-map-serial

snyk-fix-d9d0081a4c7f9137bdb59d0d50a141a2

snyk-fix-7415cea5a11acd66753540ca2c598c63

job-yml-update

feature/android-allow-selecting-routes

fix/up-sequence

fix/dns-hash-update

snyk-fix-967adae9863f17f108ce8948d9117b8d

log/getaccount-by-peer

signal-suppressor

dns-exit-node

feature/auto-updates

feature/cache-srv-key

merged-fixes

fix/missed-offers-and-debug

debug-and-fixes

poc-wasm-clean-backend-s2s

test/remote-debug

debug-api

fix/remove-gpo-if-empty

fix/test-freebsd

fix/mysql-setup

fix/remove-logout-btn

handle-existing-domain-user

chore/unify-domain-validation

snyk-fix-c5fafc8a50ce1f29046e25a1fc346185

feat/profile-edit-btn

snyk-fix-a54966211e18d4cf67e5a2757cc006d1

log-short-id

feat/logout-ephemeral

log-checks

batch-wg-ops

nb-interface-default

feat/aws-integration

add/race-test

feature/relay-feature-versioning

fix/systemd-service-logs

poc/preprocessed-map

add-account-onboarding

bind-ipv6

fix/merge-main

logs/peerlogs-addpeer

feature/net-297-network-migration

feature/support-skip-auto-apply-exit-node-routes

set-cmd

set-command-with-cursor

feature/limit-update-channel

stop-using-locking-share

feature/poc-lazy-detection

feature/net-248-removal-of-sync-mutex-locks

test/multiple-peer-logging

preresolve

add-ns-punnycode-support

apply-routes-early

windows-search-domains

fix/connecting-route-filter

feature/management/rest-client/impersonate

debug-local-records

resource-fields-snake-case

test/grpc-rate-limit

traffic-correlation-policy

feature/rest-client-options

feat/events-metrics

feature/buf-cli

test/add-ratelimiter

test/remove-write-lock-on-add-peer

fix/add-peer-semaphore

feature/users-roles-endpoint

mlsmaycon-patch-1

debug-user-role

chore/primary-key-on-networks

feature/update-account-peers-buffer-startup

remove-ubuntu2004-runners

refactor/permissions-no-pat-allowed

ref/logrus-factory

use-conntrack-zone

deploy/permissions-account

feature/lazy-connection-idle

ref/improve-test-cov

restore-pr-3440

test/increase-grpc-timeouts

feat/buffer-account-peers-update

test/networkmapgeneration-changes

feature/base-manager

feature/flow-receiver

chore/benchmark-with-large-runner

refactor/handshake-initiator

client/ui-update-systray-icons

userspace-router

wgwatcher-test

output-if-key-already-exists

fix/relay-reconnection

feature/port-forwarding-client-codecleaning

detached2

test/callbacks-nil-iceconninfo

refactor/optimize-peer-expiration

enable-udp-port-for-docker-template

fix/relay-update

feature/apply-posture-netmap

fix/group-update-existing-resource

conntrack-stats

upgrade-okta-sdk

multi-price

test/conn-stat

set-min-parallel-tests-for-management

dns-interceptor

debug-dns

router-dns

add-static-system-info

debug-0.29.4

debug-0.33.0

account-refactoring

relay/2800_quic

route-get-account-refactoring

test/seed-random-routes

feature/get-account-refactoring

test/reconnect-race-condition

refactor/get-account-usage

feature/add-session-id-to-update-channel

improve-ipv4conn

fix/async-pion-event-handling

debug

add-offload

feature/validate-group-association-debug

fix/limit-conn-for-sqlite

test/engine-iface

test/transaction-for-jwt-sync

fix/engine-stop-in-foreground

feature/add-mysql-support

test-migration

refactor/header-size-values

relay/eliminate-gob

test/signal-dispatcher-with-relay

relay/debug

validate-icon

feature/ipv6-support

use-pre-expanded-peers-map

feature/use-signal-dispatcher

validate/peer-status

add-read-write-times

fix/sync-peer-race

feature/relay-status

netmap

evaluate/network-map-hash

fix/lower-dns-resolve-interval-on-fail

feature/relay

fix/go-mod-version

upgrade-nftables

synology-userspace-mode

fix/use-ip-for-default-routes-on-darwin

fix/proxy_close

enable-release-workflow-on-pr

deploy/peer-performance

feature/permanent-turn

feature/permanent-turn-proxy

deploy/posture-check-sqlite

feature/optimize_sqlite_save

debug-ios-behavior

fix/delete-route-only-after-adding

tshoot/windows-logger

remove-new-routing

refactor/eliminate-repo-dependency

add-arm-to-ci

refactor-demo-account-object

test/abc2

test/abc

send-ssh-rosenpass-config-meta

refactor-demo

ensure-schedule-never-runs-non-positive

feature/peer-validator-groupmgm

feature/peer-validator-fix

fix/include-active-dashboard-users

fix/handle-canceling-schedule

fix/geo-download

debug-google-workspace

yury/resolve-ip-to-location

feature/extend-sysinfo

sqlite-async-peer-status

yury/add-postgresql-store

fix/route

test-build

posture-checks-poc

debug-keycloak-idp

poc/netstack

for-pascal-tmp

peer-logout-management

manual-peer-logout

detached

chore/refactor-management

test/dns-bind

fix/enforce-acl-for-containers

yury/use-sync-map-in-updatechannel

fix/events-key-handling

filter-cache-on-load-account

fix/user-expiration

handle-user-context-cancellation

nb-client-k8s-statefulset

fake-addr

fix/iptables_in_docker

ebpf-debug

update-getting-started-flow-use-postgres

fix/peer_list_notification

feature/device-authentication-with-client-secret

feature/keep_alive

feat-groups-from-jwt

separate_proxy_from_wgconfig

fix/wg_conn

wg_conn_fix

wg_bind_parallel_processing

fix-rollback-get-acls

proxy_cfg_cleanup

performance-improvement-rego

update-lock-log-level

feat-client-side-acl

refactor/move_grpcserver_logic_to_account_manager

feature/event-storage

feature/update-idp-redeeming-invite

feature/api-peer-info

return-groupminimum-setupkey

feature/interface-bind

documentation_enhancement

fix-peer-registration

ssh

users_cache

pass-client-caller

client_caller_type

revert-283-feat-fix-windows-installer

periodic-peer-updates

ebpf

braginini/wasm

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: DYNR/netbird#28792