[PR #6209] [management] NetworkMap components migration to client #27749

Open
opened 2026-08-05 07:09:10 -04:00 by saavagebueno · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/netbirdio/netbird/pull/6209
Author: @crn4
Created: 5/19/2026
Status: 🔄 Open

Base: mainHead: nmap/components-impl


📝 Commits (10+)

  • 2de0283 init int inds migration
  • 4543780 grpc components encoding with optimisations
  • b194af4 wire size benches fix
  • 9083bdb capabilities conditioning
  • b9a0186 fix routes filtering in account componnents
  • 672b057 fix Group.Copy losing AccountSeqID
  • 9bbbafa int id for networks and posture checks migration
  • 582cd70 client side and components on shared folder
  • 728057e missed files for client side and shared files
  • 3a1bbeb review comments

📊 Changes

70 files changed (+9507 additions, -853 deletions)

View changed files

📝 .github/workflows/golangci-lint.yml (+2 -2)
📝 client/internal/engine.go (+67 -3)
📝 dns/nameserver.go (+3 -0)
📝 idp/dex/config.go (+1 -1)
📝 idp/dex/provider.go (+1 -2)
idp/dex/sqlite_cgo.go (+15 -0)
idp/dex/sqlite_nocgo.go (+15 -0)
📝 management/internals/controllers/network_map/controller/controller.go (+104 -10)
📝 management/internals/controllers/network_map/interface.go (+4 -0)
📝 management/internals/controllers/network_map/interface_mock.go (+33 -0)
management/internals/shared/grpc/components_encoder.go (+813 -0)
management/internals/shared/grpc/components_encoder_test.go (+879 -0)
management/internals/shared/grpc/components_envelope_response.go (+192 -0)
management/internals/shared/grpc/components_envelope_response_test.go (+184 -0)
📝 management/internals/shared/grpc/conversion.go (+8 -281)
📝 management/internals/shared/grpc/conversion_test.go (+6 -5)
📝 management/internals/shared/grpc/server.go (+25 -1)
📝 management/server/account.go (+8 -0)
📝 management/server/account_test.go (+10 -0)
📝 management/server/group.go (+20 -1)

...and 50 more files

📄 Description

Describe your changes

NetworkMap components migration. Server emits a compact
NetworkMapEnvelope instead of an expanded proto.NetworkMap for peers that
advertise PeerCapabilityComponentNetworkMap; capable clients decode it,
run Calculate() locally, and merge an optional ProxyPatch. Legacy peers
and the kill-switch path keep ToSyncResponse unchanged.

What's in the envelope

  • Every ID-reference on the wire is uint32 (peer index or per-account
    AccountSeqID). No xid strings travel. The client synthesises consistent
    string IDs (p_<idx>, g_<seq>, pol_<seq>, r_<seq>, nres_<seq>,
    pc_<seq>, net_<seq>, nsg_<seq>).
  • New shared package shared/management/networkmap holds the encode helpers
    (extracted from conversion.go), the decoder, and the client pipeline
    EnvelopeToNetworkMap.
  • Server-side helper ToComponentSyncResponse wraps the envelope into a
    SyncResponse; computeSSHEnabledForPeer mirrors both Calculate()
    SSH-activation branches (explicit NetbirdSSH + legacy implicit TCP/22)
    so PeerConfig.SshConfig.SshEnabled stays consistent with the legacy path.

Stack

Checklist

  • Is it a bug fix
  • Is a typo/documentation fix
  • Is a feature enhancement
  • It is a refactor
  • Created tests that fail without the change (if possible)

By submitting this pull request, you confirm that you have read and agree to the terms of the Contributor License Agreement.

Documentation

Select exactly one:

  • I added/updated documentation for this change
  • Documentation is not needed for this change (explain why)

technical refactor

Docs PR URL (required if "docs added" is checked)

Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

Summary by CodeRabbit

  • New Features

    • Added component-based network map encoding for more efficient peer synchronization
    • Implemented NetworkMapEnvelope protocol format with compact wire encoding using sequence IDs
    • Added server-side kill-switch control for component network maps via environment variable
  • Refactor

    • Consolidated DNS and route conversion utilities into shared networkmap module
    • Unified peer network map response handling to support both component and legacy formats
  • Tests

    • Added comprehensive test coverage for component encoding/decoding and round-trip validation
    • Added store-level tests for sequence ID allocation and entity persistence
    • Added wire format benchmarks and breakdown diagnostics

Review Change Stack


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/netbirdio/netbird/pull/6209 **Author:** [@crn4](https://github.com/crn4) **Created:** 5/19/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `nmap/components-impl` --- ### 📝 Commits (10+) - [`2de0283`](https://github.com/netbirdio/netbird/commit/2de02839715f2bae0c7833595064a37ebb69f6c7) init int inds migration - [`4543780`](https://github.com/netbirdio/netbird/commit/4543780ef0b194bd4f33d874502e46df7b1611cc) grpc components encoding with optimisations - [`b194af4`](https://github.com/netbirdio/netbird/commit/b194af48b81a73c7ae38f41b75ae9792181dcf56) wire size benches fix - [`9083bdb`](https://github.com/netbirdio/netbird/commit/9083bdb97758d890ab89661383ceb23d0d44cfc4) capabilities conditioning - [`b9a0186`](https://github.com/netbirdio/netbird/commit/b9a0186200227d3d0b92f7af0d81e7ca8095c950) fix routes filtering in account componnents - [`672b057`](https://github.com/netbirdio/netbird/commit/672b057aa00bd080ee1749f0a57431e6baac0f37) fix Group.Copy losing AccountSeqID - [`9bbbafa`](https://github.com/netbirdio/netbird/commit/9bbbafaf69cb9e763e943c084785e3626a096221) int id for networks and posture checks migration - [`582cd70`](https://github.com/netbirdio/netbird/commit/582cd700861520ac7d135d053fd1d27f1885a0fa) client side and components on shared folder - [`728057e`](https://github.com/netbirdio/netbird/commit/728057ef15f5a34e22ea03028846c10529213aac) missed files for client side and shared files - [`3a1bbeb`](https://github.com/netbirdio/netbird/commit/3a1bbeba906ed913cd8c382b4004ce069812131f) review comments ### 📊 Changes **70 files changed** (+9507 additions, -853 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/golangci-lint.yml` (+2 -2) 📝 `client/internal/engine.go` (+67 -3) 📝 `dns/nameserver.go` (+3 -0) 📝 `idp/dex/config.go` (+1 -1) 📝 `idp/dex/provider.go` (+1 -2) ➕ `idp/dex/sqlite_cgo.go` (+15 -0) ➕ `idp/dex/sqlite_nocgo.go` (+15 -0) 📝 `management/internals/controllers/network_map/controller/controller.go` (+104 -10) 📝 `management/internals/controllers/network_map/interface.go` (+4 -0) 📝 `management/internals/controllers/network_map/interface_mock.go` (+33 -0) ➕ `management/internals/shared/grpc/components_encoder.go` (+813 -0) ➕ `management/internals/shared/grpc/components_encoder_test.go` (+879 -0) ➕ `management/internals/shared/grpc/components_envelope_response.go` (+192 -0) ➕ `management/internals/shared/grpc/components_envelope_response_test.go` (+184 -0) 📝 `management/internals/shared/grpc/conversion.go` (+8 -281) 📝 `management/internals/shared/grpc/conversion_test.go` (+6 -5) 📝 `management/internals/shared/grpc/server.go` (+25 -1) 📝 `management/server/account.go` (+8 -0) 📝 `management/server/account_test.go` (+10 -0) 📝 `management/server/group.go` (+20 -1) _...and 50 more files_ </details> ### 📄 Description ## Describe your changes NetworkMap components migration. Server emits a compact `NetworkMapEnvelope` instead of an expanded `proto.NetworkMap` for peers that advertise `PeerCapabilityComponentNetworkMap`; capable clients decode it, run `Calculate()` locally, and merge an optional `ProxyPatch`. Legacy peers and the kill-switch path keep `ToSyncResponse` unchanged. ### What's in the envelope - Every ID-reference on the wire is `uint32` (peer index or per-account `AccountSeqID`). No `xid` strings travel. The client synthesises consistent string IDs (`p_<idx>`, `g_<seq>`, `pol_<seq>`, `r_<seq>`, `nres_<seq>`, `pc_<seq>`, `net_<seq>`, `nsg_<seq>`). - New shared package `shared/management/networkmap` holds the encode helpers (extracted from `conversion.go`), the decoder, and the client pipeline `EnvelopeToNetworkMap`. - Server-side helper `ToComponentSyncResponse` wraps the envelope into a `SyncResponse`; `computeSSHEnabledForPeer` mirrors both `Calculate()` SSH-activation branches (explicit `NetbirdSSH` + legacy implicit TCP/22) so `PeerConfig.SshConfig.SshEnabled` stays consistent with the legacy path. ## Issue ticket number and link ## Stack <!-- branch-stack --> ### Checklist - [ ] Is it a bug fix - [ ] Is a typo/documentation fix - [x] Is a feature enhancement - [x] It is a refactor - [ ] Created tests that fail without the change (if possible) > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md). ## Documentation Select exactly one: - [ ] I added/updated documentation for this change - [x] Documentation is **not needed** for this change (explain why) technical refactor ### Docs PR URL (required if "docs added" is checked) Paste the PR link from https://github.com/netbirdio/docs here: https://github.com/netbirdio/docs/pull/__ <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added component-based network map encoding for more efficient peer synchronization * Implemented NetworkMapEnvelope protocol format with compact wire encoding using sequence IDs * Added server-side kill-switch control for component network maps via environment variable * **Refactor** * Consolidated DNS and route conversion utilities into shared networkmap module * Unified peer network map response handling to support both component and legacy formats * **Tests** * Added comprehensive test coverage for component encoding/decoding and round-trip validation * Added store-level tests for sequence ID allocation and entity persistence * Added wire format benchmarks and breakdown diagnostics <!-- review_stack_entry_start --> [![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/netbirdio/netbird/pull/6209?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
saavagebueno added the pull-request label 2026-08-05 07:09:10 -04:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#27749