Compare commits

...

74 Commits

Author SHA1 Message Date
Hugo Hakim Damer
8b0398c0db Add support for IPv6 networks (on Linux clients) (#1459)
* Feat add basic support for IPv6 networks

Newly generated networks automatically generate an IPv6 prefix of size
64 within the ULA address range, devices obtain a randomly generated
address within this prefix.

Currently, this is Linux only and does not yet support all features
(routes currently cause an error).

* Fix firewall configuration for IPv6 networks

* Fix routing configuration for IPv6 networks

* Feat provide info on IPv6 support for specific client to mgmt server

* Feat allow configuration of IPv6 support through API, improve stability

* Feat add IPv6 support to new firewall implementation

* Fix peer list item response not containing IPv6 address

* Fix nftables breaking on IPv6 address change

* Fix build issues for non-linux systems

* Fix intermittent disconnections when IPv6 is enabled

* Fix test issues and make some minor revisions

* Fix some more testing issues

* Fix more CI issues due to IPv6

* Fix more testing issues

* Add inheritance of IPv6 enablement status from groups

* Fix IPv6 events not having associated messages

* Address first review comments regarding IPv6 support

* Fix IPv6 table being created even when IPv6 is disabled

Also improved stability of IPv6 route and firewall handling on client side

* Fix IPv6 routes not being removed

* Fix DNS IPv6 issues, limit IPv6 nameservers to IPv6 peers

* Improve code for IPv6 DNS server selection, add AAAA custom records

* Ensure IPv6 routes can only exist for IPv6 routing peers

* Fix IPv6 network generation randomness

* Fix a bunch of compilation issues and test failures

* Replace method calls that are unavailable in Go 1.21

* Fix nil dereference in cleanUpDefaultForwardRules6

* Fix nil pointer dereference when persisting IPv6 network in sqlite

* Clean up of client-side code changes for IPv6

* Fix nil dereference in rule mangling and compilation issues

* Add a bunch of client-side test cases for IPv6

* Fix IPv6 tests running on unsupported environments

* Fix import cycle in tests

* Add missing method SupportsIPv6() for windows

* Require IPv6 default route for IPv6 tests

* Fix panics in routemanager tests on non-linux

* Fix some more route manager tests concerning IPv6

* Add some final client-side tests

* Add IPv6 tests for management code, small fixes

* Fix linting issues

* Fix small test suite issues

* Fix linter issues and builds on macOS and Windows again

* fix builds for iOS because of IPv6 breakage
2024-08-13 17:26:27 +02:00
Gabriel Górski
4da29451d0 Add missing openid scope when requesting JWT token (#2089)
According to the Zitadel documentation, `openid` scope is required
when requesting JWT tokens.

Apparently Zitadel was accepting requests without it until very
recently. Now lack thereof causes 400 Bad Requests which makes it
impossible to authenticate to the Netbird dashboard.

https://zitadel.com/docs/guides/integrate/service-users/client-credentials#2-authenticating-a-service-user-and-request-a-token
2024-06-04 10:46:24 +02:00
Viktor Liu
9b3449753e Ignore candidates whose IP falls into a routed network. (#2084)
This will prevent peer connections via other peers.
2024-06-03 17:31:37 +02:00
Maycon Santos
456629811b Prevent using expired ctx when sending metrics (#2088) 2024-06-03 12:41:15 +02:00
Zoltan Papp
c311d0d19e Fill the UI version info in system meta on Android (#2077) 2024-05-31 17:26:56 +02:00
pascal-fischer
521f7dd39f Improve login performance (#2061) 2024-05-31 16:41:12 +02:00
pascal-fischer
f9ec0a9a2e Fix PKCE auth html (#2079) 2024-05-30 17:22:58 +02:00
pascal-fischer
012235ff12 Add FindExistingPostureCheck (#2075) 2024-05-30 15:22:42 +02:00
Maycon Santos
f176807ebe Add extra logs for account not found, peer login and getAccount (#2053) 2024-05-27 12:29:28 +02:00
Maycon Santos
d4c47eaf8a Don't allow delete group from peer groups (#2055) 2024-05-27 11:06:43 +02:00
Bethuel Mmbaga
d35a79d3b5 Upgrade gRPC and OpenTelemetry packages for compatibility (#2003)
Upgrades `go.opentelemetry.io/otel` from version` v1.11.1` to `v1.26.0`. The upgrade addresses compatibility issues caused by the removal of several sub-packages in the latest OpenTelemetry release, which were causing broken dependencies.

**Key Changes:**
- Upgraded `go.opentelemetry.io/otel` from `v1.11.1` to `v1.26.0`.

- Fixed broken dependencies by replacing the deprecated sub-packages:
  - `go.opentelemetry.io/otel/metric/instrument`
  - `go.opentelemetry.io/otel/metric/instrument/asyncint64`
  - `go.opentelemetry.io/otel/metric/instrument/syncint64`
  
- Upgraded `google.golang.org/grpc` from `v1.56.3`  to `v1.64.0` which deprecate `Dial` and `DialContext` to `NewClient`.
2024-05-27 08:39:18 +02:00
Maycon Santos
6a2929011d Refactor firewall manager check (#2054)
Some systems don't play nice with a test chain
So we dropped the idea, and instead we check for the filter table

With this check, we might face a case where iptables is selected once and on the 
next netbird up/down it will go back to using nftables
2024-05-27 08:37:32 +02:00
Maycon Santos
e877c9d6c1 Update CODE_OF_CONDUCT.md (#2048) 2024-05-24 17:29:14 +02:00
Maycon Santos
7a1c96ebf4 Remove extra error mapping (#2050) 2024-05-24 14:46:11 +02:00
Zoltan Papp
41fe9f84ec Extend integrated validator with error handling (#2044) 2024-05-24 13:29:25 +02:00
Viktor Liu
d13fb0e379 Restore netbird state and log level after debug (#2047) 2024-05-24 13:27:41 +02:00
Maycon Santos
f3214527ea Use info log-level for firewall manager discover (#2045)
* Use info log-level for firewall manager discover

* Update client/firewall/create_linux.go

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>

---------

Co-authored-by: Viktor Liu <17948409+lixmal@users.noreply.github.com>
2024-05-24 13:03:19 +02:00
Maycon Santos
69048bfd34 Revert "Accept any XDG_ environment variable to determine desktop (#2037)" (#2042)
This reverts commit 67e2185964.
2024-05-23 23:15:02 +02:00
Maycon Santos
29a2d93873 Log global lock acquisition per user (#2039) 2024-05-23 17:09:58 +02:00
Maycon Santos
6b01b0020e Enhance firewall manager checks to detect unsupported iptables (#2038)
Our nftables firewall manager may cause issues when rules are created using older iptable versions
2024-05-23 16:09:51 +02:00
Maycon Santos
9d3db68805 Return the proper error when a peer is deleted (#2035)
this fixes an issue causing peers to keep retrying the connection after a peer is removed from the management system
2024-05-23 14:59:09 +02:00
Maycon Santos
2e315311e0 Fix the initial daemon retry interval (#2036) 2024-05-23 14:52:52 +02:00
Maycon Santos
67e2185964 Accept any XDG_ environment variable to determine desktop (#2037) 2024-05-23 12:34:19 +02:00
Maycon Santos
89149dc6f4 Increase the status checks timeout (#2033)
Some systems might respond with a small delay depending on various factors. Increasing the timeout to reduce the number of false-positive reports
2024-05-23 10:54:01 +02:00
Matthew R Kasun
5a1f8f13a2 use the next available port for wireguard (#2024)
check if WgPort is available, if not find the next free port
2024-05-22 18:42:56 +02:00
Viktor Liu
e71059d245 Add dummy ipv6 to macos interface (#2025) 2024-05-22 12:32:01 +02:00
Maycon Santos
91fa2e20a0 Store location information in peer event meta (#1994) 2024-05-22 12:31:16 +02:00
Zoltan Papp
61034aaf4d Gracefully conn worker shutdown (#2022)
Because the connWorker are operating with the e.peerConns list we must ensure all workers exited before we modify the content of the e.peerConns list.
If we do not do that the engine will start new connWorkers for the exists ones, and they start connection for the same peers in parallel.
2024-05-22 11:15:29 +02:00
Maycon Santos
b8717b8956 Update the GUI status when daemon unavailable (#2012)
in case we got no status we mark the GUI app as disconnected
2024-05-21 15:45:49 +02:00
pascal-fischer
50201d63c2 Increase garbage collection on ios (#1981) 2024-05-17 15:58:29 +02:00
pascal-fischer
d11b39282b Enable namserver deactivation if unresponsive on iOS (#1982) 2024-05-17 12:59:46 +02:00
Viktor Liu
bd58eea8ea Refactor network monitor to wait for stop (#1992) 2024-05-17 09:43:18 +02:00
Bethuel Mmbaga
a5811a2d7d Implement experimental PostgreSQL store (#1939)
* migrate sqlite store to
 generic sql store

* fix conflicts

* init postgres store

* Add postgres store tests

* Refactor postgres store engine name

* fix tests

* Run postgres store tests on linux only

* fix tests

* Refactor

* cascade policy rules on policy deletion

* fix tests

* run postgres cases in new db

* close store connection after tests

* refactor

* using testcontainers

* sync go sum

* remove postgres service

* remove store cleanup

* go mod tidy

* remove env

* use postgres as engine and initialize test store with testcontainer

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-05-16 19:28:37 +03:00
Bethuel Mmbaga
a680f80ed9 Add installer support for Synology (#1984)
* add installer support for the synology

* skip ui installation for Synology

* Fix conflicts
2024-05-15 19:03:49 +03:00
Thorleif Jacobsen
10fbdc2c4a CentOS installations might have "apt" as "annotation processing tool", fixed so it checks for apt-get (#1955) 2024-05-15 16:33:12 +02:00
Viktor Liu
1444fbe104 Don't cancel proxy ctx on conn close (#1986) 2024-05-15 09:10:57 +02:00
Maycon Santos
650bca7ca8 Fix lost root zone handler (#1975)
When there is a connection issue with the
 root zone upstream we remove it from the
 dns mux, and we need to add it again
2024-05-13 18:11:08 +02:00
Ishan Arora
570e28d227 Fix typo in systemd .service files (#1972) 2024-05-13 11:40:57 +02:00
pascal-fischer
272ade07a8 Add route selection to iOS (#1944) 2024-05-10 10:47:16 +02:00
Bethuel Mmbaga
263abe4862 Fix windows route exec path (#1946)
* Enable release workflow on PR and upload binaries

 add GetSystem32Command to validate if a command is in the path

it will fall back to the full system32, assuming the OS driver is C

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-05-09 13:48:15 +02:00
Krzysztof Nazarewski
ceee421a05 unify Config generation, loading and updating (#1586)
* config.go: pull unified Config.apply() out of createNewConfig() and update()

as a bonus it ensures returned Config object doesn't have any configuration
values missing
2024-05-08 18:58:31 +02:00
pascal-fischer
0a75da6fb7 Remove GetNetworkMap stacktrace(#1941) 2024-05-07 19:19:30 +02:00
Viktor Liu
920877964f Monitor network changes and restart engine on detection (#1904) 2024-05-07 18:50:34 +02:00
pascal-fischer
2e0047daea Improve Sync performance (#1901) 2024-05-07 14:30:03 +02:00
Bethuel Mmbaga
ce0718fcb5 Migrate blob net ip fields to json serializer (#1906)
* serialize net.IP as json

* migrate net ip field from blob to json

* run net ip migration

* remove duplicate index

* Refactor

* Add tests

* fix tests

* migrate null blob values
2024-05-07 14:01:45 +03:00
Zoltan Papp
c590518e0c Feature/exit node Android (#1916)
Support exit node on Android.
With the protect socket function, we mark every connection that should be used out of VPN.
2024-05-07 12:28:30 +02:00
Carlos Hernandez
f309b120cd Retry reading routing table (bsd) (#1914)
* Retry reading routing table (bsd)

Similar to #1817, BSD base OSes will return "cannot allocate memory"
errors when routing table is expanding.
2024-05-07 09:51:43 +02:00
Maycon Santos
7357a9954c Fix a panic when management is behind an invalid proxy (#1930)
- Add a new error on gRPC client that doesn't pass the incorrect status from the gRPC client
- Try login only if we have a server public key
2024-05-06 18:04:32 +02:00
Zoltan Papp
13b63eebc1 Remove comments from iptables commands (#1928) 2024-05-06 17:12:34 +02:00
Zoltan Papp
735ed7ab34 Fix resolv.conf repairer logic (#1931)
Stop the file repairer before doing the restore
2024-05-06 17:01:00 +02:00
Carlos Hernandez
961d9198ef Fix removeAllowedIP (#1913)
Current implementation of removeAllowedIP recreates the wg iface,
killing all open ports and connections. This is due to that "lines" is
the complete output of `get` from wg-usp and not the specific interface
which changes should be applied to.
2024-05-06 15:33:08 +02:00
Misha Bragin
df4ca01848 Return system serial on a peer HTTP API call (#1929) 2024-05-06 14:49:03 +02:00
Viktor Liu
4e7c17756c Refactor Route IDs (#1891) 2024-05-06 14:47:49 +02:00
Viktor Liu
6a4935139d Ignore cloned routes on bsd (#1915) 2024-05-02 23:12:59 +02:00
pascal-fischer
35dd991776 Fix best route selection (#1903)
* fix route comparison to current selected route + adding tests

* add comment and debug log

* adjust log message

---------

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
2024-05-02 11:51:03 +02:00
Maycon Santos
3598418206 Update the check interval for new geo db and change log level (#1908)
Update log level to trace and update the check db interval from 60s to 300s
2024-04-30 17:54:29 +02:00
Viktor Liu
e435e39158 Fix route selection IDs (#1890) 2024-04-29 18:43:14 +02:00
Maycon Santos
fd26e989e3 Check if channel exist before sending network map (#1894)
Check for connection channel before calculating and sending the network map
2024-04-29 18:31:52 +02:00
Viktor Liu
4424162bce Add client debug features (#1884)
* Add status anonymization
* Add OS/arch to the status command
* Use human-friendly last-update status messages
* Add debug bundle command to collect (anonymized) logs
* Add debug log level command
* And debug for a certain time span command
2024-04-26 17:20:10 +02:00
Viktor Liu
54b045d9ca Replaces powershell with the route command and cache route lookups on windows (#1880) 2024-04-26 16:37:27 +02:00
Bethuel Mmbaga
71c6437bab add content type before writing header (#1887) 2024-04-25 21:20:24 +02:00
pascal-fischer
7b254cb966 add methods to manage rosenpass settings for iOS (#1879) 2024-04-23 19:26:03 +02:00
pascal-fischer
8f3a0f2c38 Add retry to IdP cache lookup (#1882) 2024-04-23 19:23:43 +02:00
pascal-fischer
1f33e2e003 Support exit nodes on iOS (#1878) 2024-04-23 19:12:16 +02:00
pascal-fischer
1e6addaa65 Add account locks to getAccountWithAuthorizationClaims method (#1847) 2024-04-23 19:09:58 +02:00
Viktor Liu
f51dc13f8c Add route selection functionality for CLI and GUI (#1865) 2024-04-23 14:42:53 +02:00
dependabot[bot]
3477108ce7 Bump golang.org/x/net from 0.20.0 to 0.23.0 (#1867)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.20.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-23 12:48:25 +02:00
Maycon Santos
012e624296 Fix DNS not found query response (#1877)
for local queries, we should return NXDOMAIN instead of NOERROR

Also, updated gomobile for Android and iOS builds
2024-04-23 10:20:09 +02:00
Maycon Santos
4c5e987e02 Add support for GUI app to display error (#1844) 2024-04-22 11:57:38 +02:00
Maycon Santos
a80c8b0176 Redeem invite only when incoming user was invited (#1861)
checks for users with pending invite status in the cache that already logged in and refresh the cache
2024-04-22 11:10:27 +02:00
Misha Bragin
9e01155d2e Add new intro image 2024-04-22 11:00:52 +02:00
Maycon Santos
3c3111ad01 Copy client binary to a directory in path (#1842) 2024-04-22 10:14:07 +02:00
Misha Bragin
b74078fd95 Use a better way to insert data in batches (#1874) 2024-04-20 22:04:20 +02:00
Viktor Liu
77488ad11a Migrate serializer:gob fields to serializer:json (#1855) 2024-04-18 18:14:21 +02:00
230 changed files with 13401 additions and 3435 deletions

View File

@@ -15,7 +15,7 @@ jobs:
strategy:
matrix:
arch: [ '386','amd64' ]
store: [ 'jsonfile', 'sqlite' ]
store: [ 'jsonfile', 'sqlite', 'postgres']
runs-on: ubuntu-latest
steps:
- name: Install Go

View File

@@ -19,7 +19,7 @@ jobs:
- name: codespell
uses: codespell-project/actions-codespell@v2
with:
ignore_words_list: erro,clienta
ignore_words_list: erro,clienta,hastable,
skip: go.mod,go.sum
only_warn: 1
golangci:

View File

@@ -38,7 +38,7 @@ jobs:
- name: Setup NDK
run: /usr/local/lib/android/sdk/cmdline-tools/7.0/bin/sdkmanager --install "ndk;23.1.7779620"
- name: install gomobile
run: go install golang.org/x/mobile/cmd/gomobile@v0.0.0-20230531173138-3c911d8e3eda
run: go install golang.org/x/mobile/cmd/gomobile@v0.0.0-20240404231514-09dbf07665ed
- name: gomobile init
run: gomobile init
- name: build android netbird lib
@@ -56,10 +56,10 @@ jobs:
with:
go-version: "1.21.x"
- name: install gomobile
run: go install golang.org/x/mobile/cmd/gomobile@v0.0.0-20230531173138-3c911d8e3eda
run: go install golang.org/x/mobile/cmd/gomobile@v0.0.0-20240404231514-09dbf07665ed
- name: gomobile init
run: gomobile init
- name: build iOS netbird lib
run: PATH=$PATH:$(go env GOPATH) gomobile bind -target=ios -bundleid=io.netbird.framework -ldflags="-X github.com/netbirdio/netbird/version.version=buildtest" -o $GITHUB_WORKSPACE/NetBirdSDK.xcframework $GITHUB_WORKSPACE/client/ios/NetBirdSDK
run: PATH=$PATH:$(go env GOPATH) gomobile bind -target=ios -bundleid=io.netbird.framework -ldflags="-X github.com/netbirdio/netbird/version.version=buildtest" -o ./NetBirdSDK.xcframework ./client/ios/NetBirdSDK
env:
CGO_ENABLED: 0

View File

@@ -7,17 +7,7 @@ on:
branches:
- main
pull_request:
paths:
- 'go.mod'
- 'go.sum'
- '.goreleaser.yml'
- '.goreleaser_ui.yaml'
- '.goreleaser_ui_darwin.yaml'
- '.github/workflows/release.yml'
- 'release_files/**'
- '**/Dockerfile'
- '**/Dockerfile.*'
- 'client/ui/**'
env:
SIGN_PIPE_VER: "v0.0.11"
@@ -106,6 +96,27 @@ jobs:
name: release
path: dist/
retention-days: 3
-
name: upload linux packages
uses: actions/upload-artifact@v3
with:
name: linux-packages
path: dist/netbird_linux**
retention-days: 3
-
name: upload windows packages
uses: actions/upload-artifact@v3
with:
name: windows-packages
path: dist/netbird_windows**
retention-days: 3
-
name: upload macos packages
uses: actions/upload-artifact@v3
with:
name: macos-packages
path: dist/netbird_darwin**
retention-days: 3
release_ui:
runs-on: ubuntu-latest

View File

@@ -130,3 +130,10 @@ issues:
- path: mock\.go
linters:
- nilnil
# Exclude specific deprecation warnings for grpc methods
- linters:
- staticcheck
text: "grpc.DialContext is deprecated"
- linters:
- staticcheck
text: "grpc.WithBlock is deprecated"

View File

@@ -5,7 +5,7 @@
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
identity and expression, level of experience, education, socioeconomic status,
nationality, personal appearance, race, caste, color, religion, or sexual
identity and orientation.

View File

@@ -44,7 +44,8 @@
### Open-Source Network Security in a Single Platform
![image](https://github.com/netbirdio/netbird/assets/700848/c0d7bae4-3301-499a-bb4e-5e4a225bf35f)
![netbird_2](https://github.com/netbirdio/netbird/assets/700848/46bc3b73-508d-4a0e-bb9a-f465d68646ab)
### Key features

View File

@@ -1,5 +1,5 @@
FROM alpine:3.18.5
RUN apk add --no-cache ca-certificates iptables ip6tables
ENV NB_FOREGROUND_MODE=true
ENTRYPOINT [ "/go/bin/netbird","up"]
COPY netbird /go/bin/netbird
ENTRYPOINT [ "/usr/local/bin/netbird","up"]
COPY netbird /usr/local/bin/netbird

View File

@@ -1,3 +1,5 @@
//go:build android
package android
import (
@@ -14,6 +16,7 @@ import (
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/formatter"
"github.com/netbirdio/netbird/iface"
"github.com/netbirdio/netbird/util/net"
)
// ConnectionListener export internal Listener for mobile
@@ -54,14 +57,17 @@ type Client struct {
ctxCancel context.CancelFunc
ctxCancelLock *sync.Mutex
deviceName string
uiVersion string
networkChangeListener listener.NetworkChangeListener
}
// NewClient instantiate a new Client
func NewClient(cfgFile, deviceName string, tunAdapter TunAdapter, iFaceDiscover IFaceDiscover, networkChangeListener NetworkChangeListener) *Client {
func NewClient(cfgFile, deviceName string, uiVersion string, tunAdapter TunAdapter, iFaceDiscover IFaceDiscover, networkChangeListener NetworkChangeListener) *Client {
net.SetAndroidProtectSocketFn(tunAdapter.ProtectSocket)
return &Client{
cfgFile: cfgFile,
deviceName: deviceName,
uiVersion: uiVersion,
tunAdapter: tunAdapter,
iFaceDiscover: iFaceDiscover,
recorder: peer.NewRecorder(""),
@@ -84,6 +90,9 @@ func (c *Client) Run(urlOpener URLOpener, dns *DNSList, dnsReadyListener DnsRead
var ctx context.Context
//nolint
ctxWithValues := context.WithValue(context.Background(), system.DeviceNameCtxKey, c.deviceName)
//nolint
ctxWithValues = context.WithValue(ctxWithValues, system.UiVersionCtxKey, c.uiVersion)
c.ctxCancelLock.Lock()
ctx, c.ctxCancel = context.WithCancel(ctxWithValues)
defer c.ctxCancel()
@@ -97,7 +106,8 @@ func (c *Client) Run(urlOpener URLOpener, dns *DNSList, dnsReadyListener DnsRead
// todo do not throw error in case of cancelled context
ctx = internal.CtxInitState(ctx)
return internal.RunClientMobile(ctx, cfg, c.recorder, c.tunAdapter, c.iFaceDiscover, c.networkChangeListener, dns.items, dnsReadyListener)
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder)
return connectClient.RunOnAndroid(c.tunAdapter, c.iFaceDiscover, c.networkChangeListener, dns.items, dnsReadyListener)
}
// RunWithoutLogin we apply this type of run function when the backed has been started without UI (i.e. after reboot).
@@ -122,7 +132,8 @@ func (c *Client) RunWithoutLogin(dns *DNSList, dnsReadyListener DnsReadyListener
// todo do not throw error in case of cancelled context
ctx = internal.CtxInitState(ctx)
return internal.RunClientMobile(ctx, cfg, c.recorder, c.tunAdapter, c.iFaceDiscover, c.networkChangeListener, dns.items, dnsReadyListener)
connectClient := internal.NewConnectClient(ctx, cfg, c.recorder)
return connectClient.RunOnAndroid(c.tunAdapter, c.iFaceDiscover, c.networkChangeListener, dns.items, dnsReadyListener)
}
// Stop the internal client and free the resources

View File

@@ -0,0 +1,212 @@
package anonymize
import (
"crypto/rand"
"fmt"
"math/big"
"net"
"net/netip"
"net/url"
"regexp"
"slices"
"strings"
)
type Anonymizer struct {
ipAnonymizer map[netip.Addr]netip.Addr
domainAnonymizer map[string]string
currentAnonIPv4 netip.Addr
currentAnonIPv6 netip.Addr
startAnonIPv4 netip.Addr
startAnonIPv6 netip.Addr
}
func DefaultAddresses() (netip.Addr, netip.Addr) {
// 192.51.100.0, 100::
return netip.AddrFrom4([4]byte{198, 51, 100, 0}), netip.AddrFrom16([16]byte{0x01})
}
func NewAnonymizer(startIPv4, startIPv6 netip.Addr) *Anonymizer {
return &Anonymizer{
ipAnonymizer: map[netip.Addr]netip.Addr{},
domainAnonymizer: map[string]string{},
currentAnonIPv4: startIPv4,
currentAnonIPv6: startIPv6,
startAnonIPv4: startIPv4,
startAnonIPv6: startIPv6,
}
}
func (a *Anonymizer) AnonymizeIP(ip netip.Addr) netip.Addr {
if ip.IsLoopback() ||
ip.IsLinkLocalUnicast() ||
ip.IsLinkLocalMulticast() ||
ip.IsInterfaceLocalMulticast() ||
ip.IsPrivate() ||
ip.IsUnspecified() ||
ip.IsMulticast() ||
isWellKnown(ip) ||
a.isInAnonymizedRange(ip) {
return ip
}
if _, ok := a.ipAnonymizer[ip]; !ok {
if ip.Is4() {
a.ipAnonymizer[ip] = a.currentAnonIPv4
a.currentAnonIPv4 = a.currentAnonIPv4.Next()
} else {
a.ipAnonymizer[ip] = a.currentAnonIPv6
a.currentAnonIPv6 = a.currentAnonIPv6.Next()
}
}
return a.ipAnonymizer[ip]
}
// isInAnonymizedRange checks if an IP is within the range of already assigned anonymized IPs
func (a *Anonymizer) isInAnonymizedRange(ip netip.Addr) bool {
if ip.Is4() && ip.Compare(a.startAnonIPv4) >= 0 && ip.Compare(a.currentAnonIPv4) <= 0 {
return true
} else if !ip.Is4() && ip.Compare(a.startAnonIPv6) >= 0 && ip.Compare(a.currentAnonIPv6) <= 0 {
return true
}
return false
}
func (a *Anonymizer) AnonymizeIPString(ip string) string {
addr, err := netip.ParseAddr(ip)
if err != nil {
return ip
}
return a.AnonymizeIP(addr).String()
}
func (a *Anonymizer) AnonymizeDomain(domain string) string {
if strings.HasSuffix(domain, "netbird.io") ||
strings.HasSuffix(domain, "netbird.selfhosted") ||
strings.HasSuffix(domain, "netbird.cloud") ||
strings.HasSuffix(domain, "netbird.stage") ||
strings.HasSuffix(domain, ".domain") {
return domain
}
parts := strings.Split(domain, ".")
if len(parts) < 2 {
return domain
}
baseDomain := parts[len(parts)-2] + "." + parts[len(parts)-1]
anonymized, ok := a.domainAnonymizer[baseDomain]
if !ok {
anonymizedBase := "anon-" + generateRandomString(5) + ".domain"
a.domainAnonymizer[baseDomain] = anonymizedBase
anonymized = anonymizedBase
}
return strings.Replace(domain, baseDomain, anonymized, 1)
}
func (a *Anonymizer) AnonymizeURI(uri string) string {
u, err := url.Parse(uri)
if err != nil {
return uri
}
var anonymizedHost string
if u.Opaque != "" {
host, port, err := net.SplitHostPort(u.Opaque)
if err == nil {
anonymizedHost = fmt.Sprintf("%s:%s", a.AnonymizeDomain(host), port)
} else {
anonymizedHost = a.AnonymizeDomain(u.Opaque)
}
u.Opaque = anonymizedHost
} else if u.Host != "" {
host, port, err := net.SplitHostPort(u.Host)
if err == nil {
anonymizedHost = fmt.Sprintf("%s:%s", a.AnonymizeDomain(host), port)
} else {
anonymizedHost = a.AnonymizeDomain(u.Host)
}
u.Host = anonymizedHost
}
return u.String()
}
func (a *Anonymizer) AnonymizeString(str string) string {
ipv4Regex := regexp.MustCompile(`\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b`)
ipv6Regex := regexp.MustCompile(`\b([0-9a-fA-F:]+:+[0-9a-fA-F]{0,4})(?:%[0-9a-zA-Z]+)?(?:\/[0-9]{1,3})?(?::[0-9]{1,5})?\b`)
str = ipv4Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
str = ipv6Regex.ReplaceAllStringFunc(str, a.AnonymizeIPString)
for domain, anonDomain := range a.domainAnonymizer {
str = strings.ReplaceAll(str, domain, anonDomain)
}
str = a.AnonymizeSchemeURI(str)
str = a.AnonymizeDNSLogLine(str)
return str
}
// AnonymizeSchemeURI finds and anonymizes URIs with stun, stuns, turn, and turns schemes.
func (a *Anonymizer) AnonymizeSchemeURI(text string) string {
re := regexp.MustCompile(`(?i)\b(stuns?:|turns?:|https?://)\S+\b`)
return re.ReplaceAllStringFunc(text, a.AnonymizeURI)
}
// AnonymizeDNSLogLine anonymizes domain names in DNS log entries by replacing them with a random string.
func (a *Anonymizer) AnonymizeDNSLogLine(logEntry string) string {
domainPattern := `dns\.Question{Name:"([^"]+)",`
domainRegex := regexp.MustCompile(domainPattern)
return domainRegex.ReplaceAllStringFunc(logEntry, func(match string) string {
parts := strings.Split(match, `"`)
if len(parts) >= 2 {
domain := parts[1]
if strings.HasSuffix(domain, ".domain") {
return match
}
randomDomain := generateRandomString(10) + ".domain"
return strings.Replace(match, domain, randomDomain, 1)
}
return match
})
}
func isWellKnown(addr netip.Addr) bool {
wellKnown := []string{
"8.8.8.8", "8.8.4.4", // Google DNS IPv4
"2001:4860:4860::8888", "2001:4860:4860::8844", // Google DNS IPv6
"1.1.1.1", "1.0.0.1", // Cloudflare DNS IPv4
"2606:4700:4700::1111", "2606:4700:4700::1001", // Cloudflare DNS IPv6
"9.9.9.9", "149.112.112.112", // Quad9 DNS IPv4
"2620:fe::fe", "2620:fe::9", // Quad9 DNS IPv6
}
if slices.Contains(wellKnown, addr.String()) {
return true
}
cgnatRangeStart := netip.AddrFrom4([4]byte{100, 64, 0, 0})
cgnatRange := netip.PrefixFrom(cgnatRangeStart, 10)
return cgnatRange.Contains(addr)
}
func generateRandomString(length int) string {
const letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
result := make([]byte, length)
for i := range result {
num, err := rand.Int(rand.Reader, big.NewInt(int64(len(letters))))
if err != nil {
continue
}
result[i] = letters[num.Int64()]
}
return string(result)
}

View File

@@ -0,0 +1,223 @@
package anonymize_test
import (
"net/netip"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/anonymize"
)
func TestAnonymizeIP(t *testing.T) {
startIPv4 := netip.MustParseAddr("198.51.100.0")
startIPv6 := netip.MustParseAddr("100::")
anonymizer := anonymize.NewAnonymizer(startIPv4, startIPv6)
tests := []struct {
name string
ip string
expect string
}{
{"Well known", "8.8.8.8", "8.8.8.8"},
{"First Public IPv4", "1.2.3.4", "198.51.100.0"},
{"Second Public IPv4", "4.3.2.1", "198.51.100.1"},
{"Repeated IPv4", "1.2.3.4", "198.51.100.0"},
{"Private IPv4", "192.168.1.1", "192.168.1.1"},
{"First Public IPv6", "2607:f8b0:4005:805::200e", "100::"},
{"Second Public IPv6", "a::b", "100::1"},
{"Repeated IPv6", "2607:f8b0:4005:805::200e", "100::"},
{"Private IPv6", "fe80::1", "fe80::1"},
{"In Range IPv4", "198.51.100.2", "198.51.100.2"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
ip := netip.MustParseAddr(tc.ip)
anonymizedIP := anonymizer.AnonymizeIP(ip)
if anonymizedIP.String() != tc.expect {
t.Errorf("%s: expected %s, got %s", tc.name, tc.expect, anonymizedIP)
}
})
}
}
func TestAnonymizeDNSLogLine(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(netip.Addr{}, netip.Addr{})
testLog := `2024-04-23T20:01:11+02:00 TRAC client/internal/dns/local.go:25: received question: dns.Question{Name:"example.com", Qtype:0x1c, Qclass:0x1}`
result := anonymizer.AnonymizeDNSLogLine(testLog)
require.NotEqual(t, testLog, result)
assert.NotContains(t, result, "example.com")
}
func TestAnonymizeDomain(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(netip.Addr{}, netip.Addr{})
tests := []struct {
name string
domain string
expectPattern string
shouldAnonymize bool
}{
{
"General Domain",
"example.com",
`^anon-[a-zA-Z0-9]+\.domain$`,
true,
},
{
"Subdomain",
"sub.example.com",
`^sub\.anon-[a-zA-Z0-9]+\.domain$`,
true,
},
{
"Protected Domain",
"netbird.io",
`^netbird\.io$`,
false,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
result := anonymizer.AnonymizeDomain(tc.domain)
if tc.shouldAnonymize {
assert.Regexp(t, tc.expectPattern, result, "The anonymized domain should match the expected pattern")
assert.NotContains(t, result, tc.domain, "The original domain should not be present in the result")
} else {
assert.Equal(t, tc.domain, result, "Protected domains should not be anonymized")
}
})
}
}
func TestAnonymizeURI(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(netip.Addr{}, netip.Addr{})
tests := []struct {
name string
uri string
regex string
}{
{
"HTTP URI with Port",
"http://example.com:80/path",
`^http://anon-[a-zA-Z0-9]+\.domain:80/path$`,
},
{
"HTTP URI without Port",
"http://example.com/path",
`^http://anon-[a-zA-Z0-9]+\.domain/path$`,
},
{
"Opaque URI with Port",
"stun:example.com:80?transport=udp",
`^stun:anon-[a-zA-Z0-9]+\.domain:80\?transport=udp$`,
},
{
"Opaque URI without Port",
"stun:example.com?transport=udp",
`^stun:anon-[a-zA-Z0-9]+\.domain\?transport=udp$`,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
result := anonymizer.AnonymizeURI(tc.uri)
assert.Regexp(t, regexp.MustCompile(tc.regex), result, "URI should match expected pattern")
require.NotContains(t, result, "example.com", "Original domain should not be present")
})
}
}
func TestAnonymizeSchemeURI(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(netip.Addr{}, netip.Addr{})
tests := []struct {
name string
input string
expect string
}{
{"STUN URI in text", "Connection made via stun:example.com", `Connection made via stun:anon-[a-zA-Z0-9]+\.domain`},
{"TURN URI in log", "Failed attempt turn:some.example.com:3478?transport=tcp: retrying", `Failed attempt turn:some.anon-[a-zA-Z0-9]+\.domain:3478\?transport=tcp: retrying`},
{"HTTPS URI in message", "Visit https://example.com for more", `Visit https://anon-[a-zA-Z0-9]+\.domain for more`},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
result := anonymizer.AnonymizeSchemeURI(tc.input)
assert.Regexp(t, tc.expect, result, "The anonymized output should match expected pattern")
require.NotContains(t, result, "example.com", "Original domain should not be present")
})
}
}
func TestAnonymizString_MemorizedDomain(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(netip.Addr{}, netip.Addr{})
domain := "example.com"
anonymizedDomain := anonymizer.AnonymizeDomain(domain)
sampleString := "This is a test string including the domain example.com which should be anonymized."
firstPassResult := anonymizer.AnonymizeString(sampleString)
secondPassResult := anonymizer.AnonymizeString(firstPassResult)
assert.Contains(t, firstPassResult, anonymizedDomain, "The domain should be anonymized in the first pass")
assert.NotContains(t, firstPassResult, domain, "The original domain should not appear in the first pass output")
assert.Equal(t, firstPassResult, secondPassResult, "The second pass should not further anonymize the string")
}
func TestAnonymizeString_DoubleURI(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(netip.Addr{}, netip.Addr{})
domain := "example.com"
anonymizedDomain := anonymizer.AnonymizeDomain(domain)
sampleString := "Check out our site at https://example.com for more info."
firstPassResult := anonymizer.AnonymizeString(sampleString)
secondPassResult := anonymizer.AnonymizeString(firstPassResult)
assert.Contains(t, firstPassResult, "https://"+anonymizedDomain, "The URI should be anonymized in the first pass")
assert.NotContains(t, firstPassResult, "https://example.com", "The original URI should not appear in the first pass output")
assert.Equal(t, firstPassResult, secondPassResult, "The second pass should not further anonymize the URI")
}
func TestAnonymizeString_IPAddresses(t *testing.T) {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
tests := []struct {
name string
input string
expect string
}{
{
name: "IPv4 Address",
input: "Error occurred at IP 122.138.1.1",
expect: "Error occurred at IP 198.51.100.0",
},
{
name: "IPv6 Address",
input: "Access attempted from 2001:db8::ff00:42",
expect: "Access attempted from 100::",
},
{
name: "IPv6 Address with Port",
input: "Access attempted from [2001:db8::ff00:42]:8080",
expect: "Access attempted from [100::]:8080",
},
{
name: "Both IPv4 and IPv6",
input: "IPv4: 142.108.0.1 and IPv6: 2001:db8::ff00:43",
expect: "IPv4: 198.51.100.1 and IPv6: 100::1",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
result := anonymizer.AnonymizeString(tc.input)
assert.Equal(t, tc.expect, result, "IP addresses should be anonymized correctly")
})
}
}

255
client/cmd/debug.go Normal file
View File

@@ -0,0 +1,255 @@
package cmd
import (
"context"
"fmt"
"time"
"github.com/spf13/cobra"
"google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/proto"
"github.com/netbirdio/netbird/client/server"
)
var debugCmd = &cobra.Command{
Use: "debug",
Short: "Debugging commands",
Long: "Provides commands for debugging and logging control within the Netbird daemon.",
}
var debugBundleCmd = &cobra.Command{
Use: "bundle",
Example: " netbird debug bundle",
Short: "Create a debug bundle",
Long: "Generates a compressed archive of the daemon's logs and status for debugging purposes.",
RunE: debugBundle,
}
var logCmd = &cobra.Command{
Use: "log",
Short: "Manage logging for the Netbird daemon",
Long: `Commands to manage logging settings for the Netbird daemon, including ICE, gRPC, and general log levels.`,
}
var logLevelCmd = &cobra.Command{
Use: "level <level>",
Short: "Set the logging level for this session",
Long: `Sets the logging level for the current session. This setting is temporary and will revert to the default on daemon restart.
Available log levels are:
panic: for panic level, highest level of severity
fatal: for fatal level errors that cause the program to exit
error: for error conditions
warn: for warning conditions
info: for informational messages
debug: for debug-level messages
trace: for trace-level messages, which include more fine-grained information than debug`,
Args: cobra.ExactArgs(1),
RunE: setLogLevel,
}
var forCmd = &cobra.Command{
Use: "for <time>",
Short: "Run debug logs for a specified duration and create a debug bundle",
Long: `Sets the logging level to trace, runs for the specified duration, and then generates a debug bundle.`,
Example: " netbird debug for 5m",
Args: cobra.ExactArgs(1),
RunE: runForDuration,
}
func debugBundle(cmd *cobra.Command, _ []string) error {
conn, err := getClient(cmd.Context())
if err != nil {
return err
}
defer conn.Close()
client := proto.NewDaemonServiceClient(conn)
resp, err := client.DebugBundle(cmd.Context(), &proto.DebugBundleRequest{
Anonymize: anonymizeFlag,
Status: getStatusOutput(cmd),
})
if err != nil {
return fmt.Errorf("failed to bundle debug: %v", status.Convert(err).Message())
}
cmd.Println(resp.GetPath())
return nil
}
func setLogLevel(cmd *cobra.Command, args []string) error {
conn, err := getClient(cmd.Context())
if err != nil {
return err
}
defer conn.Close()
client := proto.NewDaemonServiceClient(conn)
level := server.ParseLogLevel(args[0])
if level == proto.LogLevel_UNKNOWN {
return fmt.Errorf("unknown log level: %s. Available levels are: panic, fatal, error, warn, info, debug, trace\n", args[0])
}
_, err = client.SetLogLevel(cmd.Context(), &proto.SetLogLevelRequest{
Level: level,
})
if err != nil {
return fmt.Errorf("failed to set log level: %v", status.Convert(err).Message())
}
cmd.Println("Log level set successfully to", args[0])
return nil
}
func runForDuration(cmd *cobra.Command, args []string) error {
duration, err := time.ParseDuration(args[0])
if err != nil {
return fmt.Errorf("invalid duration format: %v", err)
}
conn, err := getClient(cmd.Context())
if err != nil {
return err
}
defer conn.Close()
client := proto.NewDaemonServiceClient(conn)
stat, err := client.Status(cmd.Context(), &proto.StatusRequest{})
if err != nil {
return fmt.Errorf("failed to get status: %v", status.Convert(err).Message())
}
restoreUp := stat.Status == string(internal.StatusConnected) || stat.Status == string(internal.StatusConnecting)
initialLogLevel, err := client.GetLogLevel(cmd.Context(), &proto.GetLogLevelRequest{})
if err != nil {
return fmt.Errorf("failed to get log level: %v", status.Convert(err).Message())
}
if _, err := client.Down(cmd.Context(), &proto.DownRequest{}); err != nil {
return fmt.Errorf("failed to down: %v", status.Convert(err).Message())
}
cmd.Println("Netbird down")
initialLevelTrace := initialLogLevel.GetLevel() >= proto.LogLevel_TRACE
if !initialLevelTrace {
_, err = client.SetLogLevel(cmd.Context(), &proto.SetLogLevelRequest{
Level: proto.LogLevel_TRACE,
})
if err != nil {
return fmt.Errorf("failed to set log level to TRACE: %v", status.Convert(err).Message())
}
cmd.Println("Log level set to trace.")
}
time.Sleep(1 * time.Second)
if _, err := client.Up(cmd.Context(), &proto.UpRequest{}); err != nil {
return fmt.Errorf("failed to up: %v", status.Convert(err).Message())
}
cmd.Println("Netbird up")
time.Sleep(3 * time.Second)
headerPostUp := fmt.Sprintf("----- Netbird post-up - Timestamp: %s", time.Now().Format(time.RFC3339))
statusOutput := fmt.Sprintf("%s\n%s", headerPostUp, getStatusOutput(cmd))
if waitErr := waitForDurationOrCancel(cmd.Context(), duration, cmd); waitErr != nil {
return waitErr
}
cmd.Println("\nDuration completed")
headerPreDown := fmt.Sprintf("----- Netbird pre-down - Timestamp: %s - Duration: %s", time.Now().Format(time.RFC3339), duration)
statusOutput = fmt.Sprintf("%s\n%s\n%s", statusOutput, headerPreDown, getStatusOutput(cmd))
if _, err := client.Down(cmd.Context(), &proto.DownRequest{}); err != nil {
return fmt.Errorf("failed to down: %v", status.Convert(err).Message())
}
cmd.Println("Netbird down")
time.Sleep(1 * time.Second)
if restoreUp {
if _, err := client.Up(cmd.Context(), &proto.UpRequest{}); err != nil {
return fmt.Errorf("failed to up: %v", status.Convert(err).Message())
}
cmd.Println("Netbird up")
}
if !initialLevelTrace {
if _, err := client.SetLogLevel(cmd.Context(), &proto.SetLogLevelRequest{Level: initialLogLevel.GetLevel()}); err != nil {
return fmt.Errorf("failed to restore log level: %v", status.Convert(err).Message())
}
cmd.Println("Log level restored to", initialLogLevel.GetLevel())
}
cmd.Println("Creating debug bundle...")
resp, err := client.DebugBundle(cmd.Context(), &proto.DebugBundleRequest{
Anonymize: anonymizeFlag,
Status: statusOutput,
})
if err != nil {
return fmt.Errorf("failed to bundle debug: %v", status.Convert(err).Message())
}
cmd.Println(resp.GetPath())
return nil
}
func getStatusOutput(cmd *cobra.Command) string {
var statusOutputString string
statusResp, err := getStatus(cmd.Context())
if err != nil {
cmd.PrintErrf("Failed to get status: %v\n", err)
} else {
statusOutputString = parseToFullDetailSummary(convertToStatusOutputOverview(statusResp))
}
return statusOutputString
}
func waitForDurationOrCancel(ctx context.Context, duration time.Duration, cmd *cobra.Command) error {
ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop()
startTime := time.Now()
done := make(chan struct{})
go func() {
defer close(done)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
elapsed := time.Since(startTime)
if elapsed >= duration {
return
}
remaining := duration - elapsed
cmd.Printf("\rRemaining time: %s", formatDuration(remaining))
}
}
}()
select {
case <-ctx.Done():
return ctx.Err()
case <-done:
return nil
}
}
func formatDuration(d time.Duration) string {
d = d.Round(time.Second)
h := d / time.Hour
d %= time.Hour
m := d / time.Minute
d %= time.Minute
s := d / time.Second
return fmt.Sprintf("%02d:%02d:%02d", h, m, s)
}

View File

@@ -2,9 +2,10 @@ package cmd
import (
"context"
"github.com/netbirdio/netbird/util"
"time"
"github.com/netbirdio/netbird/util"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"

View File

@@ -32,6 +32,7 @@ const (
preSharedKeyFlag = "preshared-key"
interfaceNameFlag = "interface-name"
wireguardPortFlag = "wireguard-port"
networkMonitorFlag = "network-monitor"
disableAutoConnectFlag = "disable-auto-connect"
serverSSHAllowedFlag = "allow-server-ssh"
extraIFaceBlackListFlag = "extra-iface-blacklist"
@@ -62,9 +63,11 @@ var (
serverSSHAllowed bool
interfaceName string
wireguardPort uint16
networkMonitor bool
serviceName string
autoConnectDisabled bool
extraIFaceBlackList []string
anonymizeFlag bool
rootCmd = &cobra.Command{
Use: "netbird",
Short: "",
@@ -119,6 +122,8 @@ func init() {
rootCmd.PersistentFlags().StringVarP(&setupKey, "setup-key", "k", "", "Setup key obtained from the Management Service Dashboard (used to register peer)")
rootCmd.PersistentFlags().StringVar(&preSharedKey, preSharedKeyFlag, "", "Sets Wireguard PreSharedKey property. If set, then only peers that have the same key can communicate.")
rootCmd.PersistentFlags().StringVarP(&hostName, "hostname", "n", "", "Sets a custom hostname for the device")
rootCmd.PersistentFlags().BoolVarP(&anonymizeFlag, "anonymize", "A", false, "anonymize IP addresses and non-netbird.io domains in logs and status output")
rootCmd.AddCommand(serviceCmd)
rootCmd.AddCommand(upCmd)
rootCmd.AddCommand(downCmd)
@@ -126,8 +131,20 @@ func init() {
rootCmd.AddCommand(loginCmd)
rootCmd.AddCommand(versionCmd)
rootCmd.AddCommand(sshCmd)
rootCmd.AddCommand(routesCmd)
rootCmd.AddCommand(debugCmd)
serviceCmd.AddCommand(runCmd, startCmd, stopCmd, restartCmd) // service control commands are subcommands of service
serviceCmd.AddCommand(installCmd, uninstallCmd) // service installer commands are subcommands of service
routesCmd.AddCommand(routesListCmd)
routesCmd.AddCommand(routesSelectCmd, routesDeselectCmd)
debugCmd.AddCommand(debugBundleCmd)
debugCmd.AddCommand(logCmd)
logCmd.AddCommand(logLevelCmd)
debugCmd.AddCommand(forCmd)
upCmd.PersistentFlags().StringSliceVar(&natExternalIPs, externalIPMapFlag, nil,
`Sets external IPs maps between local addresses and interfaces.`+
`You can specify a comma-separated list with a single IP and IP/IP or IP/Interface Name. `+
@@ -335,3 +352,14 @@ func migrateToNetbird(oldPath, newPath string) bool {
return true
}
func getClient(ctx context.Context) (*grpc.ClientConn, error) {
conn, err := DialClientGRPCServer(ctx, daemonAddr)
if err != nil {
return nil, fmt.Errorf("failed to connect to daemon error: %v\n"+
"If the daemon is not running please run: "+
"\nnetbird service install \nnetbird service start\n", err)
}
return conn, nil
}

131
client/cmd/route.go Normal file
View File

@@ -0,0 +1,131 @@
package cmd
import (
"fmt"
"github.com/spf13/cobra"
"google.golang.org/grpc/status"
"github.com/netbirdio/netbird/client/proto"
)
var appendFlag bool
var routesCmd = &cobra.Command{
Use: "routes",
Short: "Manage network routes",
Long: `Commands to list, select, or deselect network routes.`,
}
var routesListCmd = &cobra.Command{
Use: "list",
Aliases: []string{"ls"},
Short: "List routes",
Example: " netbird routes list",
Long: "List all available network routes.",
RunE: routesList,
}
var routesSelectCmd = &cobra.Command{
Use: "select route...|all",
Short: "Select routes",
Long: "Select a list of routes by identifiers or 'all' to clear all selections and to accept all (including new) routes.\nDefault mode is replace, use -a to append to already selected routes.",
Example: " netbird routes select all\n netbird routes select route1 route2\n netbird routes select -a route3",
Args: cobra.MinimumNArgs(1),
RunE: routesSelect,
}
var routesDeselectCmd = &cobra.Command{
Use: "deselect route...|all",
Short: "Deselect routes",
Long: "Deselect previously selected routes by identifiers or 'all' to disable accepting any routes.",
Example: " netbird routes deselect all\n netbird routes deselect route1 route2",
Args: cobra.MinimumNArgs(1),
RunE: routesDeselect,
}
func init() {
routesSelectCmd.PersistentFlags().BoolVarP(&appendFlag, "append", "a", false, "Append to current route selection instead of replacing")
}
func routesList(cmd *cobra.Command, _ []string) error {
conn, err := getClient(cmd.Context())
if err != nil {
return err
}
defer conn.Close()
client := proto.NewDaemonServiceClient(conn)
resp, err := client.ListRoutes(cmd.Context(), &proto.ListRoutesRequest{})
if err != nil {
return fmt.Errorf("failed to list routes: %v", status.Convert(err).Message())
}
if len(resp.Routes) == 0 {
cmd.Println("No routes available.")
return nil
}
cmd.Println("Available Routes:")
for _, route := range resp.Routes {
selectedStatus := "Not Selected"
if route.GetSelected() {
selectedStatus = "Selected"
}
cmd.Printf("\n - ID: %s\n Network: %s\n Status: %s\n", route.GetID(), route.GetNetwork(), selectedStatus)
}
return nil
}
func routesSelect(cmd *cobra.Command, args []string) error {
conn, err := getClient(cmd.Context())
if err != nil {
return err
}
defer conn.Close()
client := proto.NewDaemonServiceClient(conn)
req := &proto.SelectRoutesRequest{
RouteIDs: args,
}
if len(args) == 1 && args[0] == "all" {
req.All = true
} else if appendFlag {
req.Append = true
}
if _, err := client.SelectRoutes(cmd.Context(), req); err != nil {
return fmt.Errorf("failed to select routes: %v", status.Convert(err).Message())
}
cmd.Println("Routes selected successfully.")
return nil
}
func routesDeselect(cmd *cobra.Command, args []string) error {
conn, err := getClient(cmd.Context())
if err != nil {
return err
}
defer conn.Close()
client := proto.NewDaemonServiceClient(conn)
req := &proto.SelectRoutesRequest{
RouteIDs: args,
}
if len(args) == 1 && args[0] == "all" {
req.All = true
}
if _, err := client.DeselectRoutes(cmd.Context(), req); err != nil {
return fmt.Errorf("failed to deselect routes: %v", status.Convert(err).Message())
}
cmd.Println("Routes deselected successfully.")
return nil
}

View File

@@ -24,7 +24,7 @@ var (
)
var sshCmd = &cobra.Command{
Use: "ssh",
Use: "ssh [user@]host",
Args: func(cmd *cobra.Command, args []string) error {
if len(args) < 1 {
return errors.New("requires a host argument")
@@ -94,7 +94,7 @@ func runSSH(ctx context.Context, addr string, pemKey []byte, cmd *cobra.Command)
if err != nil {
cmd.Printf("Error: %v\n", err)
cmd.Printf("Couldn't connect. Please check the connection status or if the ssh server is enabled on the other peer" +
"You can verify the connection by running:\n\n" +
"\nYou can verify the connection by running:\n\n" +
" netbird status\n\n")
return err
}

View File

@@ -6,6 +6,8 @@ import (
"fmt"
"net"
"net/netip"
"os"
"runtime"
"sort"
"strings"
"time"
@@ -14,6 +16,7 @@ import (
"google.golang.org/grpc/status"
"gopkg.in/yaml.v3"
"github.com/netbirdio/netbird/client/anonymize"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/proto"
@@ -144,9 +147,9 @@ func statusFunc(cmd *cobra.Command, args []string) error {
return fmt.Errorf("failed initializing log %v", err)
}
ctx := internal.CtxInitState(context.Background())
ctx := internal.CtxInitState(cmd.Context())
resp, err := getStatus(ctx, cmd)
resp, err := getStatus(ctx)
if err != nil {
return err
}
@@ -191,7 +194,7 @@ func statusFunc(cmd *cobra.Command, args []string) error {
return nil
}
func getStatus(ctx context.Context, cmd *cobra.Command) (*proto.StatusResponse, error) {
func getStatus(ctx context.Context) (*proto.StatusResponse, error) {
conn, err := DialClientGRPCServer(ctx, daemonAddr)
if err != nil {
return nil, fmt.Errorf("failed to connect to daemon error: %v\n"+
@@ -200,7 +203,7 @@ func getStatus(ctx context.Context, cmd *cobra.Command) (*proto.StatusResponse,
}
defer conn.Close()
resp, err := proto.NewDaemonServiceClient(conn).Status(cmd.Context(), &proto.StatusRequest{GetFullPeerStatus: true})
resp, err := proto.NewDaemonServiceClient(conn).Status(ctx, &proto.StatusRequest{GetFullPeerStatus: true})
if err != nil {
return nil, fmt.Errorf("status failed: %v", status.Convert(err).Message())
}
@@ -283,6 +286,11 @@ func convertToStatusOutputOverview(resp *proto.StatusResponse) statusOutputOverv
NSServerGroups: mapNSGroups(pbFullStatus.GetDnsServers()),
}
if anonymizeFlag {
anonymizer := anonymize.NewAnonymizer(anonymize.DefaultAddresses())
anonymizeOverview(anonymizer, &overview)
}
return overview
}
@@ -525,8 +533,16 @@ func parseGeneralSummary(overview statusOutputOverview, showURL bool, showRelays
peersCountString := fmt.Sprintf("%d/%d Connected", overview.Peers.Connected, overview.Peers.Total)
goos := runtime.GOOS
goarch := runtime.GOARCH
goarm := ""
if goarch == "arm" {
goarm = fmt.Sprintf(" (ARMv%s)", os.Getenv("GOARM"))
}
summary := fmt.Sprintf(
"Daemon version: %s\n"+
"OS: %s\n"+
"Daemon version: %s\n"+
"CLI version: %s\n"+
"Management: %s\n"+
"Signal: %s\n"+
@@ -538,6 +554,7 @@ func parseGeneralSummary(overview statusOutputOverview, showURL bool, showRelays
"Quantum resistance: %s\n"+
"Routes: %s\n"+
"Peers count: %s\n",
fmt.Sprintf("%s/%s%s", goos, goarch, goarm),
overview.DaemonVersion,
version.NetbirdVersion(),
managementConnString,
@@ -593,15 +610,6 @@ func parsePeers(peers peersStateOutput, rosenpassEnabled, rosenpassPermissive bo
if peerState.IceCandidateEndpoint.Remote != "" {
remoteICEEndpoint = peerState.IceCandidateEndpoint.Remote
}
lastStatusUpdate := "-"
if !peerState.LastStatusUpdate.IsZero() {
lastStatusUpdate = peerState.LastStatusUpdate.Format("2006-01-02 15:04:05")
}
lastWireGuardHandshake := "-"
if !peerState.LastWireguardHandshake.IsZero() && peerState.LastWireguardHandshake != time.Unix(0, 0) {
lastWireGuardHandshake = peerState.LastWireguardHandshake.Format("2006-01-02 15:04:05")
}
rosenpassEnabledStatus := "false"
if rosenpassEnabled {
@@ -652,8 +660,8 @@ func parsePeers(peers peersStateOutput, rosenpassEnabled, rosenpassPermissive bo
remoteICE,
localICEEndpoint,
remoteICEEndpoint,
lastStatusUpdate,
lastWireGuardHandshake,
timeAgo(peerState.LastStatusUpdate),
timeAgo(peerState.LastWireguardHandshake),
toIEC(peerState.TransferReceived),
toIEC(peerState.TransferSent),
rosenpassEnabledStatus,
@@ -722,3 +730,129 @@ func countEnabled(dnsServers []nsServerGroupStateOutput) int {
}
return count
}
// timeAgo returns a string representing the duration since the provided time in a human-readable format.
func timeAgo(t time.Time) string {
if t.IsZero() || t.Equal(time.Unix(0, 0)) {
return "-"
}
duration := time.Since(t)
switch {
case duration < time.Second:
return "Now"
case duration < time.Minute:
seconds := int(duration.Seconds())
if seconds == 1 {
return "1 second ago"
}
return fmt.Sprintf("%d seconds ago", seconds)
case duration < time.Hour:
minutes := int(duration.Minutes())
seconds := int(duration.Seconds()) % 60
if minutes == 1 {
if seconds == 1 {
return "1 minute, 1 second ago"
} else if seconds > 0 {
return fmt.Sprintf("1 minute, %d seconds ago", seconds)
}
return "1 minute ago"
}
if seconds > 0 {
return fmt.Sprintf("%d minutes, %d seconds ago", minutes, seconds)
}
return fmt.Sprintf("%d minutes ago", minutes)
case duration < 24*time.Hour:
hours := int(duration.Hours())
minutes := int(duration.Minutes()) % 60
if hours == 1 {
if minutes == 1 {
return "1 hour, 1 minute ago"
} else if minutes > 0 {
return fmt.Sprintf("1 hour, %d minutes ago", minutes)
}
return "1 hour ago"
}
if minutes > 0 {
return fmt.Sprintf("%d hours, %d minutes ago", hours, minutes)
}
return fmt.Sprintf("%d hours ago", hours)
}
days := int(duration.Hours()) / 24
hours := int(duration.Hours()) % 24
if days == 1 {
if hours == 1 {
return "1 day, 1 hour ago"
} else if hours > 0 {
return fmt.Sprintf("1 day, %d hours ago", hours)
}
return "1 day ago"
}
if hours > 0 {
return fmt.Sprintf("%d days, %d hours ago", days, hours)
}
return fmt.Sprintf("%d days ago", days)
}
func anonymizePeerDetail(a *anonymize.Anonymizer, peer *peerStateDetailOutput) {
peer.FQDN = a.AnonymizeDomain(peer.FQDN)
if localIP, port, err := net.SplitHostPort(peer.IceCandidateEndpoint.Local); err == nil {
peer.IceCandidateEndpoint.Local = fmt.Sprintf("%s:%s", a.AnonymizeIPString(localIP), port)
}
if remoteIP, port, err := net.SplitHostPort(peer.IceCandidateEndpoint.Remote); err == nil {
peer.IceCandidateEndpoint.Remote = fmt.Sprintf("%s:%s", a.AnonymizeIPString(remoteIP), port)
}
for i, route := range peer.Routes {
peer.Routes[i] = a.AnonymizeIPString(route)
}
for i, route := range peer.Routes {
prefix, err := netip.ParsePrefix(route)
if err == nil {
ip := a.AnonymizeIPString(prefix.Addr().String())
peer.Routes[i] = fmt.Sprintf("%s/%d", ip, prefix.Bits())
}
}
}
func anonymizeOverview(a *anonymize.Anonymizer, overview *statusOutputOverview) {
for i, peer := range overview.Peers.Details {
peer := peer
anonymizePeerDetail(a, &peer)
overview.Peers.Details[i] = peer
}
overview.ManagementState.URL = a.AnonymizeURI(overview.ManagementState.URL)
overview.ManagementState.Error = a.AnonymizeString(overview.ManagementState.Error)
overview.SignalState.URL = a.AnonymizeURI(overview.SignalState.URL)
overview.SignalState.Error = a.AnonymizeString(overview.SignalState.Error)
overview.IP = a.AnonymizeIPString(overview.IP)
for i, detail := range overview.Relays.Details {
detail.URI = a.AnonymizeURI(detail.URI)
detail.Error = a.AnonymizeString(detail.Error)
overview.Relays.Details[i] = detail
}
for i, nsGroup := range overview.NSServerGroups {
for j, domain := range nsGroup.Domains {
overview.NSServerGroups[i].Domains[j] = a.AnonymizeDomain(domain)
}
for j, ns := range nsGroup.Servers {
host, port, err := net.SplitHostPort(ns)
if err == nil {
overview.NSServerGroups[i].Servers[j] = fmt.Sprintf("%s:%s", a.AnonymizeIPString(host), port)
}
}
}
for i, route := range overview.Routes {
prefix, err := netip.ParsePrefix(route)
if err == nil {
ip := a.AnonymizeIPString(prefix.Addr().String())
overview.Routes[i] = fmt.Sprintf("%s/%d", ip, prefix.Bits())
}
}
overview.FQDN = a.AnonymizeDomain(overview.FQDN)
}

View File

@@ -3,6 +3,8 @@ package cmd
import (
"bytes"
"encoding/json"
"fmt"
"runtime"
"testing"
"time"
@@ -487,9 +489,15 @@ dnsServers:
}
func TestParsingToDetail(t *testing.T) {
// Calculate time ago based on the fixture dates
lastConnectionUpdate1 := timeAgo(overview.Peers.Details[0].LastStatusUpdate)
lastHandshake1 := timeAgo(overview.Peers.Details[0].LastWireguardHandshake)
lastConnectionUpdate2 := timeAgo(overview.Peers.Details[1].LastStatusUpdate)
lastHandshake2 := timeAgo(overview.Peers.Details[1].LastWireguardHandshake)
detail := parseToFullDetailSummary(overview)
expectedDetail :=
expectedDetail := fmt.Sprintf(
`Peers detail:
peer-1.awesome-domain.com:
NetBird IP: 192.168.178.101
@@ -500,8 +508,8 @@ func TestParsingToDetail(t *testing.T) {
Direct: true
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Last connection update: 2001-01-01 01:01:01
Last WireGuard handshake: 2001-01-01 01:01:02
Last connection update: %s
Last WireGuard handshake: %s
Transfer status (received/sent) 200 B/100 B
Quantum resistance: false
Routes: 10.1.0.0/24
@@ -516,15 +524,16 @@ func TestParsingToDetail(t *testing.T) {
Direct: false
ICE candidate (Local/Remote): relay/prflx
ICE candidate endpoints (Local/Remote): 10.0.0.1:10001/10.0.10.1:10002
Last connection update: 2002-02-02 02:02:02
Last WireGuard handshake: 2002-02-02 02:02:03
Last connection update: %s
Last WireGuard handshake: %s
Transfer status (received/sent) 2.0 KiB/1000 B
Quantum resistance: false
Routes: -
Latency: 10ms
OS: %s/%s
Daemon version: 0.14.1
CLI version: development
CLI version: %s
Management: Connected to my-awesome-management.com:443
Signal: Connected to my-awesome-signal.com:443
Relays:
@@ -539,7 +548,7 @@ Interface type: Kernel
Quantum resistance: false
Routes: 10.10.0.0/24
Peers count: 2/2 Connected
`
`, lastConnectionUpdate1, lastHandshake1, lastConnectionUpdate2, lastHandshake2, runtime.GOOS, runtime.GOARCH, overview.CliVersion)
assert.Equal(t, expectedDetail, detail)
}
@@ -547,8 +556,8 @@ Peers count: 2/2 Connected
func TestParsingToShortVersion(t *testing.T) {
shortVersion := parseGeneralSummary(overview, false, false, false)
expectedString :=
`Daemon version: 0.14.1
expectedString := fmt.Sprintf("OS: %s/%s", runtime.GOOS, runtime.GOARCH) + `
Daemon version: 0.14.1
CLI version: development
Management: Connected
Signal: Connected
@@ -572,3 +581,31 @@ func TestParsingOfIP(t *testing.T) {
assert.Equal(t, "192.168.178.123\n", parsedIP)
}
func TestTimeAgo(t *testing.T) {
now := time.Now()
cases := []struct {
name string
input time.Time
expected string
}{
{"Now", now, "Now"},
{"Seconds ago", now.Add(-10 * time.Second), "10 seconds ago"},
{"One minute ago", now.Add(-1 * time.Minute), "1 minute ago"},
{"Minutes and seconds ago", now.Add(-(1*time.Minute + 30*time.Second)), "1 minute, 30 seconds ago"},
{"One hour ago", now.Add(-1 * time.Hour), "1 hour ago"},
{"Hours and minutes ago", now.Add(-(2*time.Hour + 15*time.Minute)), "2 hours, 15 minutes ago"},
{"One day ago", now.Add(-24 * time.Hour), "1 day ago"},
{"Multiple days ago", now.Add(-(72*time.Hour + 20*time.Minute)), "3 days ago"},
{"Zero time", time.Time{}, "-"},
{"Unix zero time", time.Unix(0, 0), "-"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
result := timeAgo(tc.input)
assert.Equal(t, tc.expected, result, "Failed %s", tc.name)
})
}
}

View File

@@ -14,6 +14,7 @@ import (
"google.golang.org/grpc"
"github.com/netbirdio/management-integrations/integrations"
clientProto "github.com/netbirdio/netbird/client/proto"
client "github.com/netbirdio/netbird/client/server"
mgmtProto "github.com/netbirdio/netbird/management/proto"
@@ -69,10 +70,11 @@ func startManagement(t *testing.T, config *mgmt.Config) (*grpc.Server, net.Liste
t.Fatal(err)
}
s := grpc.NewServer()
store, err := mgmt.NewStoreFromJson(config.Datadir, nil)
store, cleanUp, err := mgmt.NewTestStoreFromJson(config.Datadir)
if err != nil {
t.Fatal(err)
}
t.Cleanup(cleanUp)
peersUpdateManager := mgmt.NewPeersUpdateManager(nil)
eventStore := &activity.InMemoryEventStore{}

View File

@@ -40,6 +40,7 @@ func init() {
upCmd.PersistentFlags().BoolVarP(&foregroundMode, "foreground-mode", "F", false, "start service in foreground")
upCmd.PersistentFlags().StringVar(&interfaceName, interfaceNameFlag, iface.WgInterfaceDefault, "Wireguard interface name")
upCmd.PersistentFlags().Uint16Var(&wireguardPort, wireguardPortFlag, iface.DefaultWgPort, "Wireguard interface listening port")
upCmd.PersistentFlags().BoolVarP(&networkMonitor, networkMonitorFlag, "N", false, "Enable network monitoring")
upCmd.PersistentFlags().StringSliceVar(&extraIFaceBlackList, extraIFaceBlackListFlag, nil, "Extra list of default interfaces to ignore for listening")
}
@@ -116,6 +117,10 @@ func runInForegroundMode(ctx context.Context, cmd *cobra.Command) error {
ic.WireguardPort = &p
}
if cmd.Flag(networkMonitorFlag).Changed {
ic.NetworkMonitor = &networkMonitor
}
if rootCmd.PersistentFlags().Changed(preSharedKeyFlag) {
ic.PreSharedKey = &preSharedKey
}
@@ -147,7 +152,9 @@ func runInForegroundMode(ctx context.Context, cmd *cobra.Command) error {
var cancel context.CancelFunc
ctx, cancel = context.WithCancel(ctx)
SetupCloseHandler(ctx, cancel)
return internal.RunClient(ctx, config, peer.NewRecorder(config.ManagementURL.String()))
connectClient := internal.NewConnectClient(ctx, config, peer.NewRecorder(config.ManagementURL.String()))
return connectClient.Run()
}
func runInDaemonMode(ctx context.Context, cmd *cobra.Command) error {
@@ -226,6 +233,10 @@ func runInDaemonMode(ctx context.Context, cmd *cobra.Command) error {
loginRequest.WireguardPort = &wp
}
if cmd.Flag(networkMonitorFlag).Changed {
loginRequest.NetworkMonitor = &networkMonitor
}
var loginErr error
var loginResp *proto.LoginResponse

View File

@@ -30,3 +30,9 @@ func NewFirewall(context context.Context, iface IFaceMapper) (firewall.Manager,
}
return fm, nil
}
// Returns true if the current firewall implementation supports IPv6.
// Currently false for anything non-linux.
func SupportsIPv6() bool {
return false
}

View File

@@ -42,20 +42,20 @@ func NewFirewall(context context.Context, iface IFaceMapper) (firewall.Manager,
switch check() {
case IPTABLES:
log.Debug("creating an iptables firewall manager")
log.Info("creating an iptables firewall manager")
fm, errFw = nbiptables.Create(context, iface)
if errFw != nil {
log.Errorf("failed to create iptables manager: %s", errFw)
}
case NFTABLES:
log.Debug("creating an nftables firewall manager")
log.Info("creating an nftables firewall manager")
fm, errFw = nbnftables.Create(context, iface)
if errFw != nil {
log.Errorf("failed to create nftables manager: %s", errFw)
}
default:
errFw = fmt.Errorf("no firewall manager found")
log.Debug("no firewall manager found, try to use userspace packet filtering firewall")
log.Info("no firewall manager found, trying to use userspace packet filtering firewall")
}
if iface.IsUserspaceBind() {
@@ -70,6 +70,8 @@ func NewFirewall(context context.Context, iface IFaceMapper) (firewall.Manager,
return nil, errUsp
}
// Note for devs: When adding IPv6 support to userspace bind, the implementation of AllowNetbird() has to be
// adjusted accordingly.
if err := fm.AllowNetbird(); err != nil {
log.Errorf("failed to allow netbird interface traffic: %v", err)
}
@@ -83,18 +85,66 @@ func NewFirewall(context context.Context, iface IFaceMapper) (firewall.Manager,
return fm, nil
}
// Returns true if the current firewall implementation supports IPv6.
// Currently true if the firewall is nftables.
func SupportsIPv6() bool {
return check() == NFTABLES
}
// check returns the firewall type based on common lib checks. It returns UNKNOWN if no firewall is found.
func check() FWType {
nf := nftables.Conn{}
if _, err := nf.ListChains(); err == nil && os.Getenv(SKIP_NFTABLES_ENV) != "true" {
return NFTABLES
useIPTABLES := false
var iptablesChains []string
ip, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
if err == nil && isIptablesClientAvailable(ip) {
major, minor, _ := ip.GetIptablesVersion()
// use iptables when its version is lower than 1.8.0 which doesn't work well with our nftables manager
if major < 1 || (major == 1 && minor < 8) {
return IPTABLES
}
useIPTABLES = true
iptablesChains, err = ip.ListChains("filter")
if err != nil {
log.Errorf("failed to list iptables chains: %s", err)
useIPTABLES = false
}
}
ip, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
if err != nil {
return UNKNOWN
nf := nftables.Conn{}
if chains, err := nf.ListChains(); err == nil && os.Getenv(SKIP_NFTABLES_ENV) != "true" {
if !useIPTABLES {
return NFTABLES
}
// search for chains where table is filter
// if we find one, we assume that nftables manager can be used with iptables
for _, chain := range chains {
if chain.Table.Name == "filter" {
return NFTABLES
}
}
// check tables for the following constraints:
// 1. there is no chain in nftables for the filter table and there is at least one chain in iptables, we assume that nftables manager can not be used
// 2. there is no tables or more than one table, we assume that nftables manager can be used
// 3. there is only one table and its name is filter, we assume that nftables manager can not be used, since there was no chain in it
// 4. if we find an error we log and continue with iptables check
nbTablesList, err := nf.ListTables()
switch {
case err == nil && len(iptablesChains) > 0:
return IPTABLES
case err == nil && len(nbTablesList) != 1:
return NFTABLES
case err == nil && len(nbTablesList) == 1 && nbTablesList[0].Name == "filter":
return IPTABLES
case err != nil:
log.Errorf("failed to list nftables tables on fw manager discovery: %s", err)
}
}
if isIptablesClientAvailable(ip) {
if useIPTABLES {
return IPTABLES
}

View File

@@ -6,6 +6,7 @@ import "github.com/netbirdio/netbird/iface"
type IFaceMapper interface {
Name() string
Address() iface.WGAddress
Address6() *iface.WGAddress
IsUserspaceBind() bool
SetFilter(iface.PacketFilter) error
}

View File

@@ -24,6 +24,14 @@ type Manager struct {
router *routerManager
}
func (m *Manager) ResetV6Firewall() error {
return nil
}
func (m *Manager) V6Active() bool {
return false
}
// iFaceMapper defines subset methods of interface required for manager
type iFaceMapper interface {
Name() string

View File

@@ -87,12 +87,12 @@ func (i *routerManager) InsertRoutingRules(pair firewall.RouterPair) error {
return nil
}
// insertRoutingRule inserts an iptable rule
// insertRoutingRule inserts an iptables rule
func (i *routerManager) insertRoutingRule(keyFormat, table, chain, jump string, pair firewall.RouterPair) error {
var err error
ruleKey := firewall.GenKey(keyFormat, pair.ID)
rule := genRuleSpec(jump, ruleKey, pair.Source, pair.Destination)
rule := genRuleSpec(jump, pair.Source, pair.Destination)
existingRule, found := i.rules[ruleKey]
if found {
err = i.iptablesClient.DeleteIfExists(table, chain, existingRule...)
@@ -326,9 +326,9 @@ func (i *routerManager) createChain(table, newChain string) error {
return nil
}
// genRuleSpec generates rule specification with comment identifier
func genRuleSpec(jump, id, source, destination string) []string {
return []string{"-s", source, "-d", destination, "-j", jump, "-m", "comment", "--comment", id}
// genRuleSpec generates rule specification
func genRuleSpec(jump, source, destination string) []string {
return []string{"-s", source, "-d", destination, "-j", jump}
}
func getIptablesRuleType(table string) string {

View File

@@ -51,14 +51,12 @@ func TestIptablesManager_RestoreOrCreateContainers(t *testing.T) {
Destination: "100.100.100.0/24",
Masquerade: true,
}
forward4RuleKey := firewall.GenKey(firewall.ForwardingFormat, pair.ID)
forward4Rule := genRuleSpec(routingFinalForwardJump, forward4RuleKey, pair.Source, pair.Destination)
forward4Rule := genRuleSpec(routingFinalForwardJump, pair.Source, pair.Destination)
err = manager.iptablesClient.Insert(tableFilter, chainRTFWD, 1, forward4Rule...)
require.NoError(t, err, "inserting rule should not return error")
nat4RuleKey := firewall.GenKey(firewall.NatFormat, pair.ID)
nat4Rule := genRuleSpec(routingFinalNatJump, nat4RuleKey, pair.Source, pair.Destination)
nat4Rule := genRuleSpec(routingFinalNatJump, pair.Source, pair.Destination)
err = manager.iptablesClient.Insert(tableNat, chainRTNAT, 1, nat4Rule...)
require.NoError(t, err, "inserting rule should not return error")
@@ -75,6 +73,9 @@ func TestIptablesManager_InsertRoutingRules(t *testing.T) {
for _, testCase := range test.InsertRuleTestCases {
t.Run(testCase.Name, func(t *testing.T) {
if testCase.IsV6 {
t.Skip("Environment does not support IPv6, skipping IPv6 test...")
}
iptablesClient, err := iptables.NewWithProtocol(iptables.ProtocolIPv4)
require.NoError(t, err, "failed to init iptables client")
@@ -92,7 +93,7 @@ func TestIptablesManager_InsertRoutingRules(t *testing.T) {
require.NoError(t, err, "forwarding pair should be inserted")
forwardRuleKey := firewall.GenKey(firewall.ForwardingFormat, testCase.InputPair.ID)
forwardRule := genRuleSpec(routingFinalForwardJump, forwardRuleKey, testCase.InputPair.Source, testCase.InputPair.Destination)
forwardRule := genRuleSpec(routingFinalForwardJump, testCase.InputPair.Source, testCase.InputPair.Destination)
exists, err := iptablesClient.Exists(tableFilter, chainRTFWD, forwardRule...)
require.NoError(t, err, "should be able to query the iptables %s table and %s chain", tableFilter, chainRTFWD)
@@ -103,7 +104,7 @@ func TestIptablesManager_InsertRoutingRules(t *testing.T) {
require.Equal(t, forwardRule[:4], foundRule[:4], "stored forwarding rule should match")
inForwardRuleKey := firewall.GenKey(firewall.InForwardingFormat, testCase.InputPair.ID)
inForwardRule := genRuleSpec(routingFinalForwardJump, inForwardRuleKey, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
inForwardRule := genRuleSpec(routingFinalForwardJump, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
exists, err = iptablesClient.Exists(tableFilter, chainRTFWD, inForwardRule...)
require.NoError(t, err, "should be able to query the iptables %s table and %s chain", tableFilter, chainRTFWD)
@@ -114,7 +115,7 @@ func TestIptablesManager_InsertRoutingRules(t *testing.T) {
require.Equal(t, inForwardRule[:4], foundRule[:4], "stored income forwarding rule should match")
natRuleKey := firewall.GenKey(firewall.NatFormat, testCase.InputPair.ID)
natRule := genRuleSpec(routingFinalNatJump, natRuleKey, testCase.InputPair.Source, testCase.InputPair.Destination)
natRule := genRuleSpec(routingFinalNatJump, testCase.InputPair.Source, testCase.InputPair.Destination)
exists, err = iptablesClient.Exists(tableNat, chainRTNAT, natRule...)
require.NoError(t, err, "should be able to query the iptables %s table and %s chain", tableNat, chainRTNAT)
@@ -130,7 +131,7 @@ func TestIptablesManager_InsertRoutingRules(t *testing.T) {
}
inNatRuleKey := firewall.GenKey(firewall.InNatFormat, testCase.InputPair.ID)
inNatRule := genRuleSpec(routingFinalNatJump, inNatRuleKey, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
inNatRule := genRuleSpec(routingFinalNatJump, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
exists, err = iptablesClient.Exists(tableNat, chainRTNAT, inNatRule...)
require.NoError(t, err, "should be able to query the iptables %s table and %s chain", tableNat, chainRTNAT)
@@ -156,6 +157,9 @@ func TestIptablesManager_RemoveRoutingRules(t *testing.T) {
for _, testCase := range test.RemoveRuleTestCases {
t.Run(testCase.Name, func(t *testing.T) {
if testCase.IsV6 {
t.Skip("Environment does not support IPv6, skipping IPv6 test...")
}
iptablesClient, _ := iptables.NewWithProtocol(iptables.ProtocolIPv4)
manager, err := newRouterManager(context.TODO(), iptablesClient)
@@ -167,25 +171,25 @@ func TestIptablesManager_RemoveRoutingRules(t *testing.T) {
require.NoError(t, err, "shouldn't return error")
forwardRuleKey := firewall.GenKey(firewall.ForwardingFormat, testCase.InputPair.ID)
forwardRule := genRuleSpec(routingFinalForwardJump, forwardRuleKey, testCase.InputPair.Source, testCase.InputPair.Destination)
forwardRule := genRuleSpec(routingFinalForwardJump, testCase.InputPair.Source, testCase.InputPair.Destination)
err = iptablesClient.Insert(tableFilter, chainRTFWD, 1, forwardRule...)
require.NoError(t, err, "inserting rule should not return error")
inForwardRuleKey := firewall.GenKey(firewall.InForwardingFormat, testCase.InputPair.ID)
inForwardRule := genRuleSpec(routingFinalForwardJump, inForwardRuleKey, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
inForwardRule := genRuleSpec(routingFinalForwardJump, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
err = iptablesClient.Insert(tableFilter, chainRTFWD, 1, inForwardRule...)
require.NoError(t, err, "inserting rule should not return error")
natRuleKey := firewall.GenKey(firewall.NatFormat, testCase.InputPair.ID)
natRule := genRuleSpec(routingFinalNatJump, natRuleKey, testCase.InputPair.Source, testCase.InputPair.Destination)
natRule := genRuleSpec(routingFinalNatJump, testCase.InputPair.Source, testCase.InputPair.Destination)
err = iptablesClient.Insert(tableNat, chainRTNAT, 1, natRule...)
require.NoError(t, err, "inserting rule should not return error")
inNatRuleKey := firewall.GenKey(firewall.InNatFormat, testCase.InputPair.ID)
inNatRule := genRuleSpec(routingFinalNatJump, inNatRuleKey, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
inNatRule := genRuleSpec(routingFinalNatJump, firewall.GetInPair(testCase.InputPair).Source, firewall.GetInPair(testCase.InputPair).Destination)
err = iptablesClient.Insert(tableNat, chainRTNAT, 1, inNatRule...)
require.NoError(t, err, "inserting rule should not return error")

View File

@@ -76,6 +76,13 @@ type Manager interface {
// RemoveRoutingRules removes a routing firewall rule
RemoveRoutingRules(pair RouterPair) error
// ResetV6Firewall makes changes to the firewall to adapt to the IP address changes.
// It is expected that after calling this method ApplyFiltering will be called to re-add the firewall rules.
ResetV6Firewall() error
// V6Active returns whether IPv6 rules should/may be created by upper layers.
V6Active() bool
// Reset firewall to the default state
Reset() error

File diff suppressed because it is too large Load Diff

View File

@@ -36,17 +36,25 @@ func Create(context context.Context, wgIface iFaceMapper) (*Manager, error) {
wgIface: wgIface,
}
workTable, err := m.createWorkTable()
workTable, err := m.createWorkTable(nftables.TableFamilyIPv4)
if err != nil {
return nil, err
}
m.router, err = newRouter(context, workTable)
var workTable6 *nftables.Table
if wgIface.Address6() != nil {
workTable6, err = m.createWorkTable(nftables.TableFamilyIPv6)
if err != nil {
return nil, err
}
}
m.router, err = newRouter(context, workTable, workTable6)
if err != nil {
return nil, err
}
m.aclManager, err = newAclManager(workTable, wgIface, m.router.RouteingFwChainName())
m.aclManager, err = newAclManager(workTable, workTable6, wgIface, m.router.RouteingFwChainName())
if err != nil {
return nil, err
}
@@ -54,6 +62,54 @@ func Create(context context.Context, wgIface iFaceMapper) (*Manager, error) {
return m, nil
}
// Resets the IPv6 Firewall Table to adapt to changes in IP addresses
func (m *Manager) ResetV6Firewall() error {
// First, prepare reset by deleting all currently active rules.
workTable6, err := m.aclManager.PrepareV6Reset()
if err != nil {
return err
}
// Depending on whether we now have an IPv6 address, we now either have to create/empty an IPv6 table, or delete it.
if m.wgIface.Address6() != nil {
if workTable6 != nil {
m.rConn.FlushTable(workTable6)
} else {
workTable6, err = m.createWorkTable(nftables.TableFamilyIPv6)
if err != nil {
return err
}
}
} else {
m.rConn.DelTable(workTable6)
workTable6 = nil
}
err = m.rConn.Flush()
if err != nil {
return err
}
// Restore routing rules.
err = m.router.RestoreAfterV6Reset(workTable6)
if err != nil {
return err
}
// Restore basic firewall chains (needs to happen after routes because chains from router must exist).
// Does not restore rules (will be done later during the update, when UpdateFiltering will be called at some point)
err = m.aclManager.ReinitAfterV6Reset(workTable6)
if err != nil {
return err
}
return m.rConn.Flush()
}
func (m *Manager) V6Active() bool {
return m.aclManager.v6Active
}
// AddFiltering rule to the firewall
//
// If comment argument is empty firewall manager should set
@@ -72,7 +128,7 @@ func (m *Manager) AddFiltering(
defer m.mutex.Unlock()
rawIP := ip.To4()
if rawIP == nil {
if rawIP == nil && m.wgIface.Address6() == nil {
return nil, fmt.Errorf("unsupported IP version: %s", ip.String())
}
@@ -114,6 +170,8 @@ func (m *Manager) AllowNetbird() error {
m.mutex.Lock()
defer m.mutex.Unlock()
// Note for devs: When adding IPv6 support to uspfilter, the implementation of createDefaultAllowRules()
// must be adjusted to include IPv6 rules.
err := m.aclManager.createDefaultAllowRules()
if err != nil {
return fmt.Errorf("failed to create default allow rules: %v", err)
@@ -211,8 +269,8 @@ func (m *Manager) Flush() error {
return m.aclManager.Flush()
}
func (m *Manager) createWorkTable() (*nftables.Table, error) {
tables, err := m.rConn.ListTablesOfFamily(nftables.TableFamilyIPv4)
func (m *Manager) createWorkTable(tableFamily nftables.TableFamily) (*nftables.Table, error) {
tables, err := m.rConn.ListTablesOfFamily(tableFamily)
if err != nil {
return nil, fmt.Errorf("list of tables: %w", err)
}
@@ -223,7 +281,7 @@ func (m *Manager) createWorkTable() (*nftables.Table, error) {
}
}
table := m.rConn.AddTable(&nftables.Table{Name: tableName, Family: nftables.TableFamilyIPv4})
table := m.rConn.AddTable(&nftables.Table{Name: tableName, Family: tableFamily})
err = m.rConn.Flush()
return table, err
}

View File

@@ -19,8 +19,9 @@ import (
// iFaceMapper defines subset methods of interface required for manager
type iFaceMock struct {
NameFunc func() string
AddressFunc func() iface.WGAddress
NameFunc func() string
AddressFunc func() iface.WGAddress
Address6Func func() *iface.WGAddress
}
func (i *iFaceMock) Name() string {
@@ -37,6 +38,13 @@ func (i *iFaceMock) Address() iface.WGAddress {
panic("AddressFunc is not set")
}
func (i *iFaceMock) Address6() *iface.WGAddress {
if i.Address6Func != nil {
return i.Address6Func()
}
panic("AddressFunc is not set")
}
func (i *iFaceMock) IsUserspaceBind() bool { return false }
func TestNftablesManager(t *testing.T) {
@@ -53,6 +61,7 @@ func TestNftablesManager(t *testing.T) {
},
}
},
Address6Func: func() *iface.WGAddress { return nil },
}
// just check on the local interface
@@ -99,11 +108,9 @@ func TestNftablesManager(t *testing.T) {
Register: 1,
Data: ifname("lo"),
},
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: uint32(9),
Len: uint32(1),
&expr.Meta{
Key: expr.MetaKeyL4PROTO,
Register: 1,
},
&expr.Cmp{
Register: 1,
@@ -152,6 +159,370 @@ func TestNftablesManager(t *testing.T) {
require.NoError(t, err, "failed to reset")
}
func TestNftablesManager6Disabled(t *testing.T) {
mock := &iFaceMock{
NameFunc: func() string {
return "lo"
},
AddressFunc: func() iface.WGAddress {
return iface.WGAddress{
IP: net.ParseIP("100.96.0.1"),
Network: &net.IPNet{
IP: net.ParseIP("100.96.0.0"),
Mask: net.IPv4Mask(255, 255, 255, 0),
},
}
},
Address6Func: func() *iface.WGAddress { return nil },
}
// just check on the local interface
manager, err := Create(context.Background(), mock)
require.NoError(t, err)
time.Sleep(time.Second * 3)
defer func() {
err = manager.Reset()
require.NoError(t, err, "failed to reset")
time.Sleep(time.Second)
}()
ip := net.ParseIP("2001:db8::fedc:ba09:8765:4321")
testClient := &nftables.Conn{}
_, err = manager.AddFiltering(
ip,
fw.ProtocolTCP,
nil,
&fw.Port{Values: []int{53}},
fw.RuleDirectionIN,
fw.ActionDrop,
"",
"",
)
require.Error(t, err, "IPv6 rule should not be added when IPv6 is disabled")
err = manager.Flush()
require.NoError(t, err, "failed to flush")
rules, err := testClient.GetRules(manager.aclManager.workTable, manager.aclManager.chainInputRules)
require.NoError(t, err, "failed to get rules")
require.Len(t, rules, 0, "expected no rules")
err = manager.Reset()
require.NoError(t, err, "failed to reset")
}
func TestNftablesManager6(t *testing.T) {
if !iface.SupportsIPv6() {
t.Skip("Environment does not support IPv6, skipping IPv6 test...")
}
mock := &iFaceMock{
NameFunc: func() string {
return "lo"
},
AddressFunc: func() iface.WGAddress {
return iface.WGAddress{
IP: net.ParseIP("100.96.0.1"),
Network: &net.IPNet{
IP: net.ParseIP("100.96.0.0"),
Mask: net.IPv4Mask(255, 255, 255, 0),
},
}
},
Address6Func: func() *iface.WGAddress {
return &iface.WGAddress{
IP: net.ParseIP("2001:db8::0123:4567:890a:bcde"),
Network: &net.IPNet{
IP: net.ParseIP("2001:db8::"),
Mask: net.CIDRMask(64, 128),
},
}
},
}
// just check on the local interface
manager, err := Create(context.Background(), mock)
require.NoError(t, err)
time.Sleep(time.Second * 3)
defer func() {
err = manager.Reset()
require.NoError(t, err, "failed to reset")
time.Sleep(time.Second)
}()
require.True(t, manager.V6Active(), "IPv6 is not active even though it should be.")
ip := net.ParseIP("2001:db8::fedc:ba09:8765:4321")
testClient := &nftables.Conn{}
rule, err := manager.AddFiltering(
ip,
fw.ProtocolTCP,
nil,
&fw.Port{Values: []int{53}},
fw.RuleDirectionIN,
fw.ActionDrop,
"",
"",
)
require.NoError(t, err, "failed to add rule")
err = manager.Flush()
require.NoError(t, err, "failed to flush")
rules, err := testClient.GetRules(manager.aclManager.workTable6, manager.aclManager.chainInputRules6)
require.NoError(t, err, "failed to get rules")
require.Len(t, rules, 1, "expected 1 rules")
ipToAdd, _ := netip.AddrFromSlice(ip)
add := ipToAdd.Unmap()
expectedExprs := []expr.Any{
&expr.Meta{Key: expr.MetaKeyIIFNAME, Register: 1},
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: ifname("lo"),
},
&expr.Meta{
Key: expr.MetaKeyL4PROTO,
Register: 1,
},
&expr.Cmp{
Register: 1,
Op: expr.CmpOpEq,
Data: []byte{unix.IPPROTO_TCP},
},
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: 8,
Len: 16,
},
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: add.AsSlice(),
},
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseTransportHeader,
Offset: 2,
Len: 2,
},
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: []byte{0, 53},
},
&expr.Verdict{Kind: expr.VerdictDrop},
}
require.ElementsMatch(t, rules[0].Exprs, expectedExprs, "expected the same expressions")
for _, r := range rule {
err = manager.DeleteRule(r)
require.NoError(t, err, "failed to delete rule")
}
err = manager.Flush()
require.NoError(t, err, "failed to flush")
rules, err = testClient.GetRules(manager.aclManager.workTable6, manager.aclManager.chainInputRules6)
require.NoError(t, err, "failed to get rules")
require.Len(t, rules, 0, "expected 0 rules after deletion")
err = manager.Reset()
require.NoError(t, err, "failed to reset")
}
func TestNftablesManagerAddressReset6(t *testing.T) {
if !iface.SupportsIPv6() {
t.Skip("Environment does not support IPv6, skipping IPv6 test...")
}
mock := &iFaceMock{
NameFunc: func() string {
return "lo"
},
AddressFunc: func() iface.WGAddress {
return iface.WGAddress{
IP: net.ParseIP("100.96.0.1"),
Network: &net.IPNet{
IP: net.ParseIP("100.96.0.0"),
Mask: net.IPv4Mask(255, 255, 255, 0),
},
}
},
Address6Func: func() *iface.WGAddress {
return &iface.WGAddress{
IP: net.ParseIP("2001:db8::0123:4567:890a:bcde"),
Network: &net.IPNet{
IP: net.ParseIP("2001:db8::"),
Mask: net.CIDRMask(64, 128),
},
}
},
}
// just check on the local interface
manager, err := Create(context.Background(), mock)
require.NoError(t, err)
time.Sleep(time.Second * 3)
defer func() {
err = manager.Reset()
require.NoError(t, err, "failed to reset")
time.Sleep(time.Second)
}()
require.True(t, manager.V6Active(), "IPv6 is not active even though it should be.")
ip := net.ParseIP("2001:db8::fedc:ba09:8765:4321")
testClient := &nftables.Conn{}
_, err = manager.AddFiltering(
ip,
fw.ProtocolTCP,
nil,
&fw.Port{Values: []int{53}},
fw.RuleDirectionIN,
fw.ActionDrop,
"",
"",
)
require.NoError(t, err, "failed to add rule")
err = manager.Flush()
require.NoError(t, err, "failed to flush")
rules, err := testClient.GetRules(manager.aclManager.workTable6, manager.aclManager.chainInputRules6)
require.NoError(t, err, "failed to get rules")
require.Len(t, rules, 1, "expected 1 rules")
mock.Address6Func = func() *iface.WGAddress {
return nil
}
err = manager.ResetV6Firewall()
require.NoError(t, err, "failed to reset IPv6 firewall")
err = manager.Flush()
require.NoError(t, err, "failed to flush")
require.False(t, manager.V6Active(), "IPv6 is active even though it shouldn't be.")
tables, err := testClient.ListTablesOfFamily(nftables.TableFamilyIPv6)
require.NoError(t, err, "failed to list IPv6 tables")
for _, table := range tables {
if table.Name == tableName {
t.Errorf("When IPv6 is disabled, the netbird table should not exist.")
}
}
mock.Address6Func = func() *iface.WGAddress {
return &iface.WGAddress{
IP: net.ParseIP("2001:db8::0123:4567:890a:bcdf"),
Network: &net.IPNet{
IP: net.ParseIP("2001:db8::"),
Mask: net.CIDRMask(64, 128),
},
}
}
err = manager.ResetV6Firewall()
require.NoError(t, err, "failed to reset IPv6 firewall")
require.True(t, manager.V6Active(), "IPv6 is not active even though it should be.")
rule, err := manager.AddFiltering(
ip,
fw.ProtocolTCP,
nil,
&fw.Port{Values: []int{53}},
fw.RuleDirectionIN,
fw.ActionDrop,
"",
"",
)
require.NoError(t, err, "failed to add rule")
err = manager.Flush()
require.NoError(t, err, "failed to flush")
rules, err = testClient.GetRules(manager.aclManager.workTable6, manager.aclManager.chainInputRules6)
require.NoError(t, err, "failed to get rules")
require.Len(t, rules, 1, "expected 1 rule")
ipToAdd, _ := netip.AddrFromSlice(ip)
add := ipToAdd.Unmap()
expectedExprs := []expr.Any{
&expr.Meta{Key: expr.MetaKeyIIFNAME, Register: 1},
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: ifname("lo"),
},
&expr.Meta{
Key: expr.MetaKeyL4PROTO,
Register: 1,
},
&expr.Cmp{
Register: 1,
Op: expr.CmpOpEq,
Data: []byte{unix.IPPROTO_TCP},
},
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: 8,
Len: 16,
},
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: add.AsSlice(),
},
&expr.Payload{
DestRegister: 1,
Base: expr.PayloadBaseTransportHeader,
Offset: 2,
Len: 2,
},
&expr.Cmp{
Op: expr.CmpOpEq,
Register: 1,
Data: []byte{0, 53},
},
&expr.Verdict{Kind: expr.VerdictDrop},
}
require.ElementsMatch(t, rules[0].Exprs, expectedExprs, "expected the same expressions")
for _, r := range rule {
err = manager.DeleteRule(r)
require.NoError(t, err, "failed to delete rule")
}
err = manager.Flush()
require.NoError(t, err, "failed to flush")
rules, err = testClient.GetRules(manager.aclManager.workTable6, manager.aclManager.chainInputRules6)
require.NoError(t, err, "failed to get rules")
require.Len(t, rules, 0, "expected 0 rules after deletion")
err = manager.Reset()
require.NoError(t, err, "failed to reset")
}
func TestNFtablesCreatePerformance(t *testing.T) {
mock := &iFaceMock{
NameFunc: func() string {
@@ -166,6 +537,16 @@ func TestNFtablesCreatePerformance(t *testing.T) {
},
}
},
Address6Func: func() *iface.WGAddress {
v6addr, v6net, _ := net.ParseCIDR("fd00:1234:dead:beef::1/64")
return &iface.WGAddress{
IP: v6addr,
Network: &net.IPNet{
IP: v6net.IP,
Mask: v6net.Mask,
},
}
},
}
for _, testMax := range []int{10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000} {

View File

@@ -26,7 +26,8 @@ const (
// some presets for building nftable rules
var (
zeroXor = binaryutil.NativeEndian.PutUint32(0)
zeroXor = binaryutil.NativeEndian.PutUint32(0)
zeroXor6 = []byte{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
exprCounterAccept = []expr.Any{
&expr.Counter{},
@@ -39,48 +40,69 @@ var (
)
type router struct {
ctx context.Context
stop context.CancelFunc
conn *nftables.Conn
workTable *nftables.Table
filterTable *nftables.Table
chains map[string]*nftables.Chain
ctx context.Context
stop context.CancelFunc
conn *nftables.Conn
workTable *nftables.Table
workTable6 *nftables.Table
filterTable *nftables.Table
filterTable6 *nftables.Table
chains map[string]*nftables.Chain
chains6 map[string]*nftables.Chain
// rules is useful to avoid duplicates and to get missing attributes that we don't have when adding new rules
rules map[string]*nftables.Rule
isDefaultFwdRulesEnabled bool
rules map[string]*nftables.Rule
rules6 map[string]*nftables.Rule
isDefaultFwdRulesEnabled bool
isDefaultFwdRulesEnabled6 bool
}
func newRouter(parentCtx context.Context, workTable *nftables.Table) (*router, error) {
func newRouter(parentCtx context.Context, workTable *nftables.Table, workTable6 *nftables.Table) (*router, error) {
ctx, cancel := context.WithCancel(parentCtx)
r := &router{
ctx: ctx,
stop: cancel,
conn: &nftables.Conn{},
workTable: workTable,
chains: make(map[string]*nftables.Chain),
rules: make(map[string]*nftables.Rule),
ctx: ctx,
stop: cancel,
conn: &nftables.Conn{},
workTable: workTable,
workTable6: workTable6,
chains: make(map[string]*nftables.Chain),
chains6: make(map[string]*nftables.Chain),
rules: make(map[string]*nftables.Rule),
rules6: make(map[string]*nftables.Rule),
}
var err error
r.filterTable, err = r.loadFilterTable()
r.filterTable, r.filterTable6, err = r.loadFilterTables()
if err != nil {
if errors.Is(err, errFilterTableNotFound) {
log.Warnf("table 'filter' not found for forward rules")
log.Warnf("table 'filter' not found for forward rules for one of the supported address families-")
} else {
return nil, err
}
}
err = r.cleanUpDefaultForwardRules()
err = r.cleanUpDefaultForwardRules(false)
if err != nil {
log.Errorf("failed to clean up rules from FORWARD chain: %s", err)
}
err = r.createContainers()
err = r.cleanUpDefaultForwardRules(true)
if err != nil {
log.Errorf("failed to clean up rules from IPv6 FORWARD chain: %s", err)
}
err = r.createContainers(false)
if err != nil {
log.Errorf("failed to create containers for route: %s", err)
}
if r.workTable6 != nil {
err = r.createContainers(true)
if err != nil {
log.Errorf("failed to create v6 containers for route: %s", err)
}
}
return r, err
}
@@ -90,43 +112,99 @@ func (r *router) RouteingFwChainName() string {
// ResetForwardRules cleans existing nftables default forward rules from the system
func (r *router) ResetForwardRules() {
err := r.cleanUpDefaultForwardRules()
err := r.cleanUpDefaultForwardRules(false)
if err != nil {
log.Errorf("failed to reset forward rules: %s", err)
}
err = r.cleanUpDefaultForwardRules(true)
if err != nil {
log.Errorf("failed to reset forward rules: %s", err)
}
}
func (r *router) loadFilterTable() (*nftables.Table, error) {
func (r *router) RestoreAfterV6Reset(newWorktable6 *nftables.Table) error {
r.workTable6 = newWorktable6
if newWorktable6 != nil {
err := r.cleanUpDefaultForwardRules(true)
if err != nil {
log.Errorf("failed to clean up rules from IPv6 FORWARD chain: %s", err)
}
err = r.createContainers(true)
if err != nil {
return err
}
for name, rule := range r.rules6 {
rule = &nftables.Rule{
Table: r.workTable6,
Chain: r.chains6[rule.Chain.Name],
Exprs: rule.Exprs,
UserData: rule.UserData,
}
r.rules6[name] = r.conn.AddRule(rule)
}
}
return r.conn.Flush()
}
func (r *router) loadFilterTables() (*nftables.Table, *nftables.Table, error) {
tables, err := r.conn.ListTablesOfFamily(nftables.TableFamilyIPv4)
if err != nil {
return nil, fmt.Errorf("nftables: unable to list tables: %v", err)
return nil, nil, fmt.Errorf("nftables: unable to list tables: %v", err)
}
var table4 *nftables.Table = nil
for _, table := range tables {
if table.Name == "filter" {
return table, nil
table4 = table
break
}
}
return nil, errFilterTableNotFound
var table6 *nftables.Table = nil
tables, err = r.conn.ListTablesOfFamily(nftables.TableFamilyIPv6)
if err != nil {
return nil, nil, fmt.Errorf("nftables: unable to list tables: %v", err)
}
for _, table := range tables {
if table.Name == "filter" {
table6 = table
break
}
}
err = nil
if table4 == nil || table6 == nil {
err = errFilterTableNotFound
}
return table4, table6, err
}
func (r *router) createContainers() error {
func (r *router) createContainers(forV6 bool) error {
workTable := r.workTable
chainStorage := r.chains
if forV6 {
workTable = r.workTable6
chainStorage = r.chains6
}
r.chains[chainNameRouteingFw] = r.conn.AddChain(&nftables.Chain{
chainStorage[chainNameRouteingFw] = r.conn.AddChain(&nftables.Chain{
Name: chainNameRouteingFw,
Table: r.workTable,
Table: workTable,
})
r.chains[chainNameRoutingNat] = r.conn.AddChain(&nftables.Chain{
chainStorage[chainNameRoutingNat] = r.conn.AddChain(&nftables.Chain{
Name: chainNameRoutingNat,
Table: r.workTable,
Table: workTable,
Hooknum: nftables.ChainHookPostrouting,
Priority: nftables.ChainPriorityNATSource - 1,
Type: nftables.ChainTypeNAT,
})
err := r.refreshRulesMap()
err := r.refreshRulesMap(forV6)
if err != nil {
log.Errorf("failed to clean up rules from FORWARD chain: %s", err)
}
@@ -140,7 +218,13 @@ func (r *router) createContainers() error {
// InsertRoutingRules inserts a nftable rule pair to the forwarding chain and if enabled, to the nat chain
func (r *router) InsertRoutingRules(pair manager.RouterPair) error {
err := r.refreshRulesMap()
parsedIp, _, _ := net.ParseCIDR(pair.Source)
if parsedIp.To4() == nil && r.workTable6 == nil {
return fmt.Errorf("nftables: attempted to add IPv6 routing rule even though IPv6 is not enabled for this host")
}
err := r.refreshRulesMap(parsedIp.To4() == nil)
if err != nil {
return err
}
@@ -165,7 +249,11 @@ func (r *router) InsertRoutingRules(pair manager.RouterPair) error {
}
}
if r.filterTable != nil && !r.isDefaultFwdRulesEnabled {
filterTable := r.filterTable
if parsedIp.To4() == nil {
filterTable = r.filterTable6
}
if filterTable != nil && !r.isDefaultFwdRulesEnabled {
log.Debugf("add default accept forward rule")
r.acceptForwardRule(pair.Source)
}
@@ -191,7 +279,13 @@ func (r *router) insertRoutingRule(format, chainName string, pair manager.Router
ruleKey := manager.GenKey(format, pair.ID)
_, exists := r.rules[ruleKey]
parsedIp, _, _ := net.ParseCIDR(pair.Source)
rules := r.rules
if parsedIp.To4() == nil {
rules = r.rules6
}
_, exists := rules[ruleKey]
if exists {
err := r.removeRoutingRule(format, pair)
if err != nil {
@@ -199,18 +293,35 @@ func (r *router) insertRoutingRule(format, chainName string, pair manager.Router
}
}
r.rules[ruleKey] = r.conn.InsertRule(&nftables.Rule{
Table: r.workTable,
Chain: r.chains[chainName],
table, chain := r.workTable, r.chains[chainName]
if parsedIp.To4() == nil {
table, chain = r.workTable6, r.chains6[chainName]
}
newRule := r.conn.InsertRule(&nftables.Rule{
Table: table,
Chain: chain,
Exprs: expression,
UserData: []byte(ruleKey),
})
if parsedIp.To4() == nil {
r.rules[ruleKey] = newRule
} else {
r.rules6[ruleKey] = newRule
}
return nil
}
func (r *router) acceptForwardRule(sourceNetwork string) {
src := generateCIDRMatcherExpressions(true, sourceNetwork)
dst := generateCIDRMatcherExpressions(false, "0.0.0.0/0")
table := r.filterTable
parsedIp, _, _ := net.ParseCIDR(sourceNetwork)
if parsedIp.To4() == nil {
dst = generateCIDRMatcherExpressions(false, "::/0")
table = r.filterTable6
}
var exprs []expr.Any
exprs = append(src, append(dst, &expr.Verdict{ // nolint:gocritic
@@ -218,10 +329,10 @@ func (r *router) acceptForwardRule(sourceNetwork string) {
})...)
rule := &nftables.Rule{
Table: r.filterTable,
Table: table,
Chain: &nftables.Chain{
Name: "FORWARD",
Table: r.filterTable,
Table: table,
Type: nftables.ChainTypeFilter,
Hooknum: nftables.ChainHookForward,
Priority: nftables.ChainPriorityFilter,
@@ -233,6 +344,9 @@ func (r *router) acceptForwardRule(sourceNetwork string) {
r.conn.AddRule(rule)
src = generateCIDRMatcherExpressions(true, "0.0.0.0/0")
if parsedIp.To4() == nil {
src = generateCIDRMatcherExpressions(true, "::/0")
}
dst = generateCIDRMatcherExpressions(false, sourceNetwork)
exprs = append(src, append(dst, &expr.Verdict{ //nolint:gocritic
@@ -240,10 +354,10 @@ func (r *router) acceptForwardRule(sourceNetwork string) {
})...)
rule = &nftables.Rule{
Table: r.filterTable,
Table: table,
Chain: &nftables.Chain{
Name: "FORWARD",
Table: r.filterTable,
Table: table,
Type: nftables.ChainTypeFilter,
Hooknum: nftables.ChainHookForward,
Priority: nftables.ChainPriorityFilter,
@@ -252,12 +366,21 @@ func (r *router) acceptForwardRule(sourceNetwork string) {
UserData: []byte(userDataAcceptForwardRuleDst),
}
r.conn.AddRule(rule)
r.isDefaultFwdRulesEnabled = true
if parsedIp.To4() == nil {
r.isDefaultFwdRulesEnabled6 = true
} else {
r.isDefaultFwdRulesEnabled = true
}
}
// RemoveRoutingRules removes a nftable rule pair from forwarding and nat chains
func (r *router) RemoveRoutingRules(pair manager.RouterPair) error {
err := r.refreshRulesMap()
parsedIp, _, _ := net.ParseCIDR(pair.Source)
if parsedIp.To4() == nil && r.workTable6 == nil {
return fmt.Errorf("nftables: attempted to remove IPv6 routing rule even though IPv6 is not enabled for this host")
}
err := r.refreshRulesMap(parsedIp.To4() == nil)
if err != nil {
return err
}
@@ -282,8 +405,12 @@ func (r *router) RemoveRoutingRules(pair manager.RouterPair) error {
return err
}
if len(r.rules) == 0 {
err := r.cleanUpDefaultForwardRules()
rulesList := r.rules
if parsedIp.To4() == nil {
rulesList = r.rules6
}
if len(rulesList) == 0 {
err := r.cleanUpDefaultForwardRules(parsedIp.To4() == nil)
if err != nil {
log.Errorf("failed to clean up rules from FORWARD chain: %s", err)
}
@@ -301,7 +428,13 @@ func (r *router) RemoveRoutingRules(pair manager.RouterPair) error {
func (r *router) removeRoutingRule(format string, pair manager.RouterPair) error {
ruleKey := manager.GenKey(format, pair.ID)
rule, found := r.rules[ruleKey]
parsedIp, _, _ := net.ParseCIDR(pair.Source)
rules := r.rules
if parsedIp.To4() == nil {
rules = r.rules6
}
rule, found := rules[ruleKey]
if found {
ruleType := "forwarding"
if rule.Chain.Type == nftables.ChainTypeNAT {
@@ -315,49 +448,68 @@ func (r *router) removeRoutingRule(format string, pair manager.RouterPair) error
log.Debugf("nftables: removing %s rule for %s", ruleType, pair.Destination)
delete(r.rules, ruleKey)
delete(rules, ruleKey)
}
return nil
}
// refreshRulesMap refreshes the rule map with the latest rules. this is useful to avoid
// duplicates and to get missing attributes that we don't have when adding new rules
func (r *router) refreshRulesMap() error {
for _, chain := range r.chains {
func (r *router) refreshRulesMap(forV6 bool) error {
chainList := r.chains
if forV6 {
chainList = r.chains6
}
for _, chain := range chainList {
rules, err := r.conn.GetRules(chain.Table, chain)
if err != nil {
return fmt.Errorf("nftables: unable to list rules: %v", err)
}
for _, rule := range rules {
if len(rule.UserData) > 0 {
r.rules[string(rule.UserData)] = rule
if forV6 {
r.rules6[string(rule.UserData)] = rule
} else {
r.rules[string(rule.UserData)] = rule
}
}
}
}
return nil
}
func (r *router) cleanUpDefaultForwardRules() error {
if r.filterTable == nil {
r.isDefaultFwdRulesEnabled = false
func (r *router) cleanUpDefaultForwardRules(forV6 bool) error {
tableFamily := nftables.TableFamilyIPv4
filterTable := r.filterTable
if forV6 {
tableFamily = nftables.TableFamilyIPv6
filterTable = r.filterTable6
}
if filterTable == nil {
if forV6 {
r.isDefaultFwdRulesEnabled6 = false
} else {
r.isDefaultFwdRulesEnabled = false
}
return nil
}
chains, err := r.conn.ListChainsOfTableFamily(nftables.TableFamilyIPv4)
chains, err := r.conn.ListChainsOfTableFamily(tableFamily)
if err != nil {
return err
}
var rules []*nftables.Rule
for _, chain := range chains {
if chain.Table.Name != r.filterTable.Name {
if chain.Table.Name != filterTable.Name {
continue
}
if chain.Name != "FORWARD" {
continue
}
rules, err = r.conn.GetRules(r.filterTable, chain)
rules, err = r.conn.GetRules(filterTable, chain)
if err != nil {
return err
}
@@ -371,7 +523,12 @@ func (r *router) cleanUpDefaultForwardRules() error {
}
}
}
r.isDefaultFwdRulesEnabled = false
if forV6 {
r.isDefaultFwdRulesEnabled6 = false
} else {
r.isDefaultFwdRulesEnabled = false
}
return r.conn.Flush()
}
@@ -387,6 +544,18 @@ func generateCIDRMatcherExpressions(source bool, cidr string) []expr.Any {
} else {
offSet = 16 // dst offset
}
addrLen := uint32(4)
zeroXor := zeroXor
if ip.To4() == nil {
if source {
offSet = 8 // src offset
} else {
offSet = 24 // dst offset
}
addrLen = 16
zeroXor = zeroXor6
}
return []expr.Any{
// fetch src add
@@ -394,13 +563,13 @@ func generateCIDRMatcherExpressions(source bool, cidr string) []expr.Any {
DestRegister: 1,
Base: expr.PayloadBaseNetworkHeader,
Offset: offSet,
Len: 4,
Len: addrLen,
},
// net mask
&expr.Bitwise{
DestRegister: 1,
SourceRegister: 1,
Len: 4,
Len: addrLen,
Mask: network.Mask,
Xor: zeroXor,
},

View File

@@ -4,6 +4,7 @@ package nftables
import (
"context"
"github.com/netbirdio/netbird/iface"
"testing"
"github.com/coreos/go-iptables/iptables"
@@ -29,16 +30,19 @@ func TestNftablesManager_InsertRoutingRules(t *testing.T) {
t.Skip("nftables not supported on this OS")
}
table, err := createWorkTable()
table, table6, err := createWorkTables()
if err != nil {
t.Fatal(err)
}
defer deleteWorkTable()
defer deleteWorkTables()
for _, testCase := range test.InsertRuleTestCases {
t.Run(testCase.Name, func(t *testing.T) {
manager, err := newRouter(context.TODO(), table)
if testCase.IsV6 && table6 == nil {
t.Skip("Environment does not support IPv6, skipping IPv6 test...")
}
manager, err := newRouter(context.TODO(), table, table6)
require.NoError(t, err, "failed to create router")
nftablesTestingClient := &nftables.Conn{}
@@ -58,8 +62,13 @@ func TestNftablesManager_InsertRoutingRules(t *testing.T) {
testingExpression := append(sourceExp, destExp...) //nolint:gocritic
fwdRuleKey := firewall.GenKey(firewall.ForwardingFormat, testCase.InputPair.ID)
chains := manager.chains
if testCase.IsV6 {
chains = manager.chains6
}
found := 0
for _, chain := range manager.chains {
for _, chain := range chains {
rules, err := nftablesTestingClient.GetRules(chain.Table, chain)
require.NoError(t, err, "should list rules for %s table and %s chain", chain.Table.Name, chain.Name)
for _, rule := range rules {
@@ -75,7 +84,7 @@ func TestNftablesManager_InsertRoutingRules(t *testing.T) {
if testCase.InputPair.Masquerade {
natRuleKey := firewall.GenKey(firewall.NatFormat, testCase.InputPair.ID)
found := 0
for _, chain := range manager.chains {
for _, chain := range chains {
rules, err := nftablesTestingClient.GetRules(chain.Table, chain)
require.NoError(t, err, "should list rules for %s table and %s chain", chain.Table.Name, chain.Name)
for _, rule := range rules {
@@ -94,7 +103,7 @@ func TestNftablesManager_InsertRoutingRules(t *testing.T) {
inFwdRuleKey := firewall.GenKey(firewall.InForwardingFormat, testCase.InputPair.ID)
found = 0
for _, chain := range manager.chains {
for _, chain := range chains {
rules, err := nftablesTestingClient.GetRules(chain.Table, chain)
require.NoError(t, err, "should list rules for %s table and %s chain", chain.Table.Name, chain.Name)
for _, rule := range rules {
@@ -110,7 +119,7 @@ func TestNftablesManager_InsertRoutingRules(t *testing.T) {
if testCase.InputPair.Masquerade {
inNatRuleKey := firewall.GenKey(firewall.InNatFormat, testCase.InputPair.ID)
found := 0
for _, chain := range manager.chains {
for _, chain := range chains {
rules, err := nftablesTestingClient.GetRules(chain.Table, chain)
require.NoError(t, err, "should list rules for %s table and %s chain", chain.Table.Name, chain.Name)
for _, rule := range rules {
@@ -131,16 +140,19 @@ func TestNftablesManager_RemoveRoutingRules(t *testing.T) {
t.Skip("nftables not supported on this OS")
}
table, err := createWorkTable()
table, table6, err := createWorkTables()
if err != nil {
t.Fatal(err)
}
defer deleteWorkTable()
defer deleteWorkTables()
for _, testCase := range test.RemoveRuleTestCases {
t.Run(testCase.Name, func(t *testing.T) {
manager, err := newRouter(context.TODO(), table)
if testCase.IsV6 && table6 == nil {
t.Skip("Environment does not support IPv6, skipping IPv6 test...")
}
manager, err := newRouter(context.TODO(), table, table6)
require.NoError(t, err, "failed to create router")
nftablesTestingClient := &nftables.Conn{}
@@ -150,11 +162,18 @@ func TestNftablesManager_RemoveRoutingRules(t *testing.T) {
sourceExp := generateCIDRMatcherExpressions(true, testCase.InputPair.Source)
destExp := generateCIDRMatcherExpressions(false, testCase.InputPair.Destination)
chains := manager.chains
workTable := table
if testCase.IsV6 {
chains = manager.chains6
workTable = table6
}
forwardExp := append(sourceExp, append(destExp, exprCounterAccept...)...) //nolint:gocritic
forwardRuleKey := firewall.GenKey(firewall.ForwardingFormat, testCase.InputPair.ID)
insertedForwarding := nftablesTestingClient.InsertRule(&nftables.Rule{
Table: manager.workTable,
Chain: manager.chains[chainNameRouteingFw],
Table: workTable,
Chain: chains[chainNameRouteingFw],
Exprs: forwardExp,
UserData: []byte(forwardRuleKey),
})
@@ -163,8 +182,8 @@ func TestNftablesManager_RemoveRoutingRules(t *testing.T) {
natRuleKey := firewall.GenKey(firewall.NatFormat, testCase.InputPair.ID)
insertedNat := nftablesTestingClient.InsertRule(&nftables.Rule{
Table: manager.workTable,
Chain: manager.chains[chainNameRoutingNat],
Table: workTable,
Chain: chains[chainNameRoutingNat],
Exprs: natExp,
UserData: []byte(natRuleKey),
})
@@ -175,8 +194,8 @@ func TestNftablesManager_RemoveRoutingRules(t *testing.T) {
forwardExp = append(sourceExp, append(destExp, exprCounterAccept...)...) //nolint:gocritic
inForwardRuleKey := firewall.GenKey(firewall.InForwardingFormat, testCase.InputPair.ID)
insertedInForwarding := nftablesTestingClient.InsertRule(&nftables.Rule{
Table: manager.workTable,
Chain: manager.chains[chainNameRouteingFw],
Table: workTable,
Chain: chains[chainNameRouteingFw],
Exprs: forwardExp,
UserData: []byte(inForwardRuleKey),
})
@@ -185,8 +204,8 @@ func TestNftablesManager_RemoveRoutingRules(t *testing.T) {
inNatRuleKey := firewall.GenKey(firewall.InNatFormat, testCase.InputPair.ID)
insertedInNat := nftablesTestingClient.InsertRule(&nftables.Rule{
Table: manager.workTable,
Chain: manager.chains[chainNameRoutingNat],
Table: workTable,
Chain: chains[chainNameRoutingNat],
Exprs: natExp,
UserData: []byte(inNatRuleKey),
})
@@ -199,7 +218,7 @@ func TestNftablesManager_RemoveRoutingRules(t *testing.T) {
err = manager.RemoveRoutingRules(testCase.InputPair)
require.NoError(t, err, "shouldn't return error")
for _, chain := range manager.chains {
for _, chain := range chains {
rules, err := nftablesTestingClient.GetRules(chain.Table, chain)
require.NoError(t, err, "should list rules for %s table and %s chain", chain.Table.Name, chain.Name)
for _, rule := range rules {
@@ -238,30 +257,39 @@ func isIptablesClientAvailable(client *iptables.IPTables) bool {
return err == nil
}
func createWorkTable() (*nftables.Table, error) {
func createWorkTables() (*nftables.Table, *nftables.Table, error) {
sConn, err := nftables.New(nftables.AsLasting())
if err != nil {
return nil, err
return nil, nil, err
}
tables, err := sConn.ListTablesOfFamily(nftables.TableFamilyIPv4)
if err != nil {
return nil, err
return nil, nil, err
}
for _, t := range tables {
tables6, err := sConn.ListTablesOfFamily(nftables.TableFamilyIPv6)
if err != nil {
return nil, nil, err
}
for _, t := range append(tables, tables6...) {
if t.Name == tableName {
sConn.DelTable(t)
}
}
table := sConn.AddTable(&nftables.Table{Name: tableName, Family: nftables.TableFamilyIPv4})
var table6 *nftables.Table
if iface.SupportsIPv6() {
table6 = sConn.AddTable(&nftables.Table{Name: tableName, Family: nftables.TableFamilyIPv6})
}
err = sConn.Flush()
return table, err
return table, table6, err
}
func deleteWorkTable() {
func deleteWorkTables() {
sConn, err := nftables.New(nftables.AsLasting())
if err != nil {
return
@@ -272,6 +300,12 @@ func deleteWorkTable() {
return
}
tables6, err := sConn.ListTablesOfFamily(nftables.TableFamilyIPv6)
if err != nil {
return
}
tables = append(tables, tables6...)
for _, t := range tables {
if t.Name == tableName {
sConn.DelTable(t)

View File

@@ -8,6 +8,7 @@ var (
InsertRuleTestCases = []struct {
Name string
InputPair firewall.RouterPair
IsV6 bool
}{
{
Name: "Insert Forwarding IPV4 Rule",
@@ -27,12 +28,32 @@ var (
Masquerade: true,
},
},
{
Name: "Insert Forwarding IPV6 Rule",
InputPair: firewall.RouterPair{
ID: "zxa",
Source: "2001:db8:0123:4567::1/128",
Destination: "2001:db8:0123:abcd::/64",
Masquerade: false,
},
IsV6: true,
},
{
Name: "Insert Forwarding And Nat IPV6 Rules",
InputPair: firewall.RouterPair{
ID: "zxa",
Source: "2001:db8:0123:4567::1/128",
Destination: "2001:db8:0123:abcd::/64",
Masquerade: true,
},
IsV6: true,
},
}
RemoveRuleTestCases = []struct {
Name string
InputPair firewall.RouterPair
IpVersion string
IsV6 bool
}{
{
Name: "Remove Forwarding And Nat IPV4 Rules",
@@ -43,5 +64,15 @@ var (
Masquerade: true,
},
},
{
Name: "Remove Forwarding And Nat IPV6 Rules",
InputPair: firewall.RouterPair{
ID: "zxa",
Source: "2001:db8:0123:4567::1/128",
Destination: "2001:db8:0123:abcd::/64",
Masquerade: true,
},
IsV6: true,
},
}
)

View File

@@ -64,15 +64,18 @@ func manageFirewallRule(ruleName string, action action, extraArgs ...string) err
if action == addRule {
args = append(args, extraArgs...)
}
cmd := exec.Command("netsh", args...)
netshCmd := GetSystem32Command("netsh")
cmd := exec.Command(netshCmd, args...)
cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true}
return cmd.Run()
}
func isWindowsFirewallReachable() bool {
args := []string{"advfirewall", "show", "allprofiles", "state"}
cmd := exec.Command("netsh", args...)
netshCmd := GetSystem32Command("netsh")
cmd := exec.Command(netshCmd, args...)
cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true}
_, err := cmd.Output()
@@ -87,8 +90,23 @@ func isWindowsFirewallReachable() bool {
func isFirewallRuleActive(ruleName string) bool {
args := []string{"advfirewall", "firewall", "show", "rule", "name=" + ruleName}
cmd := exec.Command("netsh", args...)
netshCmd := GetSystem32Command("netsh")
cmd := exec.Command(netshCmd, args...)
cmd.SysProcAttr = &syscall.SysProcAttr{HideWindow: true}
_, err := cmd.Output()
return err == nil
}
// GetSystem32Command checks if a command can be found in the system path and returns it. In case it can't find it
// in the path it will return the full path of a command assuming C:\windows\system32 as the base path.
func GetSystem32Command(command string) string {
_, err := exec.LookPath(command)
if err == nil {
return command
}
log.Tracef("Command %s not found in PATH, using C:\\windows\\system32\\%s.exe path", command, command)
return "C:\\windows\\system32\\" + command + ".exe"
}

View File

@@ -24,6 +24,7 @@ var (
type IFaceMapper interface {
SetFilter(iface.PacketFilter) error
Address() iface.WGAddress
Address6() *iface.WGAddress
}
// RuleSet is a set of rules grouped by a string key
@@ -69,6 +70,14 @@ func CreateWithNativeFirewall(iface IFaceMapper, nativeFirewall firewall.Manager
return mgr, nil
}
func (m *Manager) ResetV6Firewall() error {
return nil
}
func (m *Manager) V6Active() bool {
return false
}
func create(iface IFaceMapper) (*Manager, error) {
m := &Manager{
decoders: sync.Pool{

View File

@@ -33,6 +33,10 @@ func (i *IFaceMock) Address() iface.WGAddress {
return i.AddressFunc()
}
func (i *IFaceMock) Address6() *iface.WGAddress {
return nil
}
func TestManagerCreate(t *testing.T) {
ifaceMock := &IFaceMock{
SetFilterFunc: func(iface.PacketFilter) error { return nil },

View File

@@ -16,9 +16,10 @@ import (
mgmProto "github.com/netbirdio/netbird/management/proto"
)
// Manager is a ACL rules manager
// Manager is an ACL rules manager
type Manager interface {
ApplyFiltering(networkMap *mgmProto.NetworkMap)
ResetV6Acl() error
}
// DefaultManager uses firewall manager to handle
@@ -26,16 +27,36 @@ type DefaultManager struct {
firewall firewall.Manager
ipsetCounter int
rulesPairs map[string][]firewall.Rule
rulesPairs6 map[string][]firewall.Rule
mutex sync.Mutex
}
func NewDefaultManager(fm firewall.Manager) *DefaultManager {
return &DefaultManager{
firewall: fm,
rulesPairs: make(map[string][]firewall.Rule),
firewall: fm,
rulesPairs: make(map[string][]firewall.Rule),
rulesPairs6: make(map[string][]firewall.Rule),
}
}
func (d *DefaultManager) ResetV6Acl() error {
for _, rules := range d.rulesPairs6 {
for _, r := range rules {
err := d.firewall.DeleteRule(r)
if err != nil {
return err
}
}
}
err := d.firewall.ResetV6Firewall()
if err != nil {
return err
}
d.rulesPairs6 = make(map[string][]firewall.Rule)
return nil
}
// ApplyFiltering firewall rules to the local firewall manager processed by ACL policy.
//
// If allowByDefault is true it appends allow ALL traffic rules to input and output chains.
@@ -83,6 +104,7 @@ func (d *DefaultManager) ApplyFiltering(networkMap *mgmProto.NetworkMap) {
if enableSSH {
rules = append(rules, &mgmProto.FirewallRule{
PeerIP: "0.0.0.0",
PeerIP6: "::",
Direction: mgmProto.FirewallRule_IN,
Action: mgmProto.FirewallRule_ACCEPT,
Protocol: mgmProto.FirewallRule_TCP,
@@ -97,12 +119,14 @@ func (d *DefaultManager) ApplyFiltering(networkMap *mgmProto.NetworkMap) {
rules = append(rules,
&mgmProto.FirewallRule{
PeerIP: "0.0.0.0",
PeerIP6: "::",
Direction: mgmProto.FirewallRule_IN,
Action: mgmProto.FirewallRule_ACCEPT,
Protocol: mgmProto.FirewallRule_ALL,
},
&mgmProto.FirewallRule{
PeerIP: "0.0.0.0",
PeerIP6: "::",
Direction: mgmProto.FirewallRule_OUT,
Action: mgmProto.FirewallRule_ACCEPT,
Protocol: mgmProto.FirewallRule_ALL,
@@ -111,6 +135,7 @@ func (d *DefaultManager) ApplyFiltering(networkMap *mgmProto.NetworkMap) {
}
newRulePairs := make(map[string][]firewall.Rule)
newRulePairs6 := make(map[string][]firewall.Rule)
ipsetByRuleSelectors := make(map[string]string)
for _, r := range rules {
@@ -123,7 +148,7 @@ func (d *DefaultManager) ApplyFiltering(networkMap *mgmProto.NetworkMap) {
ipsetName = fmt.Sprintf("nb%07d", d.ipsetCounter)
ipsetByRuleSelectors[selector] = ipsetName
}
pairID, rulePair, err := d.protoRuleToFirewallRule(r, ipsetName)
pairID, rulePair, rulePair6, err := d.protoRuleToFirewallRule(r, ipsetName)
if err != nil {
log.Errorf("failed to apply firewall rule: %+v, %v", r, err)
d.rollBack(newRulePairs)
@@ -132,6 +157,8 @@ func (d *DefaultManager) ApplyFiltering(networkMap *mgmProto.NetworkMap) {
if len(rules) > 0 {
d.rulesPairs[pairID] = rulePair
newRulePairs[pairID] = rulePair
d.rulesPairs6[pairID] = rulePair6
newRulePairs6[pairID] = rulePair6
}
}
@@ -146,59 +173,104 @@ func (d *DefaultManager) ApplyFiltering(networkMap *mgmProto.NetworkMap) {
delete(d.rulesPairs, pairID)
}
}
for pairID, rules := range d.rulesPairs6 {
if _, ok := newRulePairs6[pairID]; !ok {
for _, rule := range rules {
if err := d.firewall.DeleteRule(rule); err != nil {
log.Errorf("failed to delete firewall rule: %v", err)
continue
}
}
delete(d.rulesPairs6, pairID)
}
}
d.rulesPairs = newRulePairs
d.rulesPairs6 = newRulePairs6
}
func (d *DefaultManager) protoRuleToFirewallRule(
r *mgmProto.FirewallRule,
ipsetName string,
) (string, []firewall.Rule, error) {
) (string, []firewall.Rule, []firewall.Rule, error) {
ip := net.ParseIP(r.PeerIP)
if ip == nil {
return "", nil, fmt.Errorf("invalid IP address, skipping firewall rule")
return "", nil, nil, fmt.Errorf("invalid IP address, skipping firewall rule")
}
var ip6 *net.IP = nil
if d.firewall.V6Active() && r.PeerIP6 != "" {
ip6tmp := net.ParseIP(r.PeerIP6)
if ip6tmp == nil {
return "", nil, nil, fmt.Errorf("invalid IP address, skipping firewall rule")
}
ip6 = &ip6tmp
}
protocol, err := convertToFirewallProtocol(r.Protocol)
if err != nil {
return "", nil, fmt.Errorf("skipping firewall rule: %s", err)
return "", nil, nil, fmt.Errorf("skipping firewall rule: %s", err)
}
action, err := convertFirewallAction(r.Action)
if err != nil {
return "", nil, fmt.Errorf("skipping firewall rule: %s", err)
return "", nil, nil, fmt.Errorf("skipping firewall rule: %s", err)
}
var port *firewall.Port
if r.Port != "" {
value, err := strconv.Atoi(r.Port)
if err != nil {
return "", nil, fmt.Errorf("invalid port, skipping firewall rule")
return "", nil, nil, fmt.Errorf("invalid port, skipping firewall rule")
}
port = &firewall.Port{
Values: []int{value},
}
}
ruleID := d.getRuleID(ip, protocol, int(r.Direction), port, action, "")
var rules []firewall.Rule
var rules6 []firewall.Rule
ruleID := d.getRuleID(ip, ip6, protocol, int(r.Direction), port, action, "")
if rulesPair, ok := d.rulesPairs[ruleID]; ok {
return ruleID, rulesPair, nil
rules = rulesPair
}
if rulesPair6, ok := d.rulesPairs6[ruleID]; d.firewall.V6Active() && ok && ip6 != nil {
rules6 = rulesPair6
}
var rules []firewall.Rule
switch r.Direction {
case mgmProto.FirewallRule_IN:
rules, err = d.addInRules(ip, protocol, port, action, ipsetName, "")
case mgmProto.FirewallRule_OUT:
rules, err = d.addOutRules(ip, protocol, port, action, ipsetName, "")
default:
return "", nil, fmt.Errorf("invalid direction, skipping firewall rule")
if rules == nil {
switch r.Direction {
case mgmProto.FirewallRule_IN:
rules, err = d.addInRules(ip, protocol, port, action, ipsetName, "")
case mgmProto.FirewallRule_OUT:
rules, err = d.addOutRules(ip, protocol, port, action, ipsetName, "")
default:
return "", nil, nil, fmt.Errorf("invalid direction, skipping firewall rule")
}
}
if err != nil {
return "", nil, err
return "", nil, nil, err
}
return ruleID, rules, nil
if d.firewall.V6Active() && ip6 != nil && rules6 == nil {
switch r.Direction {
case mgmProto.FirewallRule_IN:
rules6, err = d.addInRules(*ip6, protocol, port, action, ipsetName, "")
case mgmProto.FirewallRule_OUT:
rules6, err = d.addOutRules(*ip6, protocol, port, action, ipsetName, "")
default:
return "", nil, nil, fmt.Errorf("invalid direction, skipping firewall rule")
}
}
if err != nil && err.Error() != "failed to add firewall rule: attempted to configure filtering for IPv6 address even though IPv6 is not active" {
return "", rules, nil, err
}
return ruleID, rules, rules6, nil
}
func (d *DefaultManager) addInRules(
@@ -226,8 +298,9 @@ func (d *DefaultManager) addInRules(
if err != nil {
return nil, fmt.Errorf("failed to add firewall rule: %v", err)
}
rules = append(rules, rule...)
return append(rules, rule...), nil
return rules, nil
}
func (d *DefaultManager) addOutRules(
@@ -255,20 +328,26 @@ func (d *DefaultManager) addOutRules(
if err != nil {
return nil, fmt.Errorf("failed to add firewall rule: %v", err)
}
rules = append(rules, rule...)
return append(rules, rule...), nil
return rules, nil
}
// getRuleID() returns unique ID for the rule based on its parameters.
func (d *DefaultManager) getRuleID(
ip net.IP,
ip6 *net.IP,
proto firewall.Protocol,
direction int,
port *firewall.Port,
action firewall.Action,
comment string,
) string {
idStr := ip.String() + string(proto) + strconv.Itoa(direction) + strconv.Itoa(int(action)) + comment
ip6Str := ""
if ip6 != nil {
ip6Str = ip6.String()
}
idStr := ip.String() + ip6Str + string(proto) + strconv.Itoa(direction) + strconv.Itoa(int(action)) + comment
if port != nil {
idStr += port.String()
}
@@ -321,6 +400,8 @@ func (d *DefaultManager) squashAcceptRules(
// it means that rules for that protocol was already optimized on the
// management side
if r.PeerIP == "0.0.0.0" {
// I don't _think_ that IPv6 is relevant here, as any optimization that has r.PeerIP6 == "::" should also
// implicitly have r.PeerIP == "0.0.0.0".
squashedRules = append(squashedRules, r)
squashedProtocols[r.Protocol] = struct{}{}
return
@@ -364,6 +445,7 @@ func (d *DefaultManager) squashAcceptRules(
// add special rule 0.0.0.0 which allows all IP's in our firewall implementations
squashedRules = append(squashedRules, &mgmProto.FirewallRule{
PeerIP: "0.0.0.0",
PeerIP6: "::",
Direction: direction,
Action: mgmProto.FirewallRule_ACCEPT,
Protocol: protocol,

View File

@@ -19,6 +19,7 @@ func TestDefaultManager(t *testing.T) {
FirewallRules: []*mgmProto.FirewallRule{
{
PeerIP: "10.93.0.1",
PeerIP6: "2001:db8::fedc:ba09:8765:0001",
Direction: mgmProto.FirewallRule_OUT,
Action: mgmProto.FirewallRule_ACCEPT,
Protocol: mgmProto.FirewallRule_TCP,
@@ -26,6 +27,7 @@ func TestDefaultManager(t *testing.T) {
},
{
PeerIP: "10.93.0.2",
PeerIP6: "2001:db8::fedc:ba09:8765:0002",
Direction: mgmProto.FirewallRule_OUT,
Action: mgmProto.FirewallRule_DROP,
Protocol: mgmProto.FirewallRule_UDP,
@@ -50,6 +52,14 @@ func TestDefaultManager(t *testing.T) {
IP: ip,
Network: network,
}).AnyTimes()
ip6, network6, err := net.ParseCIDR("2001:db8::fedc:ba09:8765:4321/64")
if err != nil {
t.Fatalf("failed to parse IP address: %v", err)
}
ifaceMock.EXPECT().Address6().Return(&iface.WGAddress{
IP: ip6,
Network: network6,
}).AnyTimes()
// we receive one rule from the management so for testing purposes ignore it
fw, err := firewall.NewFirewall(context.Background(), ifaceMock)
@@ -83,6 +93,7 @@ func TestDefaultManager(t *testing.T) {
networkMap.FirewallRules,
&mgmProto.FirewallRule{
PeerIP: "10.93.0.3",
PeerIP6: "2001:db8::fedc:ba09:8765:0003",
Direction: mgmProto.FirewallRule_IN,
Action: mgmProto.FirewallRule_DROP,
Protocol: mgmProto.FirewallRule_ICMP,
@@ -343,6 +354,7 @@ func TestDefaultManagerEnableSSHRules(t *testing.T) {
IP: ip,
Network: network,
}).AnyTimes()
ifaceMock.EXPECT().Address6().Return(nil).AnyTimes()
// we receive one rule from the management so for testing purposes ignore it
fw, err := firewall.NewFirewall(context.Background(), ifaceMock)

View File

@@ -1,5 +1,5 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: github.com/netbirdio/netbird/client/internal/acl (interfaces: IFaceMapper)
// Source: ./client/firewall/iface.go
// Package mocks is a generated GoMock package.
package mocks
@@ -48,6 +48,20 @@ func (mr *MockIFaceMapperMockRecorder) Address() *gomock.Call {
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Address", reflect.TypeOf((*MockIFaceMapper)(nil).Address))
}
// Address6 mocks base method.
func (m *MockIFaceMapper) Address6() *iface.WGAddress {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Address6")
ret0, _ := ret[0].(*iface.WGAddress)
return ret0
}
// Address6 indicates an expected call of Address6.
func (mr *MockIFaceMapperMockRecorder) Address6() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Address6", reflect.TypeOf((*MockIFaceMapper)(nil).Address6))
}
// IsUserspaceBind mocks base method.
func (m *MockIFaceMapper) IsUserspaceBind() bool {
m.ctrl.T.Helper()

View File

@@ -5,6 +5,8 @@ import (
"fmt"
"net/url"
"os"
"reflect"
"strings"
log "github.com/sirupsen/logrus"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
@@ -48,6 +50,7 @@ type ConfigInput struct {
RosenpassPermissive *bool
InterfaceName *string
WireguardPort *int
NetworkMonitor *bool
DisableAutoConnect *bool
ExtraIFaceBlackList []string
}
@@ -61,6 +64,7 @@ type Config struct {
AdminURL *url.URL
WgIface string
WgPort int
NetworkMonitor bool
IFaceBlackList []string
DisableIPv6Discovery bool
RosenpassEnabled bool
@@ -100,6 +104,14 @@ func ReadConfig(configPath string) (*Config, error) {
if _, err := util.ReadJson(configPath, config); err != nil {
return nil, err
}
// initialize through apply() without changes
if changed, err := config.apply(ConfigInput{}); err != nil {
return nil, err
} else if changed {
if err = WriteOutConfig(configPath, config); err != nil {
return nil, err
}
}
return config, nil
}
@@ -152,79 +164,15 @@ func WriteOutConfig(path string, config *Config) error {
// createNewConfig creates a new config generating a new Wireguard key and saving to file
func createNewConfig(input ConfigInput) (*Config, error) {
wgKey := generateKey()
pem, err := ssh.GeneratePrivateKey(ssh.ED25519)
if err != nil {
return nil, err
}
config := &Config{
SSHKey: string(pem),
PrivateKey: wgKey,
IFaceBlackList: []string{},
DisableIPv6Discovery: false,
NATExternalIPs: input.NATExternalIPs,
CustomDNSAddress: string(input.CustomDNSAddress),
ServerSSHAllowed: util.False(),
DisableAutoConnect: false,
// defaults to false only for new (post 0.26) configurations
ServerSSHAllowed: util.False(),
}
defaultManagementURL, err := parseURL("Management URL", DefaultManagementURL)
if err != nil {
if _, err := config.apply(input); err != nil {
return nil, err
}
config.ManagementURL = defaultManagementURL
if input.ManagementURL != "" {
URL, err := parseURL("Management URL", input.ManagementURL)
if err != nil {
return nil, err
}
config.ManagementURL = URL
}
config.WgPort = iface.DefaultWgPort
if input.WireguardPort != nil {
config.WgPort = *input.WireguardPort
}
config.WgIface = iface.WgInterfaceDefault
if input.InterfaceName != nil {
config.WgIface = *input.InterfaceName
}
if input.PreSharedKey != nil {
config.PreSharedKey = *input.PreSharedKey
}
if input.RosenpassEnabled != nil {
config.RosenpassEnabled = *input.RosenpassEnabled
}
if input.RosenpassPermissive != nil {
config.RosenpassPermissive = *input.RosenpassPermissive
}
if input.ServerSSHAllowed != nil {
config.ServerSSHAllowed = input.ServerSSHAllowed
}
defaultAdminURL, err := parseURL("Admin URL", DefaultAdminURL)
if err != nil {
return nil, err
}
config.AdminURL = defaultAdminURL
if input.AdminURL != "" {
newURL, err := parseURL("Admin Panel URL", input.AdminURL)
if err != nil {
return nil, err
}
config.AdminURL = newURL
}
// nolint:gocritic
config.IFaceBlackList = append(defaultInterfaceBlacklist, input.ExtraIFaceBlackList...)
return config, nil
}
@@ -235,104 +183,12 @@ func update(input ConfigInput) (*Config, error) {
return nil, err
}
refresh := false
if input.ManagementURL != "" && config.ManagementURL.String() != input.ManagementURL {
log.Infof("new Management URL provided, updated to %s (old value %s)",
input.ManagementURL, config.ManagementURL)
newURL, err := parseURL("Management URL", input.ManagementURL)
if err != nil {
return nil, err
}
config.ManagementURL = newURL
refresh = true
updated, err := config.apply(input)
if err != nil {
return nil, err
}
if input.AdminURL != "" && (config.AdminURL == nil || config.AdminURL.String() != input.AdminURL) {
log.Infof("new Admin Panel URL provided, updated to %s (old value %s)",
input.AdminURL, config.AdminURL)
newURL, err := parseURL("Admin Panel URL", input.AdminURL)
if err != nil {
return nil, err
}
config.AdminURL = newURL
refresh = true
}
if input.PreSharedKey != nil && config.PreSharedKey != *input.PreSharedKey {
log.Infof("new pre-shared key provided, replacing old key")
config.PreSharedKey = *input.PreSharedKey
refresh = true
}
if config.SSHKey == "" {
pem, err := ssh.GeneratePrivateKey(ssh.ED25519)
if err != nil {
return nil, err
}
config.SSHKey = string(pem)
refresh = true
}
if config.WgPort == 0 {
config.WgPort = iface.DefaultWgPort
refresh = true
}
if input.WireguardPort != nil {
config.WgPort = *input.WireguardPort
refresh = true
}
if input.InterfaceName != nil {
config.WgIface = *input.InterfaceName
refresh = true
}
if input.NATExternalIPs != nil && len(config.NATExternalIPs) != len(input.NATExternalIPs) {
config.NATExternalIPs = input.NATExternalIPs
refresh = true
}
if input.CustomDNSAddress != nil {
config.CustomDNSAddress = string(input.CustomDNSAddress)
refresh = true
}
if input.RosenpassEnabled != nil {
config.RosenpassEnabled = *input.RosenpassEnabled
refresh = true
}
if input.RosenpassPermissive != nil {
config.RosenpassPermissive = *input.RosenpassPermissive
refresh = true
}
if input.DisableAutoConnect != nil {
config.DisableAutoConnect = *input.DisableAutoConnect
refresh = true
}
if input.ServerSSHAllowed != nil {
config.ServerSSHAllowed = input.ServerSSHAllowed
refresh = true
}
if config.ServerSSHAllowed == nil {
config.ServerSSHAllowed = util.True()
refresh = true
}
if len(input.ExtraIFaceBlackList) > 0 {
for _, iFace := range util.SliceDiff(input.ExtraIFaceBlackList, config.IFaceBlackList) {
config.IFaceBlackList = append(config.IFaceBlackList, iFace)
refresh = true
}
}
if refresh {
// since we have new management URL, we need to update config file
if updated {
if err := util.WriteJson(input.ConfigPath, config); err != nil {
return nil, err
}
@@ -341,6 +197,169 @@ func update(input ConfigInput) (*Config, error) {
return config, nil
}
func (config *Config) apply(input ConfigInput) (updated bool, err error) {
if config.ManagementURL == nil {
log.Infof("using default Management URL %s", DefaultManagementURL)
config.ManagementURL, err = parseURL("Management URL", DefaultManagementURL)
if err != nil {
return false, err
}
}
if input.ManagementURL != "" && input.ManagementURL != config.ManagementURL.String() {
log.Infof("new Management URL provided, updated to %#v (old value %#v)",
input.ManagementURL, config.ManagementURL.String())
URL, err := parseURL("Management URL", input.ManagementURL)
if err != nil {
return false, err
}
config.ManagementURL = URL
updated = true
} else if config.ManagementURL == nil {
log.Infof("using default Management URL %s", DefaultManagementURL)
config.ManagementURL, err = parseURL("Management URL", DefaultManagementURL)
if err != nil {
return false, err
}
}
if config.AdminURL == nil {
log.Infof("using default Admin URL %s", DefaultManagementURL)
config.AdminURL, err = parseURL("Admin URL", DefaultAdminURL)
if err != nil {
return false, err
}
}
if input.AdminURL != "" && input.AdminURL != config.AdminURL.String() {
log.Infof("new Admin Panel URL provided, updated to %#v (old value %#v)",
input.AdminURL, config.AdminURL.String())
newURL, err := parseURL("Admin Panel URL", input.AdminURL)
if err != nil {
return updated, err
}
config.AdminURL = newURL
updated = true
}
if config.PrivateKey == "" {
log.Infof("generated new Wireguard key")
config.PrivateKey = generateKey()
updated = true
}
if config.SSHKey == "" {
log.Infof("generated new SSH key")
pem, err := ssh.GeneratePrivateKey(ssh.ED25519)
if err != nil {
return false, err
}
config.SSHKey = string(pem)
updated = true
}
if input.WireguardPort != nil && *input.WireguardPort != config.WgPort {
log.Infof("updating Wireguard port %d (old value %d)",
*input.WireguardPort, config.WgPort)
config.WgPort = *input.WireguardPort
updated = true
} else if config.WgPort == 0 {
config.WgPort = iface.DefaultWgPort
log.Infof("using default Wireguard port %d", config.WgPort)
updated = true
}
if input.InterfaceName != nil && *input.InterfaceName != config.WgIface {
log.Infof("updating Wireguard interface %#v (old value %#v)",
*input.InterfaceName, config.WgIface)
config.WgIface = *input.InterfaceName
updated = true
} else if config.WgIface == "" {
config.WgIface = iface.WgInterfaceDefault
log.Infof("using default Wireguard interface %s", config.WgIface)
updated = true
}
if input.NATExternalIPs != nil && !reflect.DeepEqual(config.NATExternalIPs, input.NATExternalIPs) {
log.Infof("updating NAT External IP [ %s ] (old value: [ %s ])",
strings.Join(input.NATExternalIPs, " "),
strings.Join(config.NATExternalIPs, " "))
config.NATExternalIPs = input.NATExternalIPs
updated = true
}
if input.PreSharedKey != nil && *input.PreSharedKey != config.PreSharedKey {
log.Infof("new pre-shared key provided, replacing old key")
config.PreSharedKey = *input.PreSharedKey
updated = true
}
if input.RosenpassEnabled != nil && *input.RosenpassEnabled != config.RosenpassEnabled {
log.Infof("switching Rosenpass to %t", *input.RosenpassEnabled)
config.RosenpassEnabled = *input.RosenpassEnabled
updated = true
}
if input.RosenpassPermissive != nil && *input.RosenpassPermissive != config.RosenpassPermissive {
log.Infof("switching Rosenpass permissive to %t", *input.RosenpassPermissive)
config.RosenpassPermissive = *input.RosenpassPermissive
updated = true
}
if input.NetworkMonitor != nil && *input.NetworkMonitor != config.NetworkMonitor {
log.Infof("switching Network Monitor to %t", *input.NetworkMonitor)
config.NetworkMonitor = *input.NetworkMonitor
updated = true
}
if input.CustomDNSAddress != nil && string(input.CustomDNSAddress) != config.CustomDNSAddress {
log.Infof("updating custom DNS address %#v (old value %#v)",
string(input.CustomDNSAddress), config.CustomDNSAddress)
config.CustomDNSAddress = string(input.CustomDNSAddress)
updated = true
}
if len(config.IFaceBlackList) == 0 {
log.Infof("filling in interface blacklist with defaults: [ %s ]",
strings.Join(defaultInterfaceBlacklist, " "))
config.IFaceBlackList = append(config.IFaceBlackList, defaultInterfaceBlacklist...)
updated = true
}
if len(input.ExtraIFaceBlackList) > 0 {
for _, iFace := range util.SliceDiff(input.ExtraIFaceBlackList, config.IFaceBlackList) {
log.Infof("adding new entry to interface blacklist: %s", iFace)
config.IFaceBlackList = append(config.IFaceBlackList, iFace)
updated = true
}
}
if input.DisableAutoConnect != nil && *input.DisableAutoConnect != config.DisableAutoConnect {
if *input.DisableAutoConnect {
log.Infof("turning off automatic connection on startup")
} else {
log.Infof("enabling automatic connection on startup")
}
config.DisableAutoConnect = *input.DisableAutoConnect
updated = true
}
if input.ServerSSHAllowed != nil && *input.ServerSSHAllowed != *config.ServerSSHAllowed {
if *input.ServerSSHAllowed {
log.Infof("enabling SSH server")
} else {
log.Infof("disabling SSH server")
}
config.ServerSSHAllowed = input.ServerSSHAllowed
updated = true
} else if config.ServerSSHAllowed == nil {
// enables SSH for configs from old versions to preserve backwards compatibility
log.Infof("falling back to enabled SSH server for pre-existing configuration")
config.ServerSSHAllowed = util.True()
updated = true
}
return updated, nil
}
// parseURL parses and validates a service URL
func parseURL(serviceName, serviceURL string) (*url.URL, error) {
parsedMgmtURL, err := url.ParseRequestURI(serviceURL)

View File

@@ -4,9 +4,11 @@ import (
"context"
"errors"
"fmt"
"net"
"runtime"
"runtime/debug"
"strings"
"sync"
"time"
"github.com/cenkalti/backoff/v4"
@@ -29,29 +31,45 @@ import (
"github.com/netbirdio/netbird/version"
)
// RunClient with main logic.
func RunClient(ctx context.Context, config *Config, statusRecorder *peer.Status) error {
return runClient(ctx, config, statusRecorder, MobileDependency{}, nil, nil, nil, nil)
type ConnectClient struct {
ctx context.Context
config *Config
statusRecorder *peer.Status
engine *Engine
engineMutex sync.Mutex
}
// RunClientWithProbes runs the client's main logic with probes attached
func RunClientWithProbes(
func NewConnectClient(
ctx context.Context,
config *Config,
statusRecorder *peer.Status,
) *ConnectClient {
return &ConnectClient{
ctx: ctx,
config: config,
statusRecorder: statusRecorder,
engineMutex: sync.Mutex{},
}
}
// Run with main logic.
func (c *ConnectClient) Run() error {
return c.run(MobileDependency{}, nil, nil, nil, nil)
}
// RunWithProbes runs the client's main logic with probes attached
func (c *ConnectClient) RunWithProbes(
mgmProbe *Probe,
signalProbe *Probe,
relayProbe *Probe,
wgProbe *Probe,
) error {
return runClient(ctx, config, statusRecorder, MobileDependency{}, mgmProbe, signalProbe, relayProbe, wgProbe)
return c.run(MobileDependency{}, mgmProbe, signalProbe, relayProbe, wgProbe)
}
// RunClientMobile with main logic on mobile system
func RunClientMobile(
ctx context.Context,
config *Config,
statusRecorder *peer.Status,
// RunOnAndroid with main logic on mobile system
func (c *ConnectClient) RunOnAndroid(
tunAdapter iface.TunAdapter,
iFaceDiscover stdnet.ExternalIFaceDiscover,
networkChangeListener listener.NetworkChangeListener,
@@ -66,29 +84,26 @@ func RunClientMobile(
HostDNSAddresses: dnsAddresses,
DnsReadyListener: dnsReadyListener,
}
return runClient(ctx, config, statusRecorder, mobileDependency, nil, nil, nil, nil)
return c.run(mobileDependency, nil, nil, nil, nil)
}
func RunClientiOS(
ctx context.Context,
config *Config,
statusRecorder *peer.Status,
func (c *ConnectClient) RunOniOS(
fileDescriptor int32,
networkChangeListener listener.NetworkChangeListener,
dnsManager dns.IosDnsManager,
) error {
// Set GC percent to 5% to reduce memory usage as iOS only allows 50MB of memory for the extension.
debug.SetGCPercent(5)
mobileDependency := MobileDependency{
FileDescriptor: fileDescriptor,
NetworkChangeListener: networkChangeListener,
DnsManager: dnsManager,
}
return runClient(ctx, config, statusRecorder, mobileDependency, nil, nil, nil, nil)
return c.run(mobileDependency, nil, nil, nil, nil)
}
func runClient(
ctx context.Context,
config *Config,
statusRecorder *peer.Status,
func (c *ConnectClient) run(
mobileDependency MobileDependency,
mgmProbe *Probe,
signalProbe *Probe,
@@ -105,7 +120,7 @@ func runClient(
// Check if client was not shut down in a clean way and restore DNS config if required.
// Otherwise, we might not be able to connect to the management server to retrieve new config.
if err := dns.CheckUncleanShutdown(config.WgIface); err != nil {
if err := dns.CheckUncleanShutdown(c.config.WgIface); err != nil {
log.Errorf("checking unclean shutdown error: %s", err)
}
@@ -119,7 +134,7 @@ func runClient(
Clock: backoff.SystemClock,
}
state := CtxGetState(ctx)
state := CtxGetState(c.ctx)
defer func() {
s, err := state.Status()
if err != nil || s != StatusNeedsLogin {
@@ -128,49 +143,49 @@ func runClient(
}()
wrapErr := state.Wrap
myPrivateKey, err := wgtypes.ParseKey(config.PrivateKey)
myPrivateKey, err := wgtypes.ParseKey(c.config.PrivateKey)
if err != nil {
log.Errorf("failed parsing Wireguard key %s: [%s]", config.PrivateKey, err.Error())
log.Errorf("failed parsing Wireguard key %s: [%s]", c.config.PrivateKey, err.Error())
return wrapErr(err)
}
var mgmTlsEnabled bool
if config.ManagementURL.Scheme == "https" {
if c.config.ManagementURL.Scheme == "https" {
mgmTlsEnabled = true
}
publicSSHKey, err := ssh.GeneratePublicKey([]byte(config.SSHKey))
publicSSHKey, err := ssh.GeneratePublicKey([]byte(c.config.SSHKey))
if err != nil {
return err
}
defer statusRecorder.ClientStop()
defer c.statusRecorder.ClientStop()
operation := func() error {
// if context cancelled we not start new backoff cycle
select {
case <-ctx.Done():
case <-c.ctx.Done():
return nil
default:
}
state.Set(StatusConnecting)
engineCtx, cancel := context.WithCancel(ctx)
engineCtx, cancel := context.WithCancel(c.ctx)
defer func() {
statusRecorder.MarkManagementDisconnected(state.err)
statusRecorder.CleanLocalPeerState()
c.statusRecorder.MarkManagementDisconnected(state.err)
c.statusRecorder.CleanLocalPeerState()
cancel()
}()
log.Debugf("connecting to the Management service %s", config.ManagementURL.Host)
mgmClient, err := mgm.NewClient(engineCtx, config.ManagementURL.Host, myPrivateKey, mgmTlsEnabled)
log.Debugf("connecting to the Management service %s", c.config.ManagementURL.Host)
mgmClient, err := mgm.NewClient(engineCtx, c.config.ManagementURL.Host, myPrivateKey, mgmTlsEnabled)
if err != nil {
return wrapErr(gstatus.Errorf(codes.FailedPrecondition, "failed connecting to Management Service : %s", err))
}
mgmNotifier := statusRecorderToMgmConnStateNotifier(statusRecorder)
mgmNotifier := statusRecorderToMgmConnStateNotifier(c.statusRecorder)
mgmClient.SetConnStateListener(mgmNotifier)
log.Debugf("connected to the Management service %s", config.ManagementURL.Host)
log.Debugf("connected to the Management service %s", c.config.ManagementURL.Host)
defer func() {
err = mgmClient.Close()
if err != nil {
@@ -188,7 +203,7 @@ func runClient(
}
return wrapErr(err)
}
statusRecorder.MarkManagementConnected()
c.statusRecorder.MarkManagementConnected()
localPeerState := peer.LocalPeerState{
IP: loginResp.GetPeerConfig().GetAddress(),
@@ -197,18 +212,18 @@ func runClient(
FQDN: loginResp.GetPeerConfig().GetFqdn(),
}
statusRecorder.UpdateLocalPeerState(localPeerState)
c.statusRecorder.UpdateLocalPeerState(localPeerState)
signalURL := fmt.Sprintf("%s://%s",
strings.ToLower(loginResp.GetWiretrusteeConfig().GetSignal().GetProtocol().String()),
loginResp.GetWiretrusteeConfig().GetSignal().GetUri(),
)
statusRecorder.UpdateSignalAddress(signalURL)
c.statusRecorder.UpdateSignalAddress(signalURL)
statusRecorder.MarkSignalDisconnected(nil)
c.statusRecorder.MarkSignalDisconnected(nil)
defer func() {
statusRecorder.MarkSignalDisconnected(state.err)
c.statusRecorder.MarkSignalDisconnected(state.err)
}()
// with the global Wiretrustee config in hand connect (just a connection, no stream yet) Signal
@@ -224,35 +239,38 @@ func runClient(
}
}()
signalNotifier := statusRecorderToSignalConnStateNotifier(statusRecorder)
signalNotifier := statusRecorderToSignalConnStateNotifier(c.statusRecorder)
signalClient.SetConnStateListener(signalNotifier)
statusRecorder.MarkSignalConnected()
c.statusRecorder.MarkSignalConnected()
peerConfig := loginResp.GetPeerConfig()
engineConfig, err := createEngineConfig(myPrivateKey, config, peerConfig)
engineConfig, err := createEngineConfig(myPrivateKey, c.config, peerConfig)
if err != nil {
log.Error(err)
return wrapErr(err)
}
engine := NewEngineWithProbes(engineCtx, cancel, signalClient, mgmClient, engineConfig, mobileDependency, statusRecorder, mgmProbe, signalProbe, relayProbe, wgProbe)
err = engine.Start()
c.engineMutex.Lock()
c.engine = NewEngineWithProbes(engineCtx, cancel, signalClient, mgmClient, engineConfig, mobileDependency, c.statusRecorder, mgmProbe, signalProbe, relayProbe, wgProbe)
c.engineMutex.Unlock()
err = c.engine.Start()
if err != nil {
log.Errorf("error while starting Netbird Connection Engine: %s", err)
return wrapErr(err)
}
log.Print("Netbird engine started, my IP is: ", peerConfig.Address)
log.Infof("Netbird engine started, the IP is: %s", peerConfig.GetAddress())
state.Set(StatusConnected)
<-engineCtx.Done()
statusRecorder.ClientTeardown()
c.statusRecorder.ClientTeardown()
backOff.Reset()
err = engine.Stop()
err = c.engine.Stop()
if err != nil {
log.Errorf("failed stopping engine %v", err)
return wrapErr(err)
@@ -267,7 +285,7 @@ func runClient(
return nil
}
statusRecorder.ClientStart()
c.statusRecorder.ClientStart()
err = backoff.Retry(operation, backOff)
if err != nil {
log.Debugf("exiting client retry loop due to unrecoverable error: %s", err)
@@ -279,15 +297,25 @@ func runClient(
return nil
}
func (c *ConnectClient) Engine() *Engine {
var e *Engine
c.engineMutex.Lock()
e = c.engine
c.engineMutex.Unlock()
return e
}
// createEngineConfig converts configuration received from Management Service to EngineConfig
func createEngineConfig(key wgtypes.Key, config *Config, peerConfig *mgmProto.PeerConfig) (*EngineConfig, error) {
engineConf := &EngineConfig{
WgIfaceName: config.WgIface,
WgAddr: peerConfig.Address,
WgAddr6: peerConfig.Address6,
IFaceBlackList: config.IFaceBlackList,
DisableIPv6Discovery: config.DisableIPv6Discovery,
WgPrivateKey: key,
WgPort: config.WgPort,
NetworkMonitor: config.NetworkMonitor,
SSHKey: []byte(config.SSHKey),
NATExternalIPs: config.NATExternalIPs,
CustomDNSAddress: config.CustomDNSAddress,
@@ -304,6 +332,15 @@ func createEngineConfig(key wgtypes.Key, config *Config, peerConfig *mgmProto.Pe
engineConf.PreSharedKey = &preSharedKey
}
port, err := freePort(config.WgPort)
if err != nil {
return nil, err
}
if port != config.WgPort {
log.Infof("using %d as wireguard port: %d is in use", port, config.WgPort)
}
engineConf.WgPort = port
return engineConf, nil
}
@@ -353,3 +390,20 @@ func statusRecorderToSignalConnStateNotifier(statusRecorder *peer.Status) signal
notifier, _ := sri.(signal.ConnStateNotifier)
return notifier
}
func freePort(start int) (int, error) {
addr := net.UDPAddr{}
if start == 0 {
start = iface.DefaultWgPort
}
for x := start; x <= 65535; x++ {
addr.Port = x
conn, err := net.ListenUDP("udp", &addr)
if err != nil {
continue
}
conn.Close()
return x, nil
}
return 0, errors.New("no free ports")
}

View File

@@ -0,0 +1,57 @@
package internal
import (
"net"
"testing"
)
func Test_freePort(t *testing.T) {
tests := []struct {
name string
port int
want int
wantErr bool
}{
{
name: "available",
port: 51820,
want: 51820,
wantErr: false,
},
{
name: "notavailable",
port: 51830,
want: 51831,
wantErr: false,
},
{
name: "noports",
port: 65535,
want: 0,
wantErr: true,
},
}
for _, tt := range tests {
c1, err := net.ListenUDP("udp", &net.UDPAddr{Port: 51830})
if err != nil {
t.Errorf("freePort error = %v", err)
}
c2, err := net.ListenUDP("udp", &net.UDPAddr{Port: 65535})
if err != nil {
t.Errorf("freePort error = %v", err)
}
t.Run(tt.name, func(t *testing.T) {
got, err := freePort(tt.port)
if (err != nil) != tt.wantErr {
t.Errorf("freePort() error = %v, wantErr %v", err, tt.wantErr)
return
}
if got != tt.want {
t.Errorf("freePort() = %v, want %v", got, tt.want)
}
})
c1.Close()
c2.Close()
}
}

View File

@@ -47,24 +47,20 @@ func (f *fileConfigurator) supportCustomPort() bool {
}
func (f *fileConfigurator) applyDNSConfig(config HostDNSConfig) error {
backupFileExist := false
_, err := os.Stat(fileDefaultResolvConfBackupLocation)
if err == nil {
backupFileExist = true
}
backupFileExist := f.isBackupFileExist()
if !config.RouteAll {
if backupFileExist {
err = f.restore()
f.repair.stopWatchFileChanges()
err := f.restore()
if err != nil {
return fmt.Errorf("unable to configure DNS for this peer using file manager without a Primary nameserver group. Restoring the original file return err: %w", err)
return fmt.Errorf("restoring the original resolv.conf file return err: %w", err)
}
}
return fmt.Errorf("unable to configure DNS for this peer using file manager without a nameserver group with all domains configured")
}
if !backupFileExist {
err = f.backup()
err := f.backup()
if err != nil {
return fmt.Errorf("unable to backup the resolv.conf file: %w", err)
}
@@ -184,6 +180,11 @@ func (f *fileConfigurator) restoreUncleanShutdownDNS(storedDNSAddress *netip.Add
return nil
}
func (f *fileConfigurator) isBackupFileExist() bool {
_, err := os.Stat(fileDefaultResolvConfBackupLocation)
return err == nil
}
func restoreResolvConfFile() error {
log.Debugf("restoring unclean shutdown: restoring %s from %s", defaultResolvConfPath, fileUncleanShutdownResolvConfLocation)

View File

@@ -0,0 +1,63 @@
package dns
import (
"fmt"
"net/netip"
"sync"
log "github.com/sirupsen/logrus"
)
type hostsDNSHolder struct {
unprotectedDNSList map[string]struct{}
mutex sync.RWMutex
}
func newHostsDNSHolder() *hostsDNSHolder {
return &hostsDNSHolder{
unprotectedDNSList: make(map[string]struct{}),
}
}
func (h *hostsDNSHolder) set(list []string) {
h.mutex.Lock()
h.unprotectedDNSList = make(map[string]struct{})
for _, dns := range list {
dnsAddr, err := h.normalizeAddress(dns)
if err != nil {
continue
}
h.unprotectedDNSList[dnsAddr] = struct{}{}
}
h.mutex.Unlock()
}
func (h *hostsDNSHolder) get() map[string]struct{} {
h.mutex.RLock()
l := h.unprotectedDNSList
h.mutex.RUnlock()
return l
}
//nolint:unused
func (h *hostsDNSHolder) isContain(upstream string) bool {
h.mutex.RLock()
defer h.mutex.RUnlock()
_, ok := h.unprotectedDNSList[upstream]
return ok
}
func (h *hostsDNSHolder) normalizeAddress(addr string) (string, error) {
a, err := netip.ParseAddr(addr)
if err != nil {
log.Errorf("invalid upstream IP address: %s, error: %s", addr, err)
return "", err
}
if a.Is4() {
return fmt.Sprintf("%s:53", addr), nil
} else {
return fmt.Sprintf("[%s]:53", addr), nil
}
}

View File

@@ -31,6 +31,8 @@ func (d *localResolver) ServeDNS(w dns.ResponseWriter, r *dns.Msg) {
response := d.lookupRecord(r)
if response != nil {
replyMessage.Answer = append(replyMessage.Answer, response)
} else {
replyMessage.Rcode = dns.RcodeNameError
}
err := w.WriteMsg(replyMessage)

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"net/netip"
"runtime"
"strings"
"sync"
@@ -54,9 +55,8 @@ type DefaultServer struct {
currentConfig HostDNSConfig
// permanent related properties
permanent bool
hostsDnsList []string
hostsDnsListLock sync.Mutex
permanent bool
hostsDNSHolder *hostsDNSHolder
// make sense on mobile only
searchDomainNotifier *notifier
@@ -113,8 +113,8 @@ func NewDefaultServerPermanentUpstream(
) *DefaultServer {
log.Debugf("host dns address list is: %v", hostsDnsList)
ds := newDefaultServer(ctx, wgInterface, newServiceViaMemory(wgInterface), statusRecorder)
ds.hostsDNSHolder.set(hostsDnsList)
ds.permanent = true
ds.hostsDnsList = hostsDnsList
ds.addHostRootZone()
ds.currentConfig = dnsConfigToHostDNSConfig(config, ds.service.RuntimeIP(), ds.service.RuntimePort())
ds.searchDomainNotifier = newNotifier(ds.SearchDomains())
@@ -147,6 +147,7 @@ func newDefaultServer(ctx context.Context, wgInterface WGIface, dnsService servi
},
wgInterface: wgInterface,
statusRecorder: statusRecorder,
hostsDNSHolder: newHostsDNSHolder(),
}
return defaultServer
@@ -202,10 +203,8 @@ func (s *DefaultServer) Stop() {
// OnUpdatedHostDNSServer update the DNS servers addresses for root zones
// It will be applied if the mgm server do not enforce DNS settings for root zone
func (s *DefaultServer) OnUpdatedHostDNSServer(hostsDnsList []string) {
s.hostsDnsListLock.Lock()
defer s.hostsDnsListLock.Unlock()
s.hostsDNSHolder.set(hostsDnsList)
s.hostsDnsList = hostsDnsList
_, ok := s.dnsMuxMap[nbdns.RootZone]
if ok {
log.Debugf("on new host DNS config but skip to apply it")
@@ -374,6 +373,7 @@ func (s *DefaultServer) buildUpstreamHandlerUpdate(nameServerGroups []*nbdns.Nam
s.wgInterface.Address().IP,
s.wgInterface.Address().Network,
s.statusRecorder,
s.hostsDNSHolder,
)
if err != nil {
return nil, fmt.Errorf("unable to create a new upstream resolver, error: %v", err)
@@ -452,9 +452,7 @@ func (s *DefaultServer) updateMux(muxUpdates []muxUpdate) {
_, found := muxUpdateMap[key]
if !found {
if !isContainRootUpdate && key == nbdns.RootZone {
s.hostsDnsListLock.Lock()
s.addHostRootZone()
s.hostsDnsListLock.Unlock()
existingHandler.stop()
} else {
existingHandler.stop()
@@ -487,7 +485,11 @@ func (s *DefaultServer) updateLocalResolver(update map[string]nbdns.SimpleRecord
}
func getNSHostPort(ns nbdns.NameServer) string {
return fmt.Sprintf("%s:%d", ns.IP.String(), ns.Port)
if ns.IP.Is4() {
return fmt.Sprintf("%s:%d", ns.IP.String(), ns.Port)
} else {
return fmt.Sprintf("[%s]:%d", ns.IP.String(), ns.Port)
}
}
// upstreamCallbacks returns two functions, the first one is used to deactivate
@@ -512,6 +514,7 @@ func (s *DefaultServer) upstreamCallbacks(
if nsGroup.Primary {
removeIndex[nbdns.RootZone] = -1
s.currentConfig.RouteAll = false
s.service.DeregisterMux(nbdns.RootZone)
}
for i, item := range s.currentConfig.Domains {
@@ -521,10 +524,15 @@ func (s *DefaultServer) upstreamCallbacks(
removeIndex[item.Domain] = i
}
}
if err := s.hostManager.applyDNSConfig(s.currentConfig); err != nil {
l.Errorf("Failed to apply nameserver deactivation on the host: %v", err)
}
if runtime.GOOS == "android" && nsGroup.Primary && len(s.hostsDNSHolder.get()) > 0 {
s.addHostRootZone()
}
s.updateNSState(nsGroup, err, false)
}
@@ -545,6 +553,7 @@ func (s *DefaultServer) upstreamCallbacks(
if nsGroup.Primary {
s.currentConfig.RouteAll = true
s.service.RegisterMux(nbdns.RootZone, handler)
}
if err := s.hostManager.applyDNSConfig(s.currentConfig); err != nil {
l.WithError(err).Error("reactivate temporary disabled nameserver group, DNS update apply")
@@ -562,25 +571,16 @@ func (s *DefaultServer) addHostRootZone() {
s.wgInterface.Address().IP,
s.wgInterface.Address().Network,
s.statusRecorder,
s.hostsDNSHolder,
)
if err != nil {
log.Errorf("unable to create a new upstream resolver, error: %v", err)
return
}
handler.upstreamServers = make([]string, len(s.hostsDnsList))
for n, ua := range s.hostsDnsList {
a, err := netip.ParseAddr(ua)
if err != nil {
log.Errorf("invalid upstream IP address: %s, error: %s", ua, err)
continue
}
ipString := ua
if !a.Is4() {
ipString = fmt.Sprintf("[%s]", ua)
}
handler.upstreamServers[n] = fmt.Sprintf("%s:53", ipString)
handler.upstreamServers = make([]string, 0)
for k := range s.hostsDNSHolder.get() {
handler.upstreamServers = append(handler.upstreamServers, k)
}
handler.deactivate = func(error) {}
handler.reactivate = func() {}

View File

@@ -38,6 +38,13 @@ func (w *mocWGIface) Address() iface.WGAddress {
Network: network,
}
}
func (w *mocWGIface) Address6() *iface.WGAddress {
ip, network, _ := net.ParseCIDR("fd00:1234:dead:beef::/64")
return &iface.WGAddress{
IP: ip,
Network: network,
}
}
func (w *mocWGIface) GetFilter() iface.PacketFilter {
return w.filter
@@ -261,7 +268,7 @@ func TestUpdateDNSServer(t *testing.T) {
if err != nil {
t.Fatal(err)
}
wgIface, err := iface.NewWGIFace(fmt.Sprintf("utun230%d", n), fmt.Sprintf("100.66.100.%d/32", n+1), 33100, privKey.String(), iface.DefaultMTU, newNet, nil)
wgIface, err := iface.NewWGIFace(fmt.Sprintf("utun230%d", n), fmt.Sprintf("100.66.100.%d/32", n+1), fmt.Sprintf("fd00:1234:dead:beef::%d/128", n+1), 33100, privKey.String(), iface.DefaultMTU, newNet, nil)
if err != nil {
t.Fatal(err)
}
@@ -339,7 +346,7 @@ func TestDNSFakeResolverHandleUpdates(t *testing.T) {
}
privKey, _ := wgtypes.GeneratePrivateKey()
wgIface, err := iface.NewWGIFace("utun2301", "100.66.100.1/32", 33100, privKey.String(), iface.DefaultMTU, newNet, nil)
wgIface, err := iface.NewWGIFace("utun2301", "100.66.100.1/32", "", 33100, privKey.String(), iface.DefaultMTU, newNet, nil)
if err != nil {
t.Errorf("build interface wireguard: %v", err)
return
@@ -595,7 +602,7 @@ func TestDNSServerUpstreamDeactivateCallback(t *testing.T) {
}
func TestDNSPermanent_updateHostDNS_emptyUpstream(t *testing.T) {
wgIFace, err := createWgInterfaceWithBind(t)
wgIFace, err := createWgInterfaceWithBind(t, false)
if err != nil {
t.Fatal("failed to initialize wg interface")
}
@@ -621,7 +628,7 @@ func TestDNSPermanent_updateHostDNS_emptyUpstream(t *testing.T) {
}
func TestDNSPermanent_updateUpstream(t *testing.T) {
wgIFace, err := createWgInterfaceWithBind(t)
wgIFace, err := createWgInterfaceWithBind(t, false)
if err != nil {
t.Fatal("failed to initialize wg interface")
}
@@ -713,7 +720,7 @@ func TestDNSPermanent_updateUpstream(t *testing.T) {
}
func TestDNSPermanent_matchOnly(t *testing.T) {
wgIFace, err := createWgInterfaceWithBind(t)
wgIFace, err := createWgInterfaceWithBind(t, false)
if err != nil {
t.Fatal("failed to initialize wg interface")
}
@@ -784,7 +791,7 @@ func TestDNSPermanent_matchOnly(t *testing.T) {
}
}
func createWgInterfaceWithBind(t *testing.T) (*iface.WGIface, error) {
func createWgInterfaceWithBind(t *testing.T, enableV6 bool) (*iface.WGIface, error) {
t.Helper()
ov := os.Getenv("NB_WG_KERNEL_DISABLED")
defer t.Setenv("NB_WG_KERNEL_DISABLED", ov)
@@ -797,7 +804,11 @@ func createWgInterfaceWithBind(t *testing.T) (*iface.WGIface, error) {
}
privKey, _ := wgtypes.GeneratePrivateKey()
wgIface, err := iface.NewWGIFace("utun2301", "100.66.100.2/24", 33100, privKey.String(), iface.DefaultMTU, newNet, nil)
v6Addr := ""
if enableV6 {
v6Addr = "fd00:1234:dead:beef::1/128"
}
wgIface, err := iface.NewWGIFace("utun2301", "100.66.100.2/24", v6Addr, 33100, privKey.String(), iface.DefaultMTU, newNet, nil)
if err != nil {
t.Fatalf("build interface wireguard: %v", err)
return nil, err

View File

@@ -5,7 +5,6 @@ import (
"errors"
"fmt"
"net"
"runtime"
"sync"
"sync/atomic"
"time"
@@ -260,13 +259,10 @@ func (u *upstreamResolverBase) disable(err error) {
return
}
// todo test the deactivation logic, it seems to affect the client
if runtime.GOOS != "ios" {
log.Warnf("Upstream resolving is Disabled for %v", reactivatePeriod)
u.deactivate(err)
u.disabled = true
go u.waitUntilResponse()
}
log.Warnf("Upstream resolving is Disabled for %v", reactivatePeriod)
u.deactivate(err)
u.disabled = true
go u.waitUntilResponse()
}
func (u *upstreamResolverBase) testNameserver(server string) error {

View File

@@ -0,0 +1,84 @@
package dns
import (
"context"
"net"
"syscall"
"time"
"github.com/miekg/dns"
"github.com/netbirdio/netbird/client/internal/peer"
nbnet "github.com/netbirdio/netbird/util/net"
)
type upstreamResolver struct {
*upstreamResolverBase
hostsDNSHolder *hostsDNSHolder
}
// newUpstreamResolver in Android we need to distinguish the DNS servers to available through VPN or outside of VPN
// In case if the assigned DNS address is available only in the protected network then the resolver will time out at the
// first time, and we need to wait for a while to start to use again the proper DNS resolver.
func newUpstreamResolver(
ctx context.Context,
_ string,
_ net.IP,
_ *net.IPNet,
statusRecorder *peer.Status,
hostsDNSHolder *hostsDNSHolder,
) (*upstreamResolver, error) {
upstreamResolverBase := newUpstreamResolverBase(ctx, statusRecorder)
c := &upstreamResolver{
upstreamResolverBase: upstreamResolverBase,
hostsDNSHolder: hostsDNSHolder,
}
upstreamResolverBase.upstreamClient = c
return c, nil
}
// exchange in case of Android if the upstream is a local resolver then we do not need to mark the socket as protected.
// In other case the DNS resolvation goes through the VPN, so we need to force to use the
func (u *upstreamResolver) exchange(ctx context.Context, upstream string, r *dns.Msg) (rm *dns.Msg, t time.Duration, err error) {
if u.isLocalResolver(upstream) {
return u.exchangeWithoutVPN(ctx, upstream, r)
} else {
return u.exchangeWithinVPN(ctx, upstream, r)
}
}
func (u *upstreamResolver) exchangeWithinVPN(ctx context.Context, upstream string, r *dns.Msg) (rm *dns.Msg, t time.Duration, err error) {
upstreamExchangeClient := &dns.Client{}
return upstreamExchangeClient.ExchangeContext(ctx, r, upstream)
}
// exchangeWithoutVPN protect the UDP socket by Android SDK to avoid to goes through the VPN
func (u *upstreamResolver) exchangeWithoutVPN(ctx context.Context, upstream string, r *dns.Msg) (rm *dns.Msg, t time.Duration, err error) {
timeout := upstreamTimeout
if deadline, ok := ctx.Deadline(); ok {
timeout = time.Until(deadline)
}
dialTimeout := timeout
nbDialer := nbnet.NewDialer()
dialer := &net.Dialer{
Control: func(network, address string, c syscall.RawConn) error {
return nbDialer.Control(network, address, c)
},
Timeout: dialTimeout,
}
upstreamExchangeClient := &dns.Client{
Dialer: dialer,
}
return upstreamExchangeClient.Exchange(r, upstream)
}
func (u *upstreamResolver) isLocalResolver(upstream string) bool {
if u.hostsDNSHolder.isContain(upstream) {
return true
}
return false
}

View File

@@ -1,4 +1,4 @@
//go:build !ios
//go:build !android && !ios
package dns
@@ -12,7 +12,7 @@ import (
"github.com/netbirdio/netbird/client/internal/peer"
)
type upstreamResolverNonIOS struct {
type upstreamResolver struct {
*upstreamResolverBase
}
@@ -22,16 +22,17 @@ func newUpstreamResolver(
_ net.IP,
_ *net.IPNet,
statusRecorder *peer.Status,
) (*upstreamResolverNonIOS, error) {
_ *hostsDNSHolder,
) (*upstreamResolver, error) {
upstreamResolverBase := newUpstreamResolverBase(ctx, statusRecorder)
nonIOS := &upstreamResolverNonIOS{
nonIOS := &upstreamResolver{
upstreamResolverBase: upstreamResolverBase,
}
upstreamResolverBase.upstreamClient = nonIOS
return nonIOS, nil
}
func (u *upstreamResolverNonIOS) exchange(ctx context.Context, upstream string, r *dns.Msg) (rm *dns.Msg, t time.Duration, err error) {
func (u *upstreamResolver) exchange(ctx context.Context, upstream string, r *dns.Msg) (rm *dns.Msg, t time.Duration, err error) {
upstreamExchangeClient := &dns.Client{}
return upstreamExchangeClient.ExchangeContext(ctx, r, upstream)
}

View File

@@ -28,6 +28,7 @@ func newUpstreamResolver(
ip net.IP,
net *net.IPNet,
statusRecorder *peer.Status,
_ *hostsDNSHolder,
) (*upstreamResolverIOS, error) {
upstreamResolverBase := newUpstreamResolverBase(ctx, statusRecorder)

View File

@@ -58,7 +58,7 @@ func TestUpstreamResolver_ServeDNS(t *testing.T) {
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
ctx, cancel := context.WithCancel(context.TODO())
resolver, _ := newUpstreamResolver(ctx, "", net.IP{}, &net.IPNet{}, nil)
resolver, _ := newUpstreamResolver(ctx, "", net.IP{}, &net.IPNet{}, nil, nil)
resolver.upstreamServers = testCase.InputServers
resolver.upstreamTimeout = testCase.timeout
if testCase.cancelCTX {

View File

@@ -2,7 +2,9 @@ package internal
import (
"context"
"errors"
"fmt"
"maps"
"math/rand"
"net"
"net/netip"
@@ -21,6 +23,7 @@ import (
"github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/internal/acl"
"github.com/netbirdio/netbird/client/internal/dns"
"github.com/netbirdio/netbird/client/internal/networkmonitor"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/relay"
"github.com/netbirdio/netbird/client/internal/rosenpass"
@@ -55,11 +58,15 @@ type EngineConfig struct {
WgIfaceName string
// WgAddr is a Wireguard local address (Netbird Network IP)
WgAddr string
WgAddr string
WgAddr6 string
// WgPrivateKey is a Wireguard private key of our peer (it MUST never leave the machine)
WgPrivateKey wgtypes.Key
// NetworkMonitor is a flag to enable network monitoring
NetworkMonitor bool
// IFaceBlackList is a list of network interfaces to ignore when discovering connection candidates (ICE related)
IFaceBlackList []string
DisableIPv6Discovery bool
@@ -111,9 +118,15 @@ type Engine struct {
// TURNs is a list of STUN servers used by ICE
TURNs []*stun.URI
cancel context.CancelFunc
// clientRoutes is the most recent list of clientRoutes received from the Management Service
clientRoutes route.HAMap
clientRoutesMu sync.RWMutex
ctx context.Context
clientCtx context.Context
clientCancel context.CancelFunc
ctx context.Context
cancel context.CancelFunc
wgInterface *iface.WGIface
wgProxyFactory *wgproxy.Factory
@@ -123,6 +136,8 @@ type Engine struct {
// networkSerial is the latest CurrentSerial (state ID) of the network sent by the Management service
networkSerial uint64
networkMonitor *networkmonitor.NetworkMonitor
sshServerFunc func(hostKeyPEM []byte, addr string) (nbssh.Server, error)
sshServer nbssh.Server
@@ -138,6 +153,8 @@ type Engine struct {
signalProbe *Probe
relayProbe *Probe
wgProbe *Probe
wgConnWorker sync.WaitGroup
}
// Peer is an instance of the Connection Peer
@@ -148,8 +165,8 @@ type Peer struct {
// NewEngine creates a new Connection Engine
func NewEngine(
ctx context.Context,
cancel context.CancelFunc,
clientCtx context.Context,
clientCancel context.CancelFunc,
signalClient signal.Client,
mgmClient mgm.Client,
config *EngineConfig,
@@ -157,8 +174,8 @@ func NewEngine(
statusRecorder *peer.Status,
) *Engine {
return NewEngineWithProbes(
ctx,
cancel,
clientCtx,
clientCancel,
signalClient,
mgmClient,
config,
@@ -173,8 +190,8 @@ func NewEngine(
// NewEngineWithProbes creates a new Connection Engine with probes attached
func NewEngineWithProbes(
ctx context.Context,
cancel context.CancelFunc,
clientCtx context.Context,
clientCancel context.CancelFunc,
signalClient signal.Client,
mgmClient mgm.Client,
config *EngineConfig,
@@ -185,9 +202,10 @@ func NewEngineWithProbes(
relayProbe *Probe,
wgProbe *Probe,
) *Engine {
return &Engine{
ctx: ctx,
cancel: cancel,
clientCtx: clientCtx,
clientCancel: clientCancel,
signal: signalClient,
mgmClient: mgmClient,
peerConns: make(map[string]*peer.Conn),
@@ -199,7 +217,6 @@ func NewEngineWithProbes(
networkSerial: 0,
sshServerFunc: nbssh.DefaultSSHServer,
statusRecorder: statusRecorder,
wgProxyFactory: wgproxy.NewFactory(config.WgPort),
mgmProbe: mgmProbe,
signalProbe: signalProbe,
relayProbe: relayProbe,
@@ -211,16 +228,31 @@ func (e *Engine) Stop() error {
e.syncMsgMux.Lock()
defer e.syncMsgMux.Unlock()
if e.cancel != nil {
e.cancel()
}
// stopping network monitor first to avoid starting the engine again
if e.networkMonitor != nil {
e.networkMonitor.Stop()
}
log.Info("Network monitor: stopped")
err := e.removeAllPeers()
if err != nil {
return err
}
e.clientRoutesMu.Lock()
e.clientRoutes = nil
e.clientRoutesMu.Unlock()
// very ugly but we want to remove peers from the WireGuard interface first before removing interface.
// Removing peers happens in the conn.CLose() asynchronously
// Removing peers happens in the conn.Close() asynchronously
time.Sleep(500 * time.Millisecond)
e.close()
e.wgConnWorker.Wait()
log.Infof("stopped Netbird Engine")
return nil
}
@@ -232,6 +264,13 @@ func (e *Engine) Start() error {
e.syncMsgMux.Lock()
defer e.syncMsgMux.Unlock()
if e.cancel != nil {
e.cancel()
}
e.ctx, e.cancel = context.WithCancel(e.clientCtx)
e.wgProxyFactory = wgproxy.NewFactory(e.ctx, e.config.WgPort)
wgIface, err := e.newWgIface()
if err != nil {
log.Errorf("failed creating wireguard interface instance %s: [%s]", e.config.WgIfaceName, err)
@@ -315,6 +354,9 @@ func (e *Engine) Start() error {
e.receiveManagementEvents()
e.receiveProbeEvents()
// starting network monitor at the very last to avoid disruptions
e.startNetworkMonitor()
return nil
}
@@ -562,6 +604,32 @@ func (e *Engine) updateConfig(conf *mgmProto.PeerConfig) error {
log.Infof("updated peer address from %s to %s", oldAddr, conf.Address)
}
if e.wgInterface.Address6() == nil && conf.Address6 != "" ||
e.wgInterface.Address6() != nil && e.wgInterface.Address6().String() != conf.Address6 {
oldAddr := "none"
if e.wgInterface.Address6() != nil {
oldAddr = e.wgInterface.Address6().String()
}
newAddr := "none"
if conf.Address6 != "" {
newAddr = conf.Address6
}
log.Debugf("updating peer IPv6 address from %s to %s", oldAddr, newAddr)
err := e.wgInterface.UpdateAddr6(conf.Address6)
if err != nil {
return err
}
e.config.WgAddr6 = conf.Address6
err = e.acl.ResetV6Acl()
if err != nil {
return err
}
e.routeManager.ResetV6Routes()
log.Infof("updated peer IPv6 address from %s to %s", oldAddr, conf.Address6)
}
if conf.GetSshConfig() != nil {
err := e.updateSSH(conf.GetSshConfig())
if err != nil {
@@ -571,6 +639,7 @@ func (e *Engine) updateConfig(conf *mgmProto.PeerConfig) error {
e.statusRecorder.UpdateLocalPeerState(peer.LocalPeerState{
IP: e.config.WgAddr,
IP6: e.config.WgAddr6,
PubKey: e.config.WgPrivateKey.PublicKey().String(),
KernelInterface: iface.WireGuardModuleIsLoaded(),
FQDN: conf.GetFqdn(),
@@ -583,12 +652,12 @@ func (e *Engine) updateConfig(conf *mgmProto.PeerConfig) error {
// E.g. when a new peer has been registered and we are allowed to connect to it.
func (e *Engine) receiveManagementEvents() {
go func() {
err := e.mgmClient.Sync(e.handleSync)
err := e.mgmClient.Sync(e.ctx, e.handleSync)
if err != nil {
// happens if management is unavailable for a long time.
// We want to cancel the operation of the whole client
_ = CtxGetState(e.ctx).Wrap(ErrResetConnection)
e.cancel()
e.clientCancel()
return
}
log.Debugf("stopped receiving updates from Management Service")
@@ -695,11 +764,16 @@ func (e *Engine) updateNetworkMap(networkMap *mgmProto.NetworkMap) error {
if protoRoutes == nil {
protoRoutes = []*mgmProto.Route{}
}
err := e.routeManager.UpdateRoutes(serial, toRoutes(protoRoutes))
_, clientRoutes, err := e.routeManager.UpdateRoutes(serial, toRoutes(protoRoutes))
if err != nil {
log.Errorf("failed to update routes, err: %v", err)
log.Errorf("failed to update clientRoutes, err: %v", err)
}
e.clientRoutesMu.Lock()
e.clientRoutes = clientRoutes
e.clientRoutesMu.Unlock()
protoDNSConfig := networkMap.GetDNSConfig()
if protoDNSConfig == nil {
protoDNSConfig = &mgmProto.DNSConfig{}
@@ -728,9 +802,9 @@ func toRoutes(protoRoutes []*mgmProto.Route) []*route.Route {
for _, protoRoute := range protoRoutes {
_, prefix, _ := route.ParseNetwork(protoRoute.Network)
convertedRoute := &route.Route{
ID: protoRoute.ID,
ID: route.ID(protoRoute.ID),
Network: prefix,
NetID: protoRoute.NetID,
NetID: route.NetID(protoRoute.NetID),
NetworkType: route.NetworkType(protoRoute.NetworkType),
Peer: protoRoute.Peer,
Metric: int(protoRoute.Metric),
@@ -832,18 +906,25 @@ func (e *Engine) addNewPeer(peerConfig *mgmProto.RemotePeerConfig) error {
log.Warnf("error adding peer %s to status recorder, got error: %v", peerKey, err)
}
e.wgConnWorker.Add(1)
go e.connWorker(conn, peerKey)
}
return nil
}
func (e *Engine) connWorker(conn *peer.Conn, peerKey string) {
defer e.wgConnWorker.Done()
for {
// randomize starting time a bit
min := 500
max := 2000
time.Sleep(time.Duration(rand.Intn(max-min)+min) * time.Millisecond)
duration := time.Duration(rand.Intn(max-min)+min) * time.Millisecond
select {
case <-e.ctx.Done():
return
case <-time.After(duration):
}
// if peer has been removed -> give up
if !e.peerExists(peerKey) {
@@ -861,11 +942,12 @@ func (e *Engine) connWorker(conn *peer.Conn, peerKey string) {
conn.UpdateStunTurn(append(e.STUNs, e.TURNs...))
e.syncMsgMux.Unlock()
err := conn.Open()
err := conn.Open(e.ctx)
if err != nil {
log.Debugf("connection to peer %s failed: %v", peerKey, err)
switch err.(type) {
case *peer.ConnectionClosedError:
var connectionClosedError *peer.ConnectionClosedError
switch {
case errors.As(err, &connectionClosedError):
// conn has been forced to close, so we exit the loop
return
default:
@@ -976,7 +1058,7 @@ func (e *Engine) createPeerConn(pubKey string, allowedIPs string) (*peer.Conn, e
func (e *Engine) receiveSignalEvents() {
go func() {
// connect to a stream of messages coming from the signal server
err := e.signal.Receive(func(msg *sProto.Message) error {
err := e.signal.Receive(e.ctx, func(msg *sProto.Message) error {
e.syncMsgMux.Lock()
defer e.syncMsgMux.Unlock()
@@ -1040,7 +1122,8 @@ func (e *Engine) receiveSignalEvents() {
log.Errorf("failed on parsing remote candidate %s -> %s", candidate, err)
return err
}
conn.OnRemoteCandidate(candidate)
conn.OnRemoteCandidate(candidate, e.GetClientRoutes())
case sProto.Body_MODE:
}
@@ -1050,7 +1133,7 @@ func (e *Engine) receiveSignalEvents() {
// happens if signal is unavailable for a long time.
// We want to cancel the operation of the whole client
_ = CtxGetState(e.ctx).Wrap(ErrResetConnection)
e.cancel()
e.clientCancel()
return
}
}()
@@ -1111,13 +1194,16 @@ func (e *Engine) parseNATExternalIPMappings() []string {
}
func (e *Engine) close() {
if err := e.wgProxyFactory.Free(); err != nil {
log.Errorf("failed closing ebpf proxy: %s", err)
if e.wgProxyFactory != nil {
if err := e.wgProxyFactory.Free(); err != nil {
log.Errorf("failed closing ebpf proxy: %s", err)
}
}
// stop/restore DNS first so dbus and friends don't complain because of a missing interface
if e.dnsServer != nil {
e.dnsServer.Stop()
e.dnsServer = nil
}
if e.routeManager != nil {
@@ -1180,7 +1266,7 @@ func (e *Engine) newWgIface() (*iface.WGIface, error) {
default:
}
return iface.NewWGIFace(e.config.WgIfaceName, e.config.WgAddr, e.config.WgPort, e.config.WgPrivateKey.String(), iface.DefaultMTU, transportNet, mArgs)
return iface.NewWGIFace(e.config.WgIfaceName, e.config.WgAddr, e.config.WgAddr6, e.config.WgPort, e.config.WgPrivateKey.String(), iface.DefaultMTU, transportNet, mArgs)
}
func (e *Engine) wgInterfaceCreate() (err error) {
@@ -1229,6 +1315,31 @@ func (e *Engine) newDnsServer() ([]*route.Route, dns.Server, error) {
}
}
// GetClientRoutes returns the current routes from the route map
func (e *Engine) GetClientRoutes() route.HAMap {
e.clientRoutesMu.RLock()
defer e.clientRoutesMu.RUnlock()
return maps.Clone(e.clientRoutes)
}
// GetClientRoutesWithNetID returns the current routes from the route map, but the keys consist of the network ID only
func (e *Engine) GetClientRoutesWithNetID() map[route.NetID][]*route.Route {
e.clientRoutesMu.RLock()
defer e.clientRoutesMu.RUnlock()
routes := make(map[route.NetID][]*route.Route, len(e.clientRoutes))
for id, v := range e.clientRoutes {
routes[id.NetID()] = v
}
return routes
}
// GetRouteManager returns the route manager
func (e *Engine) GetRouteManager() routemanager.Manager {
return e.routeManager
}
func findIPFromInterfaceName(ifaceName string) (net.IP, error) {
iface, err := net.InterfaceByName(ifaceName)
if err != nil {
@@ -1329,3 +1440,26 @@ func (e *Engine) probeSTUNs() []relay.ProbeResult {
func (e *Engine) probeTURNs() []relay.ProbeResult {
return relay.ProbeAll(e.ctx, relay.ProbeTURN, e.TURNs)
}
func (e *Engine) startNetworkMonitor() {
if !e.config.NetworkMonitor {
log.Infof("Network monitor is disabled, not starting")
return
}
e.networkMonitor = networkmonitor.New()
go func() {
err := e.networkMonitor.Start(e.ctx, func() {
log.Infof("Network monitor detected network change, restarting engine")
if err := e.Stop(); err != nil {
log.Errorf("Failed to stop engine: %v", err)
}
if err := e.Start(); err != nil {
log.Errorf("Failed to start engine: %v", err)
}
})
if err != nil && !errors.Is(err, networkmonitor.ErrStopped) {
log.Errorf("Network monitor: %v", err)
}
}()
}

View File

@@ -22,6 +22,7 @@ import (
"google.golang.org/grpc/keepalive"
"github.com/netbirdio/management-integrations/integrations"
"github.com/netbirdio/netbird/client/internal/dns"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/routemanager"
@@ -215,7 +216,7 @@ func TestEngine_UpdateNetworkMap(t *testing.T) {
if err != nil {
t.Fatal(err)
}
engine.wgInterface, err = iface.NewWGIFace("utun102", "100.64.0.1/24", engine.config.WgPort, key.String(), iface.DefaultMTU, newNet, nil)
engine.wgInterface, err = iface.NewWGIFace("utun102", "100.64.0.1/24", "", engine.config.WgPort, key.String(), iface.DefaultMTU, newNet, nil)
if err != nil {
t.Fatal(err)
}
@@ -228,6 +229,7 @@ func TestEngine_UpdateNetworkMap(t *testing.T) {
t.Fatal(err)
}
engine.udpMux = bind.NewUniversalUDPMuxDefault(bind.UniversalUDPMuxParams{UDPConn: conn})
engine.ctx = ctx
type testCase struct {
name string
@@ -391,7 +393,7 @@ func TestEngine_Sync(t *testing.T) {
// feed updates to Engine via mocked Management client
updates := make(chan *mgmtProto.SyncResponse)
defer close(updates)
syncFunc := func(msgHandler func(msg *mgmtProto.SyncResponse) error) error {
syncFunc := func(ctx context.Context, msgHandler func(msg *mgmtProto.SyncResponse) error) error {
for msg := range updates {
err := msgHandler(msg)
if err != nil {
@@ -407,6 +409,7 @@ func TestEngine_Sync(t *testing.T) {
WgPrivateKey: key,
WgPort: 33100,
}, MobileDependency{}, peer.NewRecorder("https://mgm"))
engine.ctx = ctx
engine.dnsServer = &dns.MockServer{
UpdateDNSServerFunc: func(serial uint64, update nbdns.Config) error { return nil },
@@ -562,14 +565,16 @@ func TestEngine_UpdateNetworkMapWithRoutes(t *testing.T) {
engine := NewEngine(ctx, cancel, &signal.MockClient{}, &mgmt.MockClient{}, &EngineConfig{
WgIfaceName: wgIfaceName,
WgAddr: wgAddr,
WgAddr6: "",
WgPrivateKey: key,
WgPort: 33100,
}, MobileDependency{}, peer.NewRecorder("https://mgm"))
engine.ctx = ctx
newNet, err := stdnet.NewNet()
if err != nil {
t.Fatal(err)
}
engine.wgInterface, err = iface.NewWGIFace(wgIfaceName, wgAddr, engine.config.WgPort, key.String(), iface.DefaultMTU, newNet, nil)
engine.wgInterface, err = iface.NewWGIFace(wgIfaceName, wgAddr, engine.config.WgAddr6, engine.config.WgPort, key.String(), iface.DefaultMTU, newNet, nil)
assert.NoError(t, err, "shouldn't return error")
input := struct {
inputSerial uint64
@@ -577,10 +582,10 @@ func TestEngine_UpdateNetworkMapWithRoutes(t *testing.T) {
}{}
mockRouteManager := &routemanager.MockManager{
UpdateRoutesFunc: func(updateSerial uint64, newRoutes []*route.Route) error {
UpdateRoutesFunc: func(updateSerial uint64, newRoutes []*route.Route) (map[route.ID]*route.Route, route.HAMap, error) {
input.inputSerial = updateSerial
input.inputRoutes = newRoutes
return testCase.inputErr
return nil, nil, testCase.inputErr
},
}
@@ -597,8 +602,8 @@ func TestEngine_UpdateNetworkMapWithRoutes(t *testing.T) {
err = engine.updateNetworkMap(testCase.networkMap)
assert.NoError(t, err, "shouldn't return error")
assert.Equal(t, testCase.expectedSerial, input.inputSerial, "serial should match")
assert.Len(t, input.inputRoutes, testCase.expectedLen, "routes len should match")
assert.Equal(t, testCase.expectedRoutes, input.inputRoutes, "routes should match")
assert.Len(t, input.inputRoutes, testCase.expectedLen, "clientRoutes len should match")
assert.Equal(t, testCase.expectedRoutes, input.inputRoutes, "clientRoutes should match")
})
}
}
@@ -731,19 +736,22 @@ func TestEngine_UpdateNetworkMapWithDNSUpdate(t *testing.T) {
engine := NewEngine(ctx, cancel, &signal.MockClient{}, &mgmt.MockClient{}, &EngineConfig{
WgIfaceName: wgIfaceName,
WgAddr: wgAddr,
WgAddr6: "",
WgPrivateKey: key,
WgPort: 33100,
}, MobileDependency{}, peer.NewRecorder("https://mgm"))
engine.ctx = ctx
newNet, err := stdnet.NewNet()
if err != nil {
t.Fatal(err)
}
engine.wgInterface, err = iface.NewWGIFace(wgIfaceName, wgAddr, 33100, key.String(), iface.DefaultMTU, newNet, nil)
engine.wgInterface, err = iface.NewWGIFace(wgIfaceName, wgAddr, engine.config.WgAddr6, 33100, key.String(), iface.DefaultMTU, newNet, nil)
assert.NoError(t, err, "shouldn't return error")
mockRouteManager := &routemanager.MockManager{
UpdateRoutesFunc: func(updateSerial uint64, newRoutes []*route.Route) error {
return nil
UpdateRoutesFunc: func(updateSerial uint64, newRoutes []*route.Route) (map[route.ID]*route.Route, route.HAMap, error) {
return nil, nil, nil
},
}
@@ -1002,7 +1010,9 @@ func createEngine(ctx context.Context, cancel context.CancelFunc, setupKey strin
WgPort: wgPort,
}
return NewEngine(ctx, cancel, signalClient, mgmtClient, conf, MobileDependency{}, peer.NewRecorder("https://mgm")), nil
e, err := NewEngine(ctx, cancel, signalClient, mgmtClient, conf, MobileDependency{}, peer.NewRecorder("https://mgm")), nil
e.ctx = ctx
return e, err
}
func startSignal() (*grpc.Server, string, error) {
@@ -1041,7 +1051,7 @@ func startManagement(dataDir string) (*grpc.Server, string, error) {
return nil, "", err
}
s := grpc.NewServer(grpc.KeepaliveEnforcementPolicy(kaep), grpc.KeepaliveParams(kasp))
store, err := server.NewStoreFromJson(config.Datadir, nil)
store, _, err := server.NewTestStoreFromJson(config.Datadir)
if err != nil {
return nil, "", err
}

View File

@@ -68,7 +68,7 @@ func Login(ctx context.Context, config *Config, setupKey string, jwtToken string
}
serverKey, err := doMgmLogin(ctx, mgmClient, pubSSHKey)
if isRegistrationNeeded(err) {
if serverKey != nil && isRegistrationNeeded(err) {
log.Debugf("peer registration required")
_, err = registerPeer(ctx, *serverKey, mgmClient, setupKey, jwtToken, pubSSHKey)
return err

View File

@@ -0,0 +1,21 @@
package networkmonitor
import (
"context"
"errors"
"sync"
)
var ErrStopped = errors.New("monitor has been stopped")
// NetworkMonitor watches for changes in network configuration.
type NetworkMonitor struct {
cancel context.CancelFunc
wg sync.WaitGroup
mu sync.Mutex
}
// New creates a new network monitor.
func New() *NetworkMonitor {
return &NetworkMonitor{}
}

View File

@@ -0,0 +1,133 @@
//go:build (darwin && !ios) || dragonfly || freebsd || netbsd || openbsd
package networkmonitor
import (
"context"
"fmt"
"net"
"net/netip"
"syscall"
"unsafe"
log "github.com/sirupsen/logrus"
"golang.org/x/net/route"
"golang.org/x/sys/unix"
"github.com/netbirdio/netbird/client/internal/routemanager"
)
func checkChange(ctx context.Context, nexthopv4 netip.Addr, intfv4 *net.Interface, nexthopv6 netip.Addr, intfv6 *net.Interface, callback func()) error {
fd, err := unix.Socket(syscall.AF_ROUTE, syscall.SOCK_RAW, syscall.AF_UNSPEC)
if err != nil {
return fmt.Errorf("failed to open routing socket: %v", err)
}
defer func() {
if err := unix.Close(fd); err != nil {
log.Errorf("Network monitor: failed to close routing socket: %v", err)
}
}()
for {
select {
case <-ctx.Done():
return ErrStopped
default:
buf := make([]byte, 2048)
n, err := unix.Read(fd, buf)
if err != nil {
log.Errorf("Network monitor: failed to read from routing socket: %v", err)
continue
}
if n < unix.SizeofRtMsghdr {
log.Errorf("Network monitor: read from routing socket returned less than expected: %d bytes", n)
continue
}
msg := (*unix.RtMsghdr)(unsafe.Pointer(&buf[0]))
switch msg.Type {
// handle interface state changes
case unix.RTM_IFINFO:
ifinfo, err := parseInterfaceMessage(buf[:n])
if err != nil {
log.Errorf("Network monitor: error parsing interface message: %v", err)
continue
}
if msg.Flags&unix.IFF_UP != 0 {
continue
}
if (intfv4 == nil || ifinfo.Index != intfv4.Index) && (intfv6 == nil || ifinfo.Index != intfv6.Index) {
continue
}
log.Infof("Network monitor: monitored interface (%s) is down.", ifinfo.Name)
go callback()
// handle route changes
case unix.RTM_ADD, syscall.RTM_DELETE:
route, err := parseRouteMessage(buf[:n])
if err != nil {
log.Errorf("Network monitor: error parsing routing message: %v", err)
continue
}
if !route.Dst.Addr().IsUnspecified() {
continue
}
intf := "<nil>"
if route.Interface != nil {
intf = route.Interface.Name
}
switch msg.Type {
case unix.RTM_ADD:
log.Infof("Network monitor: default route changed: via %s, interface %s", route.Gw, intf)
go callback()
case unix.RTM_DELETE:
if intfv4 != nil && route.Gw.Compare(nexthopv4) == 0 || intfv6 != nil && route.Gw.Compare(nexthopv6) == 0 {
log.Infof("Network monitor: default route removed: via %s, interface %s", route.Gw, intf)
go callback()
}
}
}
}
}
}
func parseInterfaceMessage(buf []byte) (*route.InterfaceMessage, error) {
msgs, err := route.ParseRIB(route.RIBTypeInterface, buf)
if err != nil {
return nil, fmt.Errorf("parse RIB: %v", err)
}
if len(msgs) != 1 {
return nil, fmt.Errorf("unexpected RIB message msgs: %v", msgs)
}
msg, ok := msgs[0].(*route.InterfaceMessage)
if !ok {
return nil, fmt.Errorf("unexpected RIB message type: %T", msgs[0])
}
return msg, nil
}
func parseRouteMessage(buf []byte) (*routemanager.Route, error) {
msgs, err := route.ParseRIB(route.RIBTypeRoute, buf)
if err != nil {
return nil, fmt.Errorf("parse RIB: %v", err)
}
if len(msgs) != 1 {
return nil, fmt.Errorf("unexpected RIB message msgs: %v", msgs)
}
msg, ok := msgs[0].(*route.RouteMessage)
if !ok {
return nil, fmt.Errorf("unexpected RIB message type: %T", msgs[0])
}
return routemanager.MsgToRoute(msg)
}

View File

@@ -0,0 +1,84 @@
//go:build !ios && !android
package networkmonitor
import (
"context"
"errors"
"fmt"
"net"
"net/netip"
"runtime/debug"
"github.com/cenkalti/backoff/v4"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/routemanager"
)
// Start begins monitoring network changes. When a change is detected, it calls the callback asynchronously and returns.
func (nw *NetworkMonitor) Start(ctx context.Context, callback func()) (err error) {
if ctx.Err() != nil {
return ctx.Err()
}
nw.mu.Lock()
ctx, nw.cancel = context.WithCancel(ctx)
nw.mu.Unlock()
nw.wg.Add(1)
defer nw.wg.Done()
var nexthop4, nexthop6 netip.Addr
var intf4, intf6 *net.Interface
operation := func() error {
var errv4, errv6 error
nexthop4, intf4, errv4 = routemanager.GetNextHop(netip.IPv4Unspecified())
nexthop6, intf6, errv6 = routemanager.GetNextHop(netip.IPv6Unspecified())
if errv4 != nil && errv6 != nil {
return errors.New("failed to get default next hops")
}
if errv4 == nil {
log.Debugf("Network monitor: IPv4 default route: %s, interface: %s", nexthop4, intf4.Name)
}
if errv6 == nil {
log.Debugf("Network monitor: IPv6 default route: %s, interface: %s", nexthop6, intf6.Name)
}
// continue if either route was found
return nil
}
expBackOff := backoff.WithContext(backoff.NewExponentialBackOff(), ctx)
if err := backoff.Retry(operation, expBackOff); err != nil {
return fmt.Errorf("failed to get default next hops: %w", err)
}
// recover in case sys ops panic
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("panic occurred: %v, stack trace: %s", r, string(debug.Stack()))
}
}()
if err := checkChange(ctx, nexthop4, intf4, nexthop6, intf6, callback); err != nil {
return fmt.Errorf("check change: %w", err)
}
return nil
}
// Stop stops the network monitor.
func (nw *NetworkMonitor) Stop() {
nw.mu.Lock()
defer nw.mu.Unlock()
if nw.cancel != nil {
nw.cancel()
nw.wg.Wait()
}
}

View File

@@ -0,0 +1,81 @@
//go:build !android
package networkmonitor
import (
"context"
"errors"
"fmt"
"net"
"net/netip"
"syscall"
log "github.com/sirupsen/logrus"
"github.com/vishvananda/netlink"
)
func checkChange(ctx context.Context, nexthopv4 netip.Addr, intfv4 *net.Interface, nexthop6 netip.Addr, intfv6 *net.Interface, callback func()) error {
if intfv4 == nil && intfv6 == nil {
return errors.New("no interfaces available")
}
linkChan := make(chan netlink.LinkUpdate)
done := make(chan struct{})
defer close(done)
if err := netlink.LinkSubscribe(linkChan, done); err != nil {
return fmt.Errorf("subscribe to link updates: %v", err)
}
routeChan := make(chan netlink.RouteUpdate)
if err := netlink.RouteSubscribe(routeChan, done); err != nil {
return fmt.Errorf("subscribe to route updates: %v", err)
}
log.Info("Network monitor: started")
for {
select {
case <-ctx.Done():
return ErrStopped
// handle interface state changes
case update := <-linkChan:
if (intfv4 == nil || update.Index != int32(intfv4.Index)) && (intfv6 == nil || update.Index != int32(intfv6.Index)) {
continue
}
switch update.Header.Type {
case syscall.RTM_DELLINK:
log.Infof("Network monitor: monitored interface (%s) is gone", update.Link.Attrs().Name)
go callback()
return nil
case syscall.RTM_NEWLINK:
if (update.IfInfomsg.Flags&syscall.IFF_RUNNING) == 0 && update.Link.Attrs().OperState == netlink.OperDown {
log.Infof("Network monitor: monitored interface (%s) is down.", update.Link.Attrs().Name)
go callback()
return nil
}
}
// handle route changes
case route := <-routeChan:
// default route and main table
if route.Dst != nil || route.Table != syscall.RT_TABLE_MAIN {
continue
}
switch route.Type {
// triggered on added/replaced routes
case syscall.RTM_NEWROUTE:
log.Infof("Network monitor: default route changed: via %s, interface %d", route.Gw, route.LinkIndex)
go callback()
return nil
case syscall.RTM_DELROUTE:
if intfv4 != nil && route.Gw.Equal(nexthopv4.AsSlice()) || intfv6 != nil && route.Gw.Equal(nexthop6.AsSlice()) {
log.Infof("Network monitor: default route removed: via %s, interface %d", route.Gw, route.LinkIndex)
go callback()
return nil
}
}
}
}
}

View File

@@ -0,0 +1,12 @@
//go:build ios || android
package networkmonitor
import "context"
func (nw *NetworkMonitor) Start(context.Context, func()) error {
return nil
}
func (nw *NetworkMonitor) Stop() {
}

View File

@@ -0,0 +1,215 @@
package networkmonitor
import (
"context"
"fmt"
"net"
"net/netip"
"time"
log "github.com/sirupsen/logrus"
"github.com/netbirdio/netbird/client/internal/routemanager"
)
const (
unreachable = 0
incomplete = 1
probe = 2
delay = 3
stale = 4
reachable = 5
permanent = 6
tbd = 7
)
const interval = 10 * time.Second
func checkChange(ctx context.Context, nexthopv4 netip.Addr, intfv4 *net.Interface, nexthopv6 netip.Addr, intfv6 *net.Interface, callback func()) error {
var neighborv4, neighborv6 *routemanager.Neighbor
{
initialNeighbors, err := getNeighbors()
if err != nil {
return fmt.Errorf("get neighbors: %w", err)
}
if n, ok := initialNeighbors[nexthopv4]; ok {
neighborv4 = &n
}
if n, ok := initialNeighbors[nexthopv6]; ok {
neighborv6 = &n
}
}
log.Debugf("Network monitor: initial IPv4 neighbor: %v, IPv6 neighbor: %v", neighborv4, neighborv6)
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return ErrStopped
case <-ticker.C:
if changed(nexthopv4, intfv4, neighborv4, nexthopv6, intfv6, neighborv6) {
go callback()
return nil
}
}
}
}
func changed(
nexthopv4 netip.Addr,
intfv4 *net.Interface,
neighborv4 *routemanager.Neighbor,
nexthopv6 netip.Addr,
intfv6 *net.Interface,
neighborv6 *routemanager.Neighbor,
) bool {
neighbors, err := getNeighbors()
if err != nil {
log.Errorf("network monitor: error fetching current neighbors: %v", err)
return false
}
if neighborChanged(nexthopv4, neighborv4, neighbors) || neighborChanged(nexthopv6, neighborv6, neighbors) {
return true
}
routes, err := getRoutes()
if err != nil {
log.Errorf("network monitor: error fetching current routes: %v", err)
return false
}
if routeChanged(nexthopv4, intfv4, routes) || routeChanged(nexthopv6, intfv6, routes) {
return true
}
return false
}
// routeChanged checks if the default routes still point to our nexthop/interface
func routeChanged(nexthop netip.Addr, intf *net.Interface, routes map[netip.Prefix]routemanager.Route) bool {
if !nexthop.IsValid() {
return false
}
var unspec netip.Prefix
if nexthop.Is6() {
unspec = netip.PrefixFrom(netip.IPv6Unspecified(), 0)
} else {
unspec = netip.PrefixFrom(netip.IPv4Unspecified(), 0)
}
if r, ok := routes[unspec]; ok {
if r.Nexthop != nexthop || compareIntf(r.Interface, intf) != 0 {
intf := "<nil>"
if r.Interface != nil {
intf = r.Interface.Name
}
log.Infof("network monitor: default route changed: %s via %s (%s)", r.Destination, r.Nexthop, intf)
return true
}
} else {
log.Infof("network monitor: default route is gone")
return true
}
return false
}
func neighborChanged(nexthop netip.Addr, neighbor *routemanager.Neighbor, neighbors map[netip.Addr]routemanager.Neighbor) bool {
if neighbor == nil {
return false
}
// TODO: consider non-local nexthops, e.g. on point-to-point interfaces
if n, ok := neighbors[nexthop]; ok {
if n.State != reachable && n.State != permanent {
log.Infof("network monitor: neighbor %s (%s) is not reachable: %s", neighbor.IPAddress, neighbor.LinkLayerAddress, stateFromInt(n.State))
return true
} else if n.InterfaceIndex != neighbor.InterfaceIndex {
log.Infof(
"network monitor: neighbor %s (%s) changed interface from '%s' (%d) to '%s' (%d): %s",
neighbor.IPAddress,
neighbor.LinkLayerAddress,
neighbor.InterfaceAlias,
neighbor.InterfaceIndex,
n.InterfaceAlias,
n.InterfaceIndex,
stateFromInt(n.State),
)
return true
}
} else {
log.Infof("network monitor: neighbor %s (%s) is gone", neighbor.IPAddress, neighbor.LinkLayerAddress)
return true
}
return false
}
func getNeighbors() (map[netip.Addr]routemanager.Neighbor, error) {
entries, err := routemanager.GetNeighbors()
if err != nil {
return nil, fmt.Errorf("get neighbors: %w", err)
}
neighbours := make(map[netip.Addr]routemanager.Neighbor, len(entries))
for _, entry := range entries {
neighbours[entry.IPAddress] = entry
}
return neighbours, nil
}
func getRoutes() (map[netip.Prefix]routemanager.Route, error) {
entries, err := routemanager.GetRoutes()
if err != nil {
return nil, fmt.Errorf("get routes: %w", err)
}
routes := make(map[netip.Prefix]routemanager.Route, len(entries))
for _, entry := range entries {
routes[entry.Destination] = entry
}
return routes, nil
}
func stateFromInt(state uint8) string {
switch state {
case unreachable:
return "unreachable"
case incomplete:
return "incomplete"
case probe:
return "probe"
case delay:
return "delay"
case stale:
return "stale"
case reachable:
return "reachable"
case permanent:
return "permanent"
case tbd:
return "tbd"
default:
return "unknown"
}
}
func compareIntf(a, b *net.Interface) int {
if a == nil && b == nil {
return 0
}
if a == nil {
return -1
}
if b == nil {
return 1
}
return a.Index - b.Index
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"net"
"net/netip"
"runtime"
"strings"
"sync"
@@ -18,6 +19,7 @@ import (
"github.com/netbirdio/netbird/client/internal/wgproxy"
"github.com/netbirdio/netbird/iface"
"github.com/netbirdio/netbird/iface/bind"
"github.com/netbirdio/netbird/route"
signal "github.com/netbirdio/netbird/signal/client"
sProto "github.com/netbirdio/netbird/signal/proto"
nbnet "github.com/netbirdio/netbird/util/net"
@@ -276,7 +278,7 @@ func (conn *Conn) candidateTypes() []ice.CandidateType {
// Open opens connection to the remote peer starting ICE candidate gathering process.
// Blocks until connection has been closed or connection timeout.
// ConnStatus will be set accordingly
func (conn *Conn) Open() error {
func (conn *Conn) Open(ctx context.Context) error {
log.Debugf("trying to connect to peer %s", conn.config.Key)
peerState := State{
@@ -336,7 +338,7 @@ func (conn *Conn) Open() error {
// at this point we received offer/answer and we are ready to gather candidates
conn.mu.Lock()
conn.status = StatusConnecting
conn.ctx, conn.notifyDisconnected = context.WithCancel(context.Background())
conn.ctx, conn.notifyDisconnected = context.WithCancel(ctx)
defer conn.notifyDisconnected()
conn.mu.Unlock()
@@ -353,7 +355,7 @@ func (conn *Conn) Open() error {
err = conn.agent.GatherCandidates()
if err != nil {
return err
return fmt.Errorf("gather candidates: %v", err)
}
// will block until connection succeeded
@@ -370,7 +372,7 @@ func (conn *Conn) Open() error {
return err
}
// dynamically set remote WireGuard port is other side specified a different one from the default one
// dynamically set remote WireGuard port if other side specified a different one from the default one
remoteWgPort := iface.DefaultWgPort
if remoteOfferAnswer.WgListenPort != 0 {
remoteWgPort = remoteOfferAnswer.WgListenPort
@@ -423,7 +425,7 @@ func (conn *Conn) configureConnection(remoteConn net.Conn, remoteWgPort int, rem
var endpoint net.Addr
if isRelayCandidate(pair.Local) {
log.Debugf("setup relay connection")
conn.wgProxy = conn.wgProxyFactory.GetProxy()
conn.wgProxy = conn.wgProxyFactory.GetProxy(conn.ctx)
endpoint, err = conn.wgProxy.AddTurnConn(remoteConn)
if err != nil {
return nil, err
@@ -448,9 +450,11 @@ func (conn *Conn) configureConnection(remoteConn net.Conn, remoteWgPort int, rem
err = conn.config.WgConfig.WgInterface.UpdatePeer(conn.config.WgConfig.RemoteKey, conn.config.WgConfig.AllowedIps, defaultWgKeepAlive, endpointUdpAddr, conn.config.WgConfig.PreSharedKey)
if err != nil {
if conn.wgProxy != nil {
_ = conn.wgProxy.CloseConn()
if err := conn.wgProxy.CloseConn(); err != nil {
log.Warnf("Failed to close turn connection: %v", err)
}
}
return nil, err
return nil, fmt.Errorf("update peer: %w", err)
}
conn.status = StatusConnected
@@ -480,11 +484,15 @@ func (conn *Conn) configureConnection(remoteConn net.Conn, remoteWgPort int, rem
log.Warnf("unable to save peer's state, got error: %v", err)
}
_, ipNet, err := net.ParseCIDR(conn.config.WgConfig.AllowedIps)
_, ipNet, err := net.ParseCIDR(strings.Split(conn.config.WgConfig.AllowedIps, ",")[0])
if err != nil {
return nil, err
}
if runtime.GOOS == "ios" {
runtime.GC()
}
if conn.onConnected != nil {
conn.onConnected(conn.config.Key, remoteRosenpassPubKey, ipNet.IP.String(), remoteRosenpassAddr)
}
@@ -730,7 +738,7 @@ func (conn *Conn) Close() error {
// before conn.Open() another update from management arrives with peers: [1,2,3,4,5]
// engine adds a new Conn for 4 and 5
// therefore peer 4 has 2 Conn objects
log.Warnf("connection has been already closed or attempted closing not started coonection %s", conn.config.Key)
log.Warnf("Connection has been already closed or attempted closing not started connection %s", conn.config.Key)
return NewConnectionAlreadyClosed(conn.config.Key)
}
}
@@ -773,7 +781,7 @@ func (conn *Conn) OnRemoteAnswer(answer OfferAnswer) bool {
}
// OnRemoteCandidate Handles ICE connection Candidate provided by the remote peer.
func (conn *Conn) OnRemoteCandidate(candidate ice.Candidate) {
func (conn *Conn) OnRemoteCandidate(candidate ice.Candidate, haRoutes route.HAMap) {
log.Debugf("OnRemoteCandidate from peer %s -> %s", conn.config.Key, candidate.String())
go func() {
conn.mu.Lock()
@@ -783,6 +791,10 @@ func (conn *Conn) OnRemoteCandidate(candidate ice.Candidate) {
return
}
if candidateViaRoutes(candidate, haRoutes) {
return
}
err := conn.agent.AddRemoteCandidate(candidate)
if err != nil {
log.Errorf("error while handling remote candidate from peer %s", conn.config.Key)
@@ -800,3 +812,31 @@ func (conn *Conn) RegisterProtoSupportMeta(support []uint32) {
protoSupport := signal.ParseFeaturesSupported(support)
conn.meta.protoSupport = protoSupport
}
func candidateViaRoutes(candidate ice.Candidate, clientRoutes route.HAMap) bool {
var routePrefixes []netip.Prefix
for _, routes := range clientRoutes {
if len(routes) > 0 && routes[0] != nil {
routePrefixes = append(routePrefixes, routes[0].Network)
}
}
addr, err := netip.ParseAddr(candidate.Address())
if err != nil {
log.Errorf("Failed to parse IP address %s: %v", candidate.Address(), err)
return false
}
for _, prefix := range routePrefixes {
// default route is
if prefix.Bits() == 0 {
continue
}
if prefix.Contains(addr) {
log.Debugf("Ignoring candidate [%s], its address is part of routed network %s", candidate.String(), prefix)
return true
}
}
return false
}

View File

@@ -1,6 +1,7 @@
package peer
import (
"context"
"sync"
"testing"
"time"
@@ -35,7 +36,7 @@ func TestNewConn_interfaceFilter(t *testing.T) {
}
func TestConn_GetKey(t *testing.T) {
wgProxyFactory := wgproxy.NewFactory(connConf.LocalWgPort)
wgProxyFactory := wgproxy.NewFactory(context.Background(), connConf.LocalWgPort)
defer func() {
_ = wgProxyFactory.Free()
}()
@@ -50,7 +51,7 @@ func TestConn_GetKey(t *testing.T) {
}
func TestConn_OnRemoteOffer(t *testing.T) {
wgProxyFactory := wgproxy.NewFactory(connConf.LocalWgPort)
wgProxyFactory := wgproxy.NewFactory(context.Background(), connConf.LocalWgPort)
defer func() {
_ = wgProxyFactory.Free()
}()
@@ -87,7 +88,7 @@ func TestConn_OnRemoteOffer(t *testing.T) {
}
func TestConn_OnRemoteAnswer(t *testing.T) {
wgProxyFactory := wgproxy.NewFactory(connConf.LocalWgPort)
wgProxyFactory := wgproxy.NewFactory(context.Background(), connConf.LocalWgPort)
defer func() {
_ = wgProxyFactory.Free()
}()
@@ -123,7 +124,7 @@ func TestConn_OnRemoteAnswer(t *testing.T) {
wg.Wait()
}
func TestConn_Status(t *testing.T) {
wgProxyFactory := wgproxy.NewFactory(connConf.LocalWgPort)
wgProxyFactory := wgproxy.NewFactory(context.Background(), connConf.LocalWgPort)
defer func() {
_ = wgProxyFactory.Free()
}()
@@ -153,7 +154,7 @@ func TestConn_Status(t *testing.T) {
}
func TestConn_Close(t *testing.T) {
wgProxyFactory := wgproxy.NewFactory(connConf.LocalWgPort)
wgProxyFactory := wgproxy.NewFactory(context.Background(), connConf.LocalWgPort)
defer func() {
_ = wgProxyFactory.Free()
}()

View File

@@ -68,6 +68,7 @@ func (s *State) GetRoutes() map[string]struct{} {
// LocalPeerState contains the latest state of the local peer
type LocalPeerState struct {
IP string
IP6 string
PubKey string
KernelInterface bool
FQDN string

View File

@@ -170,7 +170,7 @@ func ProbeAll(
var wg sync.WaitGroup
for i, uri := range relays {
ctx, cancel := context.WithTimeout(ctx, 1*time.Second)
ctx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
wg.Add(1)

View File

@@ -3,6 +3,7 @@ package routemanager
import (
"context"
"fmt"
"net"
"net/netip"
"time"
@@ -32,11 +33,12 @@ type clientNetwork struct {
stop context.CancelFunc
statusRecorder *peer.Status
wgInterface *iface.WGIface
routes map[string]*route.Route
routes map[route.ID]*route.Route
routeUpdate chan routesUpdate
peerStateUpdate chan struct{}
routePeersNotifiers map[string]chan struct{}
chosenRoute *route.Route
chosenIP *net.IP
network netip.Prefix
updateSerial uint64
}
@@ -49,7 +51,7 @@ func newClientNetworkWatcher(ctx context.Context, wgInterface *iface.WGIface, st
stop: cancel,
statusRecorder: statusRecorder,
wgInterface: wgInterface,
routes: make(map[string]*route.Route),
routes: make(map[route.ID]*route.Route),
routePeersNotifiers: make(map[string]chan struct{}),
routeUpdate: make(chan routesUpdate),
peerStateUpdate: make(chan struct{}),
@@ -58,8 +60,8 @@ func newClientNetworkWatcher(ctx context.Context, wgInterface *iface.WGIface, st
return client
}
func (c *clientNetwork) getRouterPeerStatuses() map[string]routerPeerStatus {
routePeerStatuses := make(map[string]routerPeerStatus)
func (c *clientNetwork) getRouterPeerStatuses() map[route.ID]routerPeerStatus {
routePeerStatuses := make(map[route.ID]routerPeerStatus)
for _, r := range c.routes {
peerStatus, err := c.statusRecorder.GetPeer(r.Peer)
if err != nil {
@@ -89,12 +91,12 @@ func (c *clientNetwork) getRouterPeerStatuses() map[string]routerPeerStatus {
// * Latency: Routes with lower latency are prioritized.
//
// It returns the ID of the selected optimal route.
func (c *clientNetwork) getBestRouteFromStatuses(routePeerStatuses map[string]routerPeerStatus) string {
chosen := ""
func (c *clientNetwork) getBestRouteFromStatuses(routePeerStatuses map[route.ID]routerPeerStatus) route.ID {
chosen := route.ID("")
chosenScore := float64(0)
currScore := float64(0)
currID := ""
currID := route.ID("")
if c.chosenRoute != nil {
currID = c.chosenRoute.ID
}
@@ -152,11 +154,16 @@ func (c *clientNetwork) getBestRouteFromStatuses(routePeerStatuses map[string]ro
log.Warnf("the network %s has not been assigned a routing peer as no peers from the list %s are currently connected", c.network, peers)
case chosen != currID:
if currScore != 0 && currScore < chosenScore+0.1 {
// we compare the current score + 10ms to the chosen score to avoid flapping between routes
if currScore != 0 && currScore+0.01 > chosenScore {
log.Debugf("keeping current routing peer because the score difference with latency is less than 0.01(10ms), current: %f, new: %f", currScore, chosenScore)
return currID
} else {
log.Infof("new chosen route is %s with peer %s with score %f for network %s", chosen, c.routes[chosen].Peer, chosenScore, c.network)
}
var p string
if rt := c.routes[chosen]; rt != nil {
p = rt.Peer
}
log.Infof("new chosen route is %s with peer %s with score %f for network %s", chosen, p, chosenScore, c.network)
}
return chosen
@@ -215,7 +222,8 @@ func (c *clientNetwork) removeRouteFromWireguardPeer(peerKey string) error {
func (c *clientNetwork) removeRouteFromPeerAndSystem() error {
if c.chosenRoute != nil {
if err := removeVPNRoute(c.network, c.wgInterface.Name()); err != nil {
// TODO IPv6 (pass wgInterface)
if err := removeVPNRoute(c.network, c.getAsInterface()); err != nil {
return fmt.Errorf("remove route %s from system, err: %v", c.network, err)
}
@@ -255,10 +263,20 @@ func (c *clientNetwork) recalculateRouteAndUpdatePeerAndSystem() error {
return fmt.Errorf("remove route from peer: %v", err)
}
} else {
// TODO recheck IPv6
gwAddr := c.wgInterface.Address().IP
c.chosenIP = &gwAddr
if c.network.Addr().Is6() {
if c.wgInterface.Address6() == nil {
return fmt.Errorf("Could not assign IPv6 route %s for peer %s because no IPv6 address is assigned",
c.network.String(), c.wgInterface.Address().IP.String())
}
c.chosenIP = &c.wgInterface.Address6().IP
}
// otherwise add the route to the system
if err := addVPNRoute(c.network, c.wgInterface.Name()); err != nil {
if err := addVPNRoute(c.network, c.getAsInterface()); err != nil {
return fmt.Errorf("route %s couldn't be added for peer %s, err: %v",
c.network.String(), c.wgInterface.Address().IP.String(), err)
c.network.String(), c.chosenIP.String(), err)
}
}
@@ -289,7 +307,7 @@ func (c *clientNetwork) sendUpdateToClientNetworkWatcher(update routesUpdate) {
}
func (c *clientNetwork) handleUpdate(update routesUpdate) {
updateMap := make(map[string]*route.Route)
updateMap := make(map[route.ID]*route.Route)
for _, r := range update.routes {
updateMap[r.ID] = r
@@ -344,3 +362,15 @@ func (c *clientNetwork) peersStateAndUpdateWatcher() {
}
}
}
func (c *clientNetwork) getAsInterface() *net.Interface {
intf, err := net.InterfaceByName(c.wgInterface.Name())
if err != nil {
log.Warnf("Couldn't get interface by name %s: %v", c.wgInterface.Name(), err)
intf = &net.Interface{
Name: c.wgInterface.Name(),
}
}
return intf
}

View File

@@ -12,21 +12,21 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
testCases := []struct {
name string
statuses map[string]routerPeerStatus
expectedRouteID string
currentRoute string
existingRoutes map[string]*route.Route
statuses map[route.ID]routerPeerStatus
expectedRouteID route.ID
currentRoute route.ID
existingRoutes map[route.ID]*route.Route
}{
{
name: "one route",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: false,
direct: true,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -38,14 +38,14 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "one connected routes with relayed and direct",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: true,
direct: true,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -57,14 +57,14 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "one connected routes with relayed and no direct",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: true,
direct: false,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -76,14 +76,14 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "no connected peers",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: false,
relayed: false,
direct: false,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -95,7 +95,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "multiple connected peers with different metrics",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: false,
@@ -107,7 +107,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
direct: true,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: 9000,
@@ -124,7 +124,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "multiple connected peers with one relayed",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: false,
@@ -136,7 +136,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
direct: true,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -153,7 +153,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "multiple connected peers with one direct",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: false,
@@ -165,7 +165,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
direct: false,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -182,7 +182,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "multiple connected peers with different latencies",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
latency: 300 * time.Millisecond,
@@ -192,7 +192,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
latency: 10 * time.Millisecond,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -209,7 +209,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "should ignore routes with latency 0",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
latency: 0 * time.Millisecond,
@@ -219,7 +219,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
latency: 10 * time.Millisecond,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -236,12 +236,12 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
},
{
name: "current route with similar score and similar but slightly worse latency should not change",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: false,
direct: true,
latency: 12 * time.Millisecond,
latency: 15 * time.Millisecond,
},
"route2": {
connected: true,
@@ -250,7 +250,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
latency: 10 * time.Millisecond,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
@@ -265,9 +265,40 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
currentRoute: "route1",
expectedRouteID: "route1",
},
{
name: "current route with bad score should be changed to route with better score",
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: false,
direct: true,
latency: 200 * time.Millisecond,
},
"route2": {
connected: true,
relayed: false,
direct: true,
latency: 10 * time.Millisecond,
},
},
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,
Peer: "peer1",
},
"route2": {
ID: "route2",
Metric: route.MaxMetric,
Peer: "peer2",
},
},
currentRoute: "route1",
expectedRouteID: "route2",
},
{
name: "current chosen route doesn't exist anymore",
statuses: map[string]routerPeerStatus{
statuses: map[route.ID]routerPeerStatus{
"route1": {
connected: true,
relayed: false,
@@ -281,7 +312,7 @@ func TestGetBestrouteFromStatuses(t *testing.T) {
latency: 10 * time.Millisecond,
},
},
existingRoutes: map[string]*route.Route{
existingRoutes: map[route.ID]*route.Route{
"route1": {
ID: "route1",
Metric: route.MaxMetric,

View File

@@ -14,6 +14,7 @@ import (
firewall "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/internal/listener"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/routeselector"
"github.com/netbirdio/netbird/iface"
"github.com/netbirdio/netbird/route"
nbnet "github.com/netbirdio/netbird/util/net"
@@ -28,9 +29,12 @@ var defaultv6 = netip.PrefixFrom(netip.IPv6Unspecified(), 0)
// Manager is a route manager interface
type Manager interface {
Init() (peer.BeforeAddPeerHookFunc, peer.AfterRemovePeerHookFunc, error)
UpdateRoutes(updateSerial uint64, newRoutes []*route.Route) error
UpdateRoutes(updateSerial uint64, newRoutes []*route.Route) (map[route.ID]*route.Route, route.HAMap, error)
TriggerSelection(route.HAMap)
GetRouteSelector() *routeselector.RouteSelector
SetRouteChangeListener(listener listener.NetworkChangeListener)
InitialRouteRange() []string
ResetV6Routes()
EnableServerRouter(firewall firewall.Manager) error
Stop()
}
@@ -40,7 +44,8 @@ type DefaultManager struct {
ctx context.Context
stop context.CancelFunc
mux sync.Mutex
clientNetworks map[string]*clientNetwork
clientNetworks map[route.HAUniqueID]*clientNetwork
routeSelector *routeselector.RouteSelector
serverRouter serverRouter
statusRecorder *peer.Status
wgInterface *iface.WGIface
@@ -53,7 +58,8 @@ func NewManager(ctx context.Context, pubKey string, wgInterface *iface.WGIface,
dm := &DefaultManager{
ctx: mCTX,
stop: cancel,
clientNetworks: make(map[string]*clientNetwork),
clientNetworks: make(map[route.HAUniqueID]*clientNetwork),
routeSelector: routeselector.NewRouteSelector(),
statusRecorder: statusRecorder,
wgInterface: wgInterface,
pubKey: pubKey,
@@ -117,28 +123,41 @@ func (m *DefaultManager) Stop() {
}
// UpdateRoutes compares received routes with existing routes and removes, updates or adds them to the client and server maps
func (m *DefaultManager) UpdateRoutes(updateSerial uint64, newRoutes []*route.Route) error {
func (m *DefaultManager) UpdateRoutes(updateSerial uint64, newRoutes []*route.Route) (map[route.ID]*route.Route, route.HAMap, error) {
select {
case <-m.ctx.Done():
log.Infof("not updating routes as context is closed")
return m.ctx.Err()
return nil, nil, m.ctx.Err()
default:
m.mux.Lock()
defer m.mux.Unlock()
newServerRoutesMap, newClientRoutesIDMap := m.classifiesRoutes(newRoutes)
newServerRoutesMap, newClientRoutesIDMap := m.classifyRoutes(newRoutes)
m.updateClientNetworks(updateSerial, newClientRoutesIDMap)
m.notifier.onNewRoutes(newClientRoutesIDMap)
filteredClientRoutes := m.routeSelector.FilterSelected(newClientRoutesIDMap)
m.updateClientNetworks(updateSerial, filteredClientRoutes)
m.notifier.onNewRoutes(filteredClientRoutes)
if m.serverRouter != nil {
err := m.serverRouter.updateRoutes(newServerRoutesMap)
if err != nil {
return fmt.Errorf("update routes: %w", err)
return nil, nil, fmt.Errorf("update routes: %w", err)
}
}
return nil
return newServerRoutesMap, newClientRoutesIDMap, nil
}
}
// ResetV6Routes deletes all IPv6 routes (necessary if IPv6 address changes).
// It is expected that UpdateRoute is called afterwards to recreate the routing table.
func (m *DefaultManager) ResetV6Routes() {
for id, client := range m.clientNetworks {
if client.network.Addr().Is6() {
log.Debugf("stopping client network watcher due to IPv6 address change, %s", id)
client.stop()
delete(m.clientNetworks, id)
}
}
}
@@ -149,19 +168,57 @@ func (m *DefaultManager) SetRouteChangeListener(listener listener.NetworkChangeL
// InitialRouteRange return the list of initial routes. It used by mobile systems
func (m *DefaultManager) InitialRouteRange() []string {
return m.notifier.initialRouteRanges()
return m.notifier.getInitialRouteRanges()
}
func (m *DefaultManager) updateClientNetworks(updateSerial uint64, networks map[string][]*route.Route) {
// removing routes that do not exist as per the update from the Management service.
// GetRouteSelector returns the route selector
func (m *DefaultManager) GetRouteSelector() *routeselector.RouteSelector {
return m.routeSelector
}
// GetClientRoutes returns the client routes
func (m *DefaultManager) GetClientRoutes() map[route.HAUniqueID]*clientNetwork {
return m.clientNetworks
}
// TriggerSelection triggers the selection of routes, stopping deselected watchers and starting newly selected ones
func (m *DefaultManager) TriggerSelection(networks route.HAMap) {
m.mux.Lock()
defer m.mux.Unlock()
networks = m.routeSelector.FilterSelected(networks)
m.notifier.onNewRoutes(networks)
m.stopObsoleteClients(networks)
for id, routes := range networks {
if _, found := m.clientNetworks[id]; found {
// don't touch existing client network watchers
continue
}
clientNetworkWatcher := newClientNetworkWatcher(m.ctx, m.wgInterface, m.statusRecorder, routes[0].Network)
m.clientNetworks[id] = clientNetworkWatcher
go clientNetworkWatcher.peersStateAndUpdateWatcher()
clientNetworkWatcher.sendUpdateToClientNetworkWatcher(routesUpdate{routes: routes})
}
}
// stopObsoleteClients stops the client network watcher for the networks that are not in the new list
func (m *DefaultManager) stopObsoleteClients(networks route.HAMap) {
for id, client := range m.clientNetworks {
_, found := networks[id]
if !found {
log.Debugf("stopping client network watcher, %s", id)
if _, ok := networks[id]; !ok {
log.Debugf("Stopping client network watcher, %s", id)
client.stop()
delete(m.clientNetworks, id)
}
}
}
func (m *DefaultManager) updateClientNetworks(updateSerial uint64, networks route.HAMap) {
// removing routes that do not exist as per the update from the Management service.
m.stopObsoleteClients(networks)
for id, routes := range networks {
clientNetworkWatcher, found := m.clientNetworks[id]
@@ -178,15 +235,15 @@ func (m *DefaultManager) updateClientNetworks(updateSerial uint64, networks map[
}
}
func (m *DefaultManager) classifiesRoutes(newRoutes []*route.Route) (map[string]*route.Route, map[string][]*route.Route) {
newClientRoutesIDMap := make(map[string][]*route.Route)
newServerRoutesMap := make(map[string]*route.Route)
ownNetworkIDs := make(map[string]bool)
func (m *DefaultManager) classifyRoutes(newRoutes []*route.Route) (map[route.ID]*route.Route, route.HAMap) {
newClientRoutesIDMap := make(route.HAMap)
newServerRoutesMap := make(map[route.ID]*route.Route)
ownNetworkIDs := make(map[route.HAUniqueID]bool)
for _, newRoute := range newRoutes {
networkID := route.GetHAUniqueID(newRoute)
haID := route.GetHAUniqueID(newRoute)
if newRoute.Peer == m.pubKey {
ownNetworkIDs[networkID] = true
ownNetworkIDs[haID] = true
// only linux is supported for now
if runtime.GOOS != "linux" {
log.Warnf("received a route to manage, but agent doesn't support router mode on %s OS", runtime.GOOS)
@@ -197,12 +254,12 @@ func (m *DefaultManager) classifiesRoutes(newRoutes []*route.Route) (map[string]
}
for _, newRoute := range newRoutes {
networkID := route.GetHAUniqueID(newRoute)
if !ownNetworkIDs[networkID] {
haID := route.GetHAUniqueID(newRoute)
if !ownNetworkIDs[haID] {
if !isPrefixSupported(newRoute.Network) {
continue
}
newClientRoutesIDMap[networkID] = append(newClientRoutesIDMap[networkID], newRoute)
newClientRoutesIDMap[haID] = append(newClientRoutesIDMap[haID], newRoute)
}
}
@@ -210,7 +267,7 @@ func (m *DefaultManager) classifiesRoutes(newRoutes []*route.Route) (map[string]
}
func (m *DefaultManager) clientRoutes(initialRoutes []*route.Route) []*route.Route {
_, crMap := m.classifiesRoutes(initialRoutes)
_, crMap := m.classifyRoutes(initialRoutes)
rs := make([]*route.Route, 0)
for _, routes := range crMap {
rs = append(rs, routes...)
@@ -220,10 +277,7 @@ func (m *DefaultManager) clientRoutes(initialRoutes []*route.Route) []*route.Rou
func isPrefixSupported(prefix netip.Prefix) bool {
if !nbnet.CustomRoutingDisabled() {
switch runtime.GOOS {
case "linux", "windows", "darwin":
return true
}
return true
}
// If prefix is too small, lets assume it is a possible default prefix which is not yet supported

View File

@@ -36,6 +36,7 @@ func TestManagerUpdateRoutes(t *testing.T) {
serverRoutesExpected int
clientNetworkWatchersExpected int
clientNetworkWatchersExpectedAllowed int
isV6 bool
}{
{
name: "Should create 2 client networks",
@@ -65,6 +66,35 @@ func TestManagerUpdateRoutes(t *testing.T) {
inputSerial: 1,
clientNetworkWatchersExpected: 2,
},
{
name: "Should create 2 client networks (IPv6)",
inputInitRoutes: []*route.Route{},
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::7890:abcd/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
clientNetworkWatchersExpected: 2,
isV6: true,
},
{
name: "Should Create 2 Server Routes",
inputRoutes: []*route.Route{
@@ -93,6 +123,34 @@ func TestManagerUpdateRoutes(t *testing.T) {
serverRoutesExpected: 2,
clientNetworkWatchersExpected: 0,
},
{
name: "Should Create 2 Server Routes (IPv6)",
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8::7890:abcd/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
serverRoutesExpected: 2,
clientNetworkWatchersExpected: 0,
},
{
name: "Should Create 1 Route For Client And Server",
inputRoutes: []*route.Route{
@@ -121,6 +179,84 @@ func TestManagerUpdateRoutes(t *testing.T) {
serverRoutesExpected: 1,
clientNetworkWatchersExpected: 1,
},
{
name: "Should Create 1 Route For Client And Server (IPv6)",
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::7890:abcd/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
serverRoutesExpected: 1,
clientNetworkWatchersExpected: 1,
isV6: true,
},
{
name: "Should Create 1 Route For Client And Server for each IP version",
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("100.64.30.250/30"),
NetworkType: route.IPv4Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("8.8.9.9/32"),
NetworkType: route.IPv4Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "a",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::7890:abcd/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
serverRoutesExpected: 2,
clientNetworkWatchersExpected: 2,
isV6: true,
},
{
name: "Should Create 1 Route For Client and Skip Server Route On Empty Server Router",
inputRoutes: []*route.Route{
@@ -150,6 +286,36 @@ func TestManagerUpdateRoutes(t *testing.T) {
serverRoutesExpected: 0,
clientNetworkWatchersExpected: 1,
},
{
name: "Should Create 1 Route For Client and Skip Server Route On Empty Server Router (IPv6)",
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::7890:abcd/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
removeSrvRouter: true,
serverRoutesExpected: 0,
clientNetworkWatchersExpected: 1,
isV6: true,
},
{
name: "Should Create 1 HA Route and 1 Standalone",
inputRoutes: []*route.Route{
@@ -187,6 +353,44 @@ func TestManagerUpdateRoutes(t *testing.T) {
inputSerial: 1,
clientNetworkWatchersExpected: 2,
},
{
name: "Should Create 1 HA Route and 1 Standalone (IPv6)",
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeA",
Peer: remotePeerKey2,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "c",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::7890:abcd/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
clientNetworkWatchersExpected: 2,
isV6: true,
},
{
name: "No Small Client Route Should Be Added",
inputRoutes: []*route.Route{
@@ -205,6 +409,25 @@ func TestManagerUpdateRoutes(t *testing.T) {
clientNetworkWatchersExpected: 0,
clientNetworkWatchersExpectedAllowed: 1,
},
{
name: "No Small Client Route Should Be Added (IPv6)",
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("::/0"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
clientNetworkWatchersExpected: 0,
clientNetworkWatchersExpectedAllowed: 1,
isV6: true,
},
{
name: "Remove 1 Client Route",
inputInitRoutes: []*route.Route{
@@ -244,6 +467,46 @@ func TestManagerUpdateRoutes(t *testing.T) {
inputSerial: 1,
clientNetworkWatchersExpected: 1,
},
{
name: "Remove 1 Client Route (IPv6)",
inputInitRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::abcd:7890/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
clientNetworkWatchersExpected: 1,
isV6: true,
},
{
name: "Update Route to HA",
inputInitRoutes: []*route.Route{
@@ -293,6 +556,56 @@ func TestManagerUpdateRoutes(t *testing.T) {
inputSerial: 1,
clientNetworkWatchersExpected: 1,
},
{
name: "Update Route to HA (IPv6)",
inputInitRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::abcd:7890/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeA",
Peer: remotePeerKey2,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
clientNetworkWatchersExpected: 1,
isV6: true,
},
{
name: "Remove Client Routes",
inputInitRoutes: []*route.Route{
@@ -321,6 +634,35 @@ func TestManagerUpdateRoutes(t *testing.T) {
inputSerial: 1,
clientNetworkWatchersExpected: 0,
},
{
name: "Remove Client Routes (IPv6)",
inputInitRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::abcd:7890/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputRoutes: []*route.Route{},
inputSerial: 1,
clientNetworkWatchersExpected: 0,
isV6: true,
},
{
name: "Remove All Routes",
inputInitRoutes: []*route.Route{
@@ -350,6 +692,36 @@ func TestManagerUpdateRoutes(t *testing.T) {
serverRoutesExpected: 0,
clientNetworkWatchersExpected: 0,
},
{
name: "Remove All Routes (IPv6)",
inputInitRoutes: []*route.Route{
{
ID: "a",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "b",
NetID: "routeB",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::abcd:7890/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputRoutes: []*route.Route{},
inputSerial: 1,
serverRoutesExpected: 0,
clientNetworkWatchersExpected: 0,
isV6: true,
},
{
name: "HA server should not register routes from the same HA group",
inputRoutes: []*route.Route{
@@ -398,16 +770,74 @@ func TestManagerUpdateRoutes(t *testing.T) {
serverRoutesExpected: 2,
clientNetworkWatchersExpected: 1,
},
{
name: "HA server should not register routes from the same HA group (IPv6)",
inputRoutes: []*route.Route{
{
ID: "l1",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "l2",
NetID: "routeA",
Peer: localPeerKey,
Network: netip.MustParsePrefix("2001:db8::abcd:7890/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "r1",
NetID: "routeA",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
{
ID: "r2",
NetID: "routeC",
Peer: remotePeerKey1,
Network: netip.MustParsePrefix("2001:db8::abcd:789f/128"),
NetworkType: route.IPv6Network,
Metric: 9999,
Masquerade: false,
Enabled: true,
},
},
inputSerial: 1,
serverRoutesExpected: 2,
clientNetworkWatchersExpected: 1,
isV6: true,
},
}
for n, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
v6Addr := ""
//goland:noinspection GoBoolExpressions
if !iface.SupportsIPv6() && testCase.isV6 {
t.Skip("Platform does not support IPv6, skipping IPv6 test...")
} else if testCase.isV6 {
v6Addr = "2001:db8::4242:4711/128"
}
peerPrivateKey, _ := wgtypes.GeneratePrivateKey()
newNet, err := stdnet.NewNet()
if err != nil {
t.Fatal(err)
}
wgInterface, err := iface.NewWGIFace(fmt.Sprintf("utun43%d", n), "100.65.65.2/24", 33100, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
wgInterface, err := iface.NewWGIFace(fmt.Sprintf("utun43%d", n), "100.65.65.2/24", v6Addr, 33100, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
require.NoError(t, err, "should create testing WGIface interface")
defer wgInterface.Close()
@@ -428,11 +858,11 @@ func TestManagerUpdateRoutes(t *testing.T) {
}
if len(testCase.inputInitRoutes) > 0 {
err = routeManager.UpdateRoutes(testCase.inputSerial, testCase.inputRoutes)
_, _, err = routeManager.UpdateRoutes(testCase.inputSerial, testCase.inputRoutes)
require.NoError(t, err, "should update routes with init routes")
}
err = routeManager.UpdateRoutes(testCase.inputSerial+uint64(len(testCase.inputInitRoutes)), testCase.inputRoutes)
_, _, err = routeManager.UpdateRoutes(testCase.inputSerial+uint64(len(testCase.inputInitRoutes)), testCase.inputRoutes)
require.NoError(t, err, "should update routes")
expectedWatchers := testCase.clientNetworkWatchersExpected

View File

@@ -7,14 +7,17 @@ import (
firewall "github.com/netbirdio/netbird/client/firewall/manager"
"github.com/netbirdio/netbird/client/internal/listener"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/internal/routeselector"
"github.com/netbirdio/netbird/iface"
"github.com/netbirdio/netbird/route"
)
// MockManager is the mock instance of a route manager
type MockManager struct {
UpdateRoutesFunc func(updateSerial uint64, newRoutes []*route.Route) error
StopFunc func()
UpdateRoutesFunc func(updateSerial uint64, newRoutes []*route.Route) (map[route.ID]*route.Route, route.HAMap, error)
TriggerSelectionFunc func(haMap route.HAMap)
GetRouteSelectorFunc func() *routeselector.RouteSelector
StopFunc func()
}
func (m *MockManager) Init() (peer.BeforeAddPeerHookFunc, peer.AfterRemovePeerHookFunc, error) {
@@ -27,11 +30,25 @@ func (m *MockManager) InitialRouteRange() []string {
}
// UpdateRoutes mock implementation of UpdateRoutes from Manager interface
func (m *MockManager) UpdateRoutes(updateSerial uint64, newRoutes []*route.Route) error {
func (m *MockManager) UpdateRoutes(updateSerial uint64, newRoutes []*route.Route) (map[route.ID]*route.Route, route.HAMap, error) {
if m.UpdateRoutesFunc != nil {
return m.UpdateRoutesFunc(updateSerial, newRoutes)
}
return fmt.Errorf("method UpdateRoutes is not implemented")
return nil, nil, fmt.Errorf("method UpdateRoutes is not implemented")
}
func (m *MockManager) TriggerSelection(networks route.HAMap) {
if m.TriggerSelectionFunc != nil {
m.TriggerSelectionFunc(networks)
}
}
// GetRouteSelector mock implementation of GetRouteSelector from Manager interface
func (m *MockManager) GetRouteSelector() *routeselector.RouteSelector {
if m.GetRouteSelectorFunc != nil {
return m.GetRouteSelectorFunc()
}
return nil
}
// Start mock implementation of Start from Manager interface
@@ -47,6 +64,10 @@ func (m *MockManager) EnableServerRouter(firewall firewall.Manager) error {
panic("implement me")
}
func (m *MockManager) ResetV6Routes() {
panic("implement me")
}
// Stop mock implementation of Stop from Manager interface
func (m *MockManager) Stop() {
if m.StopFunc != nil {

View File

@@ -1,6 +1,7 @@
package routemanager
import (
"runtime"
"sort"
"strings"
"sync"
@@ -10,8 +11,8 @@ import (
)
type notifier struct {
initialRouteRangers []string
routeRangers []string
initialRouteRanges []string
routeRanges []string
listener listener.NetworkChangeListener
listenerMux sync.Mutex
@@ -33,10 +34,10 @@ func (n *notifier) setInitialClientRoutes(clientRoutes []*route.Route) {
nets = append(nets, r.Network.String())
}
sort.Strings(nets)
n.initialRouteRangers = nets
n.initialRouteRanges = nets
}
func (n *notifier) onNewRoutes(idMap map[string][]*route.Route) {
func (n *notifier) onNewRoutes(idMap route.HAMap) {
newNets := make([]string, 0)
for _, routes := range idMap {
for _, r := range routes {
@@ -45,11 +46,18 @@ func (n *notifier) onNewRoutes(idMap map[string][]*route.Route) {
}
sort.Strings(newNets)
if !n.hasDiff(n.initialRouteRangers, newNets) {
return
switch runtime.GOOS {
case "android":
if !n.hasDiff(n.initialRouteRanges, newNets) {
return
}
default:
if !n.hasDiff(n.routeRanges, newNets) {
return
}
}
n.routeRangers = newNets
n.routeRanges = newNets
n.notify()
}
@@ -62,7 +70,7 @@ func (n *notifier) notify() {
}
go func(l listener.NetworkChangeListener) {
l.OnNetworkChanged(strings.Join(n.routeRangers, ","))
l.OnNetworkChanged(strings.Join(addIPv6RangeIfNeeded(n.routeRanges), ","))
}(n.listener)
}
@@ -78,6 +86,20 @@ func (n *notifier) hasDiff(a []string, b []string) bool {
return false
}
func (n *notifier) initialRouteRanges() []string {
return n.initialRouteRangers
func (n *notifier) getInitialRouteRanges() []string {
return addIPv6RangeIfNeeded(n.initialRouteRanges)
}
// addIPv6RangeIfNeeded returns the input ranges with the default IPv6 range when there is an IPv4 default route.
func addIPv6RangeIfNeeded(inputRanges []string) []string {
ranges := inputRanges
for _, r := range inputRanges {
// we are intentionally adding the ipv6 default range in case of ipv4 default range
// to ensure that all traffic is managed by the tunnel interface on android
if r == "0.0.0.0/0" {
ranges = append(ranges, "::/0")
break
}
}
return ranges
}

View File

@@ -5,6 +5,7 @@ package routemanager
import (
"errors"
"fmt"
"net"
"net/netip"
"sync"
@@ -17,7 +18,7 @@ import (
type ref struct {
count int
nexthop netip.Addr
intf string
intf *net.Interface
}
type RouteManager struct {
@@ -30,8 +31,8 @@ type RouteManager struct {
mutex sync.Mutex
}
type AddRouteFunc func(prefix netip.Prefix) (nexthop netip.Addr, intf string, err error)
type RemoveRouteFunc func(prefix netip.Prefix, nexthop netip.Addr, intf string) error
type AddRouteFunc func(prefix netip.Prefix) (nexthop netip.Addr, intf *net.Interface, err error)
type RemoveRouteFunc func(prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error
func NewRouteManager(addRoute AddRouteFunc, removeRoute RemoveRouteFunc) *RouteManager {
// TODO: read initial routing table into refCountMap

View File

@@ -3,7 +3,7 @@ package routemanager
import "github.com/netbirdio/netbird/route"
type serverRouter interface {
updateRoutes(map[string]*route.Route) error
updateRoutes(map[route.ID]*route.Route) error
removeFromServerNetwork(*route.Route) error
cleanUp()
}

View File

@@ -19,7 +19,7 @@ import (
type defaultServerRouter struct {
mux sync.Mutex
ctx context.Context
routes map[string]*route.Route
routes map[route.ID]*route.Route
firewall firewall.Manager
wgInterface *iface.WGIface
statusRecorder *peer.Status
@@ -28,15 +28,15 @@ type defaultServerRouter struct {
func newServerRouter(ctx context.Context, wgInterface *iface.WGIface, firewall firewall.Manager, statusRecorder *peer.Status) (serverRouter, error) {
return &defaultServerRouter{
ctx: ctx,
routes: make(map[string]*route.Route),
routes: make(map[route.ID]*route.Route),
firewall: firewall,
wgInterface: wgInterface,
statusRecorder: statusRecorder,
}, nil
}
func (m *defaultServerRouter) updateRoutes(routesMap map[string]*route.Route) error {
serverRoutesToRemove := make([]string, 0)
func (m *defaultServerRouter) updateRoutes(routesMap map[route.ID]*route.Route) error {
serverRoutesToRemove := make([]route.ID, 0)
for routeID := range m.routes {
update, found := routesMap[routeID]
@@ -70,7 +70,7 @@ func (m *defaultServerRouter) updateRoutes(routesMap map[string]*route.Route) er
}
if len(m.routes) > 0 {
err := enableIPForwarding()
err := enableIPForwarding(m.wgInterface.Address6() != nil)
if err != nil {
return err
}
@@ -79,7 +79,7 @@ func (m *defaultServerRouter) updateRoutes(routesMap map[string]*route.Route) er
return nil
}
func (m *defaultServerRouter) removeFromServerNetwork(route *route.Route) error {
func (m *defaultServerRouter) removeFromServerNetwork(rt *route.Route) error {
select {
case <-m.ctx.Done():
log.Infof("Not removing from server network because context is done")
@@ -87,28 +87,32 @@ func (m *defaultServerRouter) removeFromServerNetwork(route *route.Route) error
default:
m.mux.Lock()
defer m.mux.Unlock()
routerPair, err := routeToRouterPair(m.wgInterface.Address().Masked().String(), route)
routingAddress := m.wgInterface.Address().Masked().String()
if rt.NetworkType == route.IPv6Network {
if m.wgInterface.Address6() == nil {
return fmt.Errorf("attempted to add route for IPv6 even though device has no v6 address")
}
routingAddress = m.wgInterface.Address6().Masked().String()
}
routerPair, err := routeToRouterPair(routingAddress, rt)
if err != nil {
return fmt.Errorf("parse prefix: %w", err)
}
err = m.firewall.RemoveRoutingRules(routerPair)
if err != nil {
return fmt.Errorf("remove routing rules: %w", err)
return err
}
delete(m.routes, route.ID)
delete(m.routes, rt.ID)
state := m.statusRecorder.GetLocalPeerState()
delete(state.Routes, route.Network.String())
delete(state.Routes, rt.Network.String())
m.statusRecorder.UpdateLocalPeerState(state)
return nil
}
}
func (m *defaultServerRouter) addToServerNetwork(route *route.Route) error {
func (m *defaultServerRouter) addToServerNetwork(rt *route.Route) error {
select {
case <-m.ctx.Done():
log.Infof("Not adding to server network because context is done")
@@ -116,8 +120,15 @@ func (m *defaultServerRouter) addToServerNetwork(route *route.Route) error {
default:
m.mux.Lock()
defer m.mux.Unlock()
routingAddress := m.wgInterface.Address().Masked().String()
if rt.NetworkType == route.IPv6Network {
if m.wgInterface.Address6() == nil {
return fmt.Errorf("attempted to add route for IPv6 even though device has no v6 address")
}
routingAddress = m.wgInterface.Address6().Masked().String()
}
routerPair, err := routeToRouterPair(m.wgInterface.Address().Masked().String(), route)
routerPair, err := routeToRouterPair(routingAddress, rt)
if err != nil {
return fmt.Errorf("parse prefix: %w", err)
}
@@ -127,13 +138,13 @@ func (m *defaultServerRouter) addToServerNetwork(route *route.Route) error {
return fmt.Errorf("insert routing rules: %w", err)
}
m.routes[route.ID] = route
m.routes[rt.ID] = rt
state := m.statusRecorder.GetLocalPeerState()
if state.Routes == nil {
state.Routes = map[string]struct{}{}
}
state.Routes[route.Network.String()] = struct{}{}
state.Routes[rt.Network.String()] = struct{}{}
m.statusRecorder.UpdateLocalPeerState(state)
return nil
@@ -144,10 +155,17 @@ func (m *defaultServerRouter) cleanUp() {
m.mux.Lock()
defer m.mux.Unlock()
for _, r := range m.routes {
routerPair, err := routeToRouterPair(m.wgInterface.Address().Masked().String(), r)
routingAddress := m.wgInterface.Address().Masked().String()
if r.NetworkType == route.IPv6Network {
if m.wgInterface.Address6() == nil {
log.Errorf("attempted to remove route for IPv6 even though device has no v6 address")
continue
}
routingAddress = m.wgInterface.Address6().Masked().String()
}
routerPair, err := routeToRouterPair(routingAddress, r)
if err != nil {
log.Errorf("Failed to convert route to router pair: %v", err)
continue
log.Errorf("parse prefix: %v", err)
}
err = m.firewall.RemoveRoutingRules(routerPair)
@@ -168,7 +186,7 @@ func routeToRouterPair(source string, route *route.Route) (firewall.RouterPair,
return firewall.RouterPair{}, err
}
return firewall.RouterPair{
ID: route.ID,
ID: string(route.ID),
Source: parsed.String(),
Destination: route.Network.Masked().String(),
Masquerade: route.Masquerade,

View File

@@ -35,7 +35,7 @@ func addRouteForCurrentDefaultGateway(prefix netip.Prefix) error {
addr = netip.IPv6Unspecified()
}
defaultGateway, _, err := getNextHop(addr)
defaultGateway, _, err := GetNextHop(addr)
if err != nil && !errors.Is(err, ErrRouteNotFound) {
return fmt.Errorf("get existing route gateway: %s", err)
}
@@ -60,31 +60,27 @@ func addRouteForCurrentDefaultGateway(prefix netip.Prefix) error {
return nil
}
var exitIntf string
gatewayHop, intf, err := getNextHop(defaultGateway)
gatewayHop, intf, err := GetNextHop(defaultGateway)
if err != nil && !errors.Is(err, ErrRouteNotFound) {
return fmt.Errorf("unable to get the next hop for the default gateway address. error: %s", err)
}
if intf != nil {
exitIntf = intf.Name
}
log.Debugf("Adding a new route for gateway %s with next hop %s", gatewayPrefix, gatewayHop)
return addToRouteTable(gatewayPrefix, gatewayHop, exitIntf)
return addToRouteTable(gatewayPrefix, gatewayHop, intf)
}
func getNextHop(ip netip.Addr) (netip.Addr, *net.Interface, error) {
func GetNextHop(ip netip.Addr) (netip.Addr, *net.Interface, error) {
r, err := netroute.New()
if err != nil {
return netip.Addr{}, nil, fmt.Errorf("new netroute: %w", err)
}
intf, gateway, preferredSrc, err := r.Route(ip.AsSlice())
if err != nil {
log.Warnf("Failed to get route for %s: %v", ip, err)
log.Debugf("Failed to get route for %s: %v", ip, err)
return netip.Addr{}, nil, ErrRouteNotFound
}
log.Debugf("Route for %s: interface %v, nexthop %v, preferred source %v", ip, intf, gateway, preferredSrc)
log.Debugf("Route for %s: interface %v nexthop %v, preferred source %v", ip, intf, gateway, preferredSrc)
if gateway == nil {
if preferredSrc == nil {
return netip.Addr{}, nil, ErrRouteNotFound
@@ -126,6 +122,16 @@ func ipToAddr(ip net.IP, intf *net.Interface) (netip.Addr, error) {
}
func existsInRouteTable(prefix netip.Prefix) (bool, error) {
linkLocalPrefix, err := netip.ParsePrefix("fe80::/10")
if err != nil {
return false, err
}
if prefix.Addr().Is6() && linkLocalPrefix.Contains(prefix.Addr()) {
// The link local prefix is not explicitly part of the routing table, but should be considered as such.
return true, nil
}
routes, err := getRoutesFromTable()
if err != nil {
return false, fmt.Errorf("get routes from table: %w", err)
@@ -153,12 +159,7 @@ func isSubRange(prefix netip.Prefix) (bool, error) {
// addRouteToNonVPNIntf adds a new route to the routing table for the given prefix and returns the next hop and interface.
// If the next hop or interface is pointing to the VPN interface, it will return the initial values.
func addRouteToNonVPNIntf(
prefix netip.Prefix,
vpnIntf *iface.WGIface,
initialNextHop netip.Addr,
initialIntf *net.Interface,
) (netip.Addr, string, error) {
func addRouteToNonVPNIntf(prefix netip.Prefix, vpnIntf *iface.WGIface, initialNextHop netip.Addr, initialIntf *net.Interface) (netip.Addr, *net.Interface, error) {
addr := prefix.Addr()
switch {
case addr.IsLoopback(),
@@ -168,39 +169,34 @@ func addRouteToNonVPNIntf(
addr.IsUnspecified(),
addr.IsMulticast():
return netip.Addr{}, "", ErrRouteNotAllowed
return netip.Addr{}, nil, ErrRouteNotAllowed
}
// Determine the exit interface and next hop for the prefix, so we can add a specific route
nexthop, intf, err := getNextHop(addr)
nexthop, intf, err := GetNextHop(addr)
if err != nil {
return netip.Addr{}, "", fmt.Errorf("get next hop: %w", err)
return netip.Addr{}, nil, fmt.Errorf("get next hop: %w", err)
}
log.Debugf("Found next hop %s for prefix %s with interface %v", nexthop, prefix, intf)
exitNextHop := nexthop
var exitIntf string
if intf != nil {
exitIntf = intf.Name
}
exitIntf := intf
vpnAddr, ok := netip.AddrFromSlice(vpnIntf.Address().IP)
if !ok {
return netip.Addr{}, "", fmt.Errorf("failed to convert vpn address to netip.Addr")
return netip.Addr{}, nil, fmt.Errorf("failed to convert vpn address to netip.Addr")
}
// if next hop is the VPN address or the interface is the VPN interface, we should use the initial values
if exitNextHop == vpnAddr || exitIntf == vpnIntf.Name() {
if exitNextHop == vpnAddr || exitIntf != nil && exitIntf.Name == vpnIntf.Name() {
log.Debugf("Route for prefix %s is pointing to the VPN interface", prefix)
exitNextHop = initialNextHop
if initialIntf != nil {
exitIntf = initialIntf.Name
}
exitIntf = initialIntf
}
log.Debugf("Adding a new route for prefix %s with next hop %s", prefix, exitNextHop)
if err := addToRouteTable(prefix, exitNextHop, exitIntf); err != nil {
return netip.Addr{}, "", fmt.Errorf("add route to table: %w", err)
return netip.Addr{}, nil, fmt.Errorf("add route to table: %w", err)
}
return exitNextHop, exitIntf, nil
@@ -208,7 +204,7 @@ func addRouteToNonVPNIntf(
// genericAddVPNRoute adds a new route to the vpn interface, it splits the default prefix
// in two /1 prefixes to avoid replacing the existing default route
func genericAddVPNRoute(prefix netip.Prefix, intf string) error {
func genericAddVPNRoute(prefix netip.Prefix, intf *net.Interface) error {
if prefix == defaultv4 {
if err := addToRouteTable(splitDefaultv4_1, netip.Addr{}, intf); err != nil {
return err
@@ -250,7 +246,7 @@ func genericAddVPNRoute(prefix netip.Prefix, intf string) error {
}
// addNonExistingRoute adds a new route to the vpn interface if it doesn't exist in the current routing table
func addNonExistingRoute(prefix netip.Prefix, intf string) error {
func addNonExistingRoute(prefix netip.Prefix, intf *net.Interface) error {
ok, err := existsInRouteTable(prefix)
if err != nil {
return fmt.Errorf("exists in route table: %w", err)
@@ -277,7 +273,7 @@ func addNonExistingRoute(prefix netip.Prefix, intf string) error {
// genericRemoveVPNRoute removes the route from the vpn interface. If a default prefix is given,
// it will remove the split /1 prefixes
func genericRemoveVPNRoute(prefix netip.Prefix, intf string) error {
func genericRemoveVPNRoute(prefix netip.Prefix, intf *net.Interface) error {
if prefix == defaultv4 {
var result *multierror.Error
if err := removeFromRouteTable(splitDefaultv4_1, netip.Addr{}, intf); err != nil {
@@ -333,17 +329,17 @@ func getPrefixFromIP(ip net.IP) (*netip.Prefix, error) {
}
func setupRoutingWithRouteManager(routeManager **RouteManager, initAddresses []net.IP, wgIface *iface.WGIface) (peer.BeforeAddPeerHookFunc, peer.AfterRemovePeerHookFunc, error) {
initialNextHopV4, initialIntfV4, err := getNextHop(netip.IPv4Unspecified())
initialNextHopV4, initialIntfV4, err := GetNextHop(netip.IPv4Unspecified())
if err != nil && !errors.Is(err, ErrRouteNotFound) {
log.Errorf("Unable to get initial v4 default next hop: %v", err)
}
initialNextHopV6, initialIntfV6, err := getNextHop(netip.IPv6Unspecified())
initialNextHopV6, initialIntfV6, err := GetNextHop(netip.IPv6Unspecified())
if err != nil && !errors.Is(err, ErrRouteNotFound) {
log.Errorf("Unable to get initial v6 default next hop: %v", err)
}
*routeManager = NewRouteManager(
func(prefix netip.Prefix) (netip.Addr, string, error) {
func(prefix netip.Prefix) (netip.Addr, *net.Interface, error) {
addr := prefix.Addr()
nexthop, intf := initialNextHopV4, initialIntfV4
if addr.Is6() {

View File

@@ -24,10 +24,10 @@ func enableIPForwarding() error {
return nil
}
func addVPNRoute(netip.Prefix, string) error {
func addVPNRoute(netip.Prefix, *net.Interface) error {
return nil
}
func removeVPNRoute(netip.Prefix, string) error {
func removeVPNRoute(netip.Prefix, *net.Interface) error {
return nil
}

View File

@@ -3,39 +3,35 @@
package routemanager
import (
"errors"
"fmt"
"net"
"net/netip"
"strconv"
"syscall"
"time"
"github.com/cenkalti/backoff/v4"
log "github.com/sirupsen/logrus"
"golang.org/x/net/route"
)
// selected BSD Route flags.
const (
RTF_UP = 0x1
RTF_GATEWAY = 0x2
RTF_HOST = 0x4
RTF_REJECT = 0x8
RTF_DYNAMIC = 0x10
RTF_MODIFIED = 0x20
RTF_STATIC = 0x800
RTF_BLACKHOLE = 0x1000
RTF_LOCAL = 0x200000
RTF_BROADCAST = 0x400000
RTF_MULTICAST = 0x800000
)
type Route struct {
Dst netip.Prefix
Gw netip.Addr
Interface *net.Interface
}
func getRoutesFromTable() ([]netip.Prefix, error) {
tab, err := route.FetchRIB(syscall.AF_UNSPEC, route.RIBTypeRoute, 0)
tab, err := retryFetchRIB()
if err != nil {
return nil, err
return nil, fmt.Errorf("fetch RIB: %v", err)
}
msgs, err := route.ParseRIB(route.RIBTypeRoute, tab)
if err != nil {
return nil, err
return nil, fmt.Errorf("parse RIB: %v", err)
}
var prefixList []netip.Prefix
for _, msg := range msgs {
m := msg.(*route.RouteMessage)
@@ -43,58 +39,121 @@ func getRoutesFromTable() ([]netip.Prefix, error) {
if m.Version < 3 || m.Version > 5 {
return nil, fmt.Errorf("unexpected RIB message version: %d", m.Version)
}
if m.Type != 4 /* RTM_GET */ {
if m.Type != syscall.RTM_GET {
return nil, fmt.Errorf("unexpected RIB message type: %d", m.Type)
}
if m.Flags&RTF_UP == 0 ||
m.Flags&(RTF_REJECT|RTF_BLACKHOLE) != 0 {
if m.Flags&syscall.RTF_UP == 0 ||
m.Flags&(syscall.RTF_REJECT|syscall.RTF_BLACKHOLE|syscall.RTF_WASCLONED) != 0 {
continue
}
if len(m.Addrs) < 3 {
log.Warnf("Unexpected RIB message Addrs: %v", m.Addrs)
route, err := MsgToRoute(m)
if err != nil {
log.Warnf("Failed to parse route message: %v", err)
continue
}
addr, ok := toNetIPAddr(m.Addrs[0])
if !ok {
continue
}
cidr := 32
if mask := m.Addrs[2]; mask != nil {
cidr, ok = toCIDR(mask)
if !ok {
log.Debugf("Unexpected RIB message Addrs[2]: %v", mask)
continue
}
}
routePrefix := netip.PrefixFrom(addr, cidr)
if routePrefix.IsValid() {
prefixList = append(prefixList, routePrefix)
if route.Dst.IsValid() {
prefixList = append(prefixList, route.Dst)
}
}
return prefixList, nil
}
func toNetIPAddr(a route.Addr) (netip.Addr, bool) {
func retryFetchRIB() ([]byte, error) {
var out []byte
operation := func() error {
var err error
out, err = route.FetchRIB(syscall.AF_UNSPEC, route.RIBTypeRoute, 0)
if errors.Is(err, syscall.ENOMEM) {
log.Debug("~etrying fetchRIB due to 'cannot allocate memory' error")
return err
} else if err != nil {
return backoff.Permanent(err)
}
return nil
}
expBackOff := backoff.NewExponentialBackOff()
expBackOff.InitialInterval = 50 * time.Millisecond
expBackOff.MaxInterval = 500 * time.Millisecond
expBackOff.MaxElapsedTime = 1 * time.Second
err := backoff.Retry(operation, expBackOff)
if err != nil {
return nil, fmt.Errorf("failed to fetch routing information: %w", err)
}
return out, nil
}
func toNetIP(a route.Addr) netip.Addr {
switch t := a.(type) {
case *route.Inet4Addr:
return netip.AddrFrom4(t.IP), true
return netip.AddrFrom4(t.IP)
case *route.Inet6Addr:
ip := netip.AddrFrom16(t.IP)
if t.ZoneID != 0 {
ip.WithZone(strconv.Itoa(t.ZoneID))
}
return ip
default:
return netip.Addr{}, false
return netip.Addr{}
}
}
func toCIDR(a route.Addr) (int, bool) {
func ones(a route.Addr) (int, error) {
switch t := a.(type) {
case *route.Inet4Addr:
mask := net.IPv4Mask(t.IP[0], t.IP[1], t.IP[2], t.IP[3])
cidr, _ := mask.Size()
return cidr, true
mask, _ := net.IPMask(t.IP[:]).Size()
return mask, nil
case *route.Inet6Addr:
mask, _ := net.IPMask(t.IP[:]).Size()
return mask, nil
default:
return 0, false
return 0, fmt.Errorf("unexpected address type: %T", a)
}
}
func MsgToRoute(msg *route.RouteMessage) (*Route, error) {
dstIP, nexthop, dstMask := msg.Addrs[0], msg.Addrs[1], msg.Addrs[2]
addr := toNetIP(dstIP)
var nexthopAddr netip.Addr
var nexthopIntf *net.Interface
switch t := nexthop.(type) {
case *route.Inet4Addr, *route.Inet6Addr:
nexthopAddr = toNetIP(t)
case *route.LinkAddr:
nexthopIntf = &net.Interface{
Index: t.Index,
Name: t.Name,
}
default:
return nil, fmt.Errorf("unexpected next hop type: %T", t)
}
var prefix netip.Prefix
if dstMask == nil {
if addr.Is4() {
prefix = netip.PrefixFrom(addr, 32)
} else {
prefix = netip.PrefixFrom(addr, 128)
}
} else {
bits, err := ones(dstMask)
if err != nil {
return nil, fmt.Errorf("failed to parse mask: %v", dstMask)
}
prefix = netip.PrefixFrom(addr, bits)
}
return &Route{
Dst: prefix,
Gw: nexthopAddr,
Interface: nexthopIntf,
}, nil
}

View File

@@ -0,0 +1,57 @@
//go:build darwin || dragonfly || freebsd || netbsd || openbsd
package routemanager
import (
"testing"
"github.com/stretchr/testify/assert"
"golang.org/x/net/route"
)
func TestBits(t *testing.T) {
tests := []struct {
name string
addr route.Addr
want int
wantErr bool
}{
{
name: "IPv4 all ones",
addr: &route.Inet4Addr{IP: [4]byte{255, 255, 255, 255}},
want: 32,
},
{
name: "IPv4 normal mask",
addr: &route.Inet4Addr{IP: [4]byte{255, 255, 255, 0}},
want: 24,
},
{
name: "IPv6 all ones",
addr: &route.Inet6Addr{IP: [16]byte{255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}},
want: 128,
},
{
name: "IPv6 normal mask",
addr: &route.Inet6Addr{IP: [16]byte{255, 255, 255, 255, 255, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0}},
want: 64,
},
{
name: "Unsupported type",
addr: &route.LinkAddr{},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ones(tt.addr)
if tt.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, tt.want, got)
}
})
}
}

View File

@@ -27,15 +27,15 @@ func cleanupRouting() error {
return cleanupRoutingWithRouteManager(routeManager)
}
func addToRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf string) error {
func addToRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error {
return routeCmd("add", prefix, nexthop, intf)
}
func removeFromRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf string) error {
func removeFromRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error {
return routeCmd("delete", prefix, nexthop, intf)
}
func routeCmd(action string, prefix netip.Prefix, nexthop netip.Addr, intf string) error {
func routeCmd(action string, prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error {
inet := "-inet"
network := prefix.String()
if prefix.IsSingleIP() {
@@ -43,18 +43,13 @@ func routeCmd(action string, prefix netip.Prefix, nexthop netip.Addr, intf strin
}
if prefix.Addr().Is6() {
inet = "-inet6"
// Special case for IPv6 split default route, pointing to the wg interface fails
// TODO: Remove once we have IPv6 support on the interface
if prefix.Bits() == 1 {
intf = "lo0"
}
}
args := []string{"-n", action, inet, network}
if nexthop.IsValid() {
args = append(args, nexthop.Unmap().String())
} else if intf != "" {
args = append(args, "-interface", intf)
} else if intf != nil {
args = append(args, "-interface", intf.Name)
}
if err := retryRouteCmd(args); err != nil {

View File

@@ -33,7 +33,7 @@ func init() {
func TestConcurrentRoutes(t *testing.T) {
baseIP := netip.MustParseAddr("192.0.2.0")
intf := "lo0"
intf := &net.Interface{Name: "lo0"}
var wg sync.WaitGroup
for i := 0; i < 1024; i++ {

View File

@@ -19,15 +19,15 @@ func cleanupRouting() error {
return nil
}
func enableIPForwarding() error {
func enableIPForwarding(includeV6 bool) error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func addVPNRoute(netip.Prefix, string) error {
func addVPNRoute(netip.Prefix, *net.Interface) error {
return nil
}
func removeVPNRoute(netip.Prefix, string) error {
func removeVPNRoute(netip.Prefix, *net.Interface) error {
return nil
}

View File

@@ -46,9 +46,6 @@ var routeManager = &RouteManager{}
// originalSysctl stores the original sysctl values before they are modified
var originalSysctl map[string]int
// determines whether to use the legacy routing setup
var isLegacy = os.Getenv("NB_USE_LEGACY_ROUTING") == "true" || nbnet.CustomRoutingDisabled()
// sysctlFailed is used as an indicator to emit a warning when default routes are configured
var sysctlFailed bool
@@ -62,6 +59,20 @@ type ruleParams struct {
description string
}
// isLegacy determines whether to use the legacy routing setup
func isLegacy() bool {
return os.Getenv("NB_USE_LEGACY_ROUTING") == "true" || nbnet.CustomRoutingDisabled()
}
// setIsLegacy sets the legacy routing setup
func setIsLegacy(b bool) {
if b {
os.Setenv("NB_USE_LEGACY_ROUTING", "true")
} else {
os.Unsetenv("NB_USE_LEGACY_ROUTING")
}
}
func getSetupRules() []ruleParams {
return []ruleParams{
{100, -1, syscall.RT_TABLE_MAIN, netlink.FAMILY_V4, false, 0, "rule with suppress prefixlen v4"},
@@ -82,7 +93,7 @@ func getSetupRules() []ruleParams {
// This table is where a default route or other specific routes received from the management server are configured,
// enabling VPN connectivity.
func setupRouting(initAddresses []net.IP, wgIface *iface.WGIface) (_ peer.BeforeAddPeerHookFunc, _ peer.AfterRemovePeerHookFunc, err error) {
if isLegacy {
if isLegacy() {
log.Infof("Using legacy routing setup")
return setupRoutingWithRouteManager(&routeManager, initAddresses, wgIface)
}
@@ -111,7 +122,7 @@ func setupRouting(initAddresses []net.IP, wgIface *iface.WGIface) (_ peer.Before
if err := addRule(rule); err != nil {
if errors.Is(err, syscall.EOPNOTSUPP) {
log.Warnf("Rule operations are not supported, falling back to the legacy routing setup")
isLegacy = true
setIsLegacy(true)
return setupRoutingWithRouteManager(&routeManager, initAddresses, wgIface)
}
return nil, nil, fmt.Errorf("%s: %w", rule.description, err)
@@ -125,7 +136,7 @@ func setupRouting(initAddresses []net.IP, wgIface *iface.WGIface) (_ peer.Before
// It systematically removes the three rules and any associated routing table entries to ensure a clean state.
// The function uses error aggregation to report any errors encountered during the cleanup process.
func cleanupRouting() error {
if isLegacy {
if isLegacy() {
return cleanupRoutingWithRouteManager(routeManager)
}
@@ -154,16 +165,16 @@ func cleanupRouting() error {
return result.ErrorOrNil()
}
func addToRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf string) error {
func addToRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error {
return addRoute(prefix, nexthop, intf, syscall.RT_TABLE_MAIN)
}
func removeFromRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf string) error {
func removeFromRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error {
return removeRoute(prefix, nexthop, intf, syscall.RT_TABLE_MAIN)
}
func addVPNRoute(prefix netip.Prefix, intf string) error {
if isLegacy {
func addVPNRoute(prefix netip.Prefix, intf *net.Interface) error {
if isLegacy() {
return genericAddVPNRoute(prefix, intf)
}
@@ -173,29 +184,17 @@ func addVPNRoute(prefix netip.Prefix, intf string) error {
// No need to check if routes exist as main table takes precedence over the VPN table via Rule 1
// TODO remove this once we have ipv6 support
if prefix == defaultv4 {
if err := addUnreachableRoute(defaultv6, NetbirdVPNTableID); err != nil {
return fmt.Errorf("add blackhole: %w", err)
}
}
if err := addRoute(prefix, netip.Addr{}, intf, NetbirdVPNTableID); err != nil {
return fmt.Errorf("add route: %w", err)
}
return nil
}
func removeVPNRoute(prefix netip.Prefix, intf string) error {
if isLegacy {
func removeVPNRoute(prefix netip.Prefix, intf *net.Interface) error {
if isLegacy() {
return genericRemoveVPNRoute(prefix, intf)
}
// TODO remove this once we have ipv6 support
if prefix == defaultv4 {
if err := removeUnreachableRoute(defaultv6, NetbirdVPNTableID); err != nil {
return fmt.Errorf("remove unreachable route: %w", err)
}
}
if err := removeRoute(prefix, netip.Addr{}, intf, NetbirdVPNTableID); err != nil {
return fmt.Errorf("remove route: %w", err)
}
@@ -244,7 +243,7 @@ func getRoutes(tableID, family int) ([]netip.Prefix, error) {
}
// addRoute adds a route to a specific routing table identified by tableID.
func addRoute(prefix netip.Prefix, addr netip.Addr, intf string, tableID int) error {
func addRoute(prefix netip.Prefix, addr netip.Addr, intf *net.Interface, tableID int) error {
route := &netlink.Route{
Scope: netlink.SCOPE_UNIVERSE,
Table: tableID,
@@ -271,6 +270,9 @@ func addRoute(prefix netip.Prefix, addr netip.Addr, intf string, tableID int) er
// addUnreachableRoute adds an unreachable route for the specified IP family and routing table.
// ipFamily should be netlink.FAMILY_V4 for IPv4 or netlink.FAMILY_V6 for IPv6.
// tableID specifies the routing table to which the unreachable route will be added.
// TODO should this be kept in for future use? If so, the linter needs to be told that this unreachable function should
//
// be kept
func addUnreachableRoute(prefix netip.Prefix, tableID int) error {
_, ipNet, err := net.ParseCIDR(prefix.String())
if err != nil {
@@ -291,6 +293,9 @@ func addUnreachableRoute(prefix netip.Prefix, tableID int) error {
return nil
}
// TODO should this be kept in for future use? If so, the linter needs to be told that this unreachable function should
//
// be kept
func removeUnreachableRoute(prefix netip.Prefix, tableID int) error {
_, ipNet, err := net.ParseCIDR(prefix.String())
if err != nil {
@@ -304,7 +309,10 @@ func removeUnreachableRoute(prefix netip.Prefix, tableID int) error {
Dst: ipNet,
}
if err := netlink.RouteDel(route); err != nil && !errors.Is(err, syscall.ESRCH) && !errors.Is(err, syscall.EAFNOSUPPORT) {
if err := netlink.RouteDel(route); err != nil &&
!errors.Is(err, syscall.ESRCH) &&
!errors.Is(err, syscall.ENOENT) &&
!errors.Is(err, syscall.EAFNOSUPPORT) {
return fmt.Errorf("netlink remove unreachable route: %w", err)
}
@@ -313,7 +321,7 @@ func removeUnreachableRoute(prefix netip.Prefix, tableID int) error {
}
// removeRoute removes a route from a specific routing table identified by tableID.
func removeRoute(prefix netip.Prefix, addr netip.Addr, intf string, tableID int) error {
func removeRoute(prefix netip.Prefix, addr netip.Addr, intf *net.Interface, tableID int) error {
_, ipNet, err := net.ParseCIDR(prefix.String())
if err != nil {
return fmt.Errorf("parse prefix %s: %w", prefix, err)
@@ -362,8 +370,14 @@ func flushRoutes(tableID, family int) error {
return result.ErrorOrNil()
}
func enableIPForwarding() error {
func enableIPForwarding(includeV6 bool) error {
_, err := setSysctl(ipv4ForwardingPath, 1, false)
if err != nil {
return err
}
if includeV6 {
_, err = setSysctl(ipv4ForwardingPath, 1, false)
}
return err
}
@@ -467,20 +481,22 @@ func removeRule(params ruleParams) error {
}
// addNextHop adds the gateway and device to the route.
func addNextHop(addr netip.Addr, intf string, route *netlink.Route) error {
if addr.IsValid() {
route.Gw = addr.AsSlice()
if intf == "" {
intf = addr.Zone()
}
func addNextHop(addr netip.Addr, intf *net.Interface, route *netlink.Route) error {
if intf != nil {
route.LinkIndex = intf.Index
}
if intf != "" {
link, err := netlink.LinkByName(intf)
if err != nil {
return fmt.Errorf("set interface %s: %w", intf, err)
if addr.IsValid() {
route.Gw = addr.AsSlice()
// if zone is set, it means the gateway is a link-local address, so we set the link index
if addr.Zone() != "" && intf == nil {
link, err := netlink.LinkByName(addr.Zone())
if err != nil {
return fmt.Errorf("get link by name for zone %s: %w", addr.Zone(), err)
}
route.LinkIndex = link.Attrs().Index
}
route.LinkIndex = link.Attrs().Index
}
return nil

View File

@@ -3,21 +3,22 @@
package routemanager
import (
"net"
"net/netip"
"runtime"
log "github.com/sirupsen/logrus"
)
func enableIPForwarding() error {
func enableIPForwarding(includeV6 bool) error {
log.Infof("Enable IP forwarding is not implemented on %s", runtime.GOOS)
return nil
}
func addVPNRoute(prefix netip.Prefix, intf string) error {
func addVPNRoute(prefix netip.Prefix, intf *net.Interface) error {
return genericAddVPNRoute(prefix, intf)
}
func removeVPNRoute(prefix netip.Prefix, intf string) error {
func removeVPNRoute(prefix netip.Prefix, intf *net.Interface) error {
return genericRemoveVPNRoute(prefix, intf)
}

View File

@@ -6,6 +6,8 @@ import (
"bytes"
"context"
"fmt"
"github.com/google/gopacket/routing"
"github.com/netbirdio/netbird/client/firewall"
"net"
"net/netip"
"os"
@@ -46,16 +48,39 @@ func TestAddRemoveRoutes(t *testing.T) {
shouldRouteToWireguard: false,
shouldBeRemoved: false,
},
{
name: "Should Add And Remove Route 2001:db8:1234:5678::/64",
prefix: netip.MustParsePrefix("2001:db8:1234:5678::/64"),
shouldRouteToWireguard: true,
shouldBeRemoved: true,
},
{
name: "Should Not Add Or Remove Route ::1/128",
prefix: netip.MustParsePrefix("::1/128"),
shouldRouteToWireguard: false,
shouldBeRemoved: false,
},
}
for n, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
t.Setenv("NB_DISABLE_ROUTE_CACHE", "true")
v6Addr := ""
hasV6DefaultRoute, err := EnvironmentHasIPv6DefaultRoute()
//goland:noinspection GoBoolExpressions
if (!iface.SupportsIPv6() || !firewall.SupportsIPv6() || !hasV6DefaultRoute || err != nil) && testCase.prefix.Addr().Is6() {
t.Skip("Platform does not support IPv6, skipping IPv6 test...")
} else if testCase.prefix.Addr().Is6() {
v6Addr = "2001:db8::4242:4711/128"
}
peerPrivateKey, _ := wgtypes.GeneratePrivateKey()
newNet, err := stdnet.NewNet()
if err != nil {
t.Fatal(err)
}
wgInterface, err := iface.NewWGIFace(fmt.Sprintf("utun53%d", n), "100.65.75.2/24", 33100, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
wgInterface, err := iface.NewWGIFace(fmt.Sprintf("utun53%d", n), "100.65.75.2/24", v6Addr, 33100, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
require.NoError(t, err, "should create testing WGIface interface")
defer wgInterface.Close()
@@ -67,7 +92,11 @@ func TestAddRemoveRoutes(t *testing.T) {
assert.NoError(t, cleanupRouting())
})
err = genericAddVPNRoute(testCase.prefix, wgInterface.Name())
index, err := net.InterfaceByName(wgInterface.Name())
require.NoError(t, err, "InterfaceByName should not return err")
intf := &net.Interface{Index: index.Index, Name: wgInterface.Name()}
err = addVPNRoute(testCase.prefix, intf)
require.NoError(t, err, "genericAddVPNRoute should not return err")
if testCase.shouldRouteToWireguard {
@@ -78,13 +107,17 @@ func TestAddRemoveRoutes(t *testing.T) {
exists, err := existsInRouteTable(testCase.prefix)
require.NoError(t, err, "existsInRouteTable should not return err")
if exists && testCase.shouldRouteToWireguard {
err = genericRemoveVPNRoute(testCase.prefix, wgInterface.Name())
err = removeVPNRoute(testCase.prefix, intf)
require.NoError(t, err, "genericRemoveVPNRoute should not return err")
prefixGateway, _, err := getNextHop(testCase.prefix.Addr())
require.NoError(t, err, "getNextHop should not return err")
prefixGateway, _, err := GetNextHop(testCase.prefix.Addr())
require.NoError(t, err, "GetNextHop should not return err")
internetGateway, _, err := getNextHop(netip.MustParseAddr("0.0.0.0"))
internetGateway, _, err := GetNextHop(netip.MustParseAddr("0.0.0.0"))
require.NoError(t, err)
if testCase.prefix.Addr().Is6() {
internetGateway, _, err = GetNextHop(netip.MustParseAddr("::/0"))
}
require.NoError(t, err)
if testCase.shouldBeRemoved {
@@ -98,7 +131,7 @@ func TestAddRemoveRoutes(t *testing.T) {
}
func TestGetNextHop(t *testing.T) {
gateway, _, err := getNextHop(netip.MustParseAddr("0.0.0.0"))
gateway, _, err := GetNextHop(netip.MustParseAddr("0.0.0.0"))
if err != nil {
t.Fatal("shouldn't return error when fetching the gateway: ", err)
}
@@ -124,7 +157,7 @@ func TestGetNextHop(t *testing.T) {
}
}
localIP, _, err := getNextHop(testingPrefix.Addr())
localIP, _, err := GetNextHop(testingPrefix.Addr())
if err != nil {
t.Fatal("shouldn't return error: ", err)
}
@@ -140,11 +173,23 @@ func TestGetNextHop(t *testing.T) {
}
func TestAddExistAndRemoveRoute(t *testing.T) {
defaultGateway, _, err := getNextHop(netip.MustParseAddr("0.0.0.0"))
defaultGateway, _, err := GetNextHop(netip.MustParseAddr("0.0.0.0"))
t.Log("defaultGateway: ", defaultGateway)
if err != nil {
t.Fatal("shouldn't return error when fetching the gateway: ", err)
}
var defaultGateway6 *netip.Addr
hasV6DefaultRoute, err := EnvironmentHasIPv6DefaultRoute()
//goland:noinspection GoBoolExpressions
if iface.SupportsIPv6() && firewall.SupportsIPv6() && hasV6DefaultRoute && err == nil {
gw6, _, err := GetNextHop(netip.MustParseAddr("::"))
gw6 = gw6.WithZone("")
defaultGateway6 = &gw6
t.Log("defaultGateway6: ", defaultGateway6)
if err != nil {
t.Fatal("shouldn't return error when fetching the IPv6 gateway: ", err)
}
}
testCases := []struct {
name string
prefix netip.Prefix
@@ -179,35 +224,87 @@ func TestAddExistAndRemoveRoute(t *testing.T) {
preExistingPrefix: netip.MustParsePrefix("100.100.0.0/16"),
shouldAddRoute: false,
},
{
name: "Should Add And Remove random Route (IPv6)",
prefix: netip.MustParsePrefix("2001:db8::abcd/128"),
shouldAddRoute: true,
},
{
name: "Should Add Route if bigger network exists (IPv6)",
prefix: netip.MustParsePrefix("2001:db8:b14d:abcd:1234::/96"),
preExistingPrefix: netip.MustParsePrefix("2001:db8:b14d:abcd::/64"),
shouldAddRoute: true,
},
{
name: "Should Add Route if smaller network exists (IPv6)",
prefix: netip.MustParsePrefix("2001:db8:b14d::/48"),
preExistingPrefix: netip.MustParsePrefix("2001:db8:b14d:abcd::/64"),
shouldAddRoute: true,
},
{
name: "Should Not Add Route if same network exists (IPv6)",
prefix: netip.MustParsePrefix("2001:db8:b14d:abcd::/64"),
preExistingPrefix: netip.MustParsePrefix("2001:db8:b14d:abcd::/64"),
shouldAddRoute: false,
},
}
if defaultGateway6 != nil {
testCases = append(testCases, []struct {
name string
prefix netip.Prefix
preExistingPrefix netip.Prefix
shouldAddRoute bool
}{
{
name: "Should Not Add Route if overlaps with default gateway (IPv6)",
prefix: netip.MustParsePrefix(defaultGateway6.String() + "/127"),
shouldAddRoute: false,
},
}...)
}
for n, testCase := range testCases {
var buf bytes.Buffer
log.SetOutput(&buf)
defer func() {
log.SetOutput(os.Stderr)
}()
t.Run(testCase.name, func(t *testing.T) {
t.Setenv("NB_USE_LEGACY_ROUTING", "true")
t.Setenv("NB_DISABLE_ROUTE_CACHE", "true")
v6Addr := ""
if testCase.prefix.Addr().Is6() && defaultGateway6 == nil {
t.Skip("Platform does not support IPv6, skipping IPv6 test...")
} else if testCase.prefix.Addr().Is6() {
v6Addr = "2001:db8::4242:4711/128"
}
peerPrivateKey, _ := wgtypes.GeneratePrivateKey()
newNet, err := stdnet.NewNet()
if err != nil {
t.Fatal(err)
}
wgInterface, err := iface.NewWGIFace(fmt.Sprintf("utun53%d", n), "100.65.75.2/24", 33100, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
wgInterface, err := iface.NewWGIFace(fmt.Sprintf("utun53%d", n), "100.65.75.2/24", v6Addr, 33100, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
require.NoError(t, err, "should create testing WGIface interface")
defer wgInterface.Close()
err = wgInterface.Create()
require.NoError(t, err, "should create testing wireguard interface")
index, err := net.InterfaceByName(wgInterface.Name())
require.NoError(t, err, "InterfaceByName should not return err")
intf := &net.Interface{Index: index.Index, Name: wgInterface.Name()}
// Prepare the environment
if testCase.preExistingPrefix.IsValid() {
err := genericAddVPNRoute(testCase.preExistingPrefix, wgInterface.Name())
err := addVPNRoute(testCase.preExistingPrefix, intf)
require.NoError(t, err, "should not return err when adding pre-existing route")
}
// Add the route
err = genericAddVPNRoute(testCase.prefix, wgInterface.Name())
err = addVPNRoute(testCase.prefix, intf)
require.NoError(t, err, "should not return err when adding route")
if testCase.shouldAddRoute {
@@ -217,7 +314,7 @@ func TestAddExistAndRemoveRoute(t *testing.T) {
require.True(t, ok, "route should exist")
// remove route again if added
err = genericRemoveVPNRoute(testCase.prefix, wgInterface.Name())
err = removeVPNRoute(testCase.prefix, intf)
require.NoError(t, err, "should not return err")
}
@@ -235,6 +332,11 @@ func TestAddExistAndRemoveRoute(t *testing.T) {
}
func TestIsSubRange(t *testing.T) {
// Note: This test may fail for IPv6 in some environments, where there actually exists another route that the
// determined prefix is a sub-range of.
hasV6DefaultRoute, err := EnvironmentHasIPv6DefaultRoute()
shouldIncludeV6Routes := iface.SupportsIPv6() && firewall.SupportsIPv6() && hasV6DefaultRoute && err == nil
addresses, err := net.InterfaceAddrs()
if err != nil {
t.Fatal("shouldn't return error when fetching interface addresses: ", err)
@@ -244,7 +346,7 @@ func TestIsSubRange(t *testing.T) {
var nonSubRangeAddressPrefixes []netip.Prefix
for _, address := range addresses {
p := netip.MustParsePrefix(address.String())
if !p.Addr().IsLoopback() && p.Addr().Is4() && p.Bits() < 32 {
if !p.Addr().IsLoopback() && (p.Addr().Is4() && p.Bits() < 32) || (p.Addr().Is6() && shouldIncludeV6Routes && p.Bits() < 128) {
p2 := netip.PrefixFrom(p.Masked().Addr(), p.Bits()+1)
subRangeAddressPrefixes = append(subRangeAddressPrefixes, p2)
nonSubRangeAddressPrefixes = append(nonSubRangeAddressPrefixes, p.Masked())
@@ -272,16 +374,37 @@ func TestIsSubRange(t *testing.T) {
}
}
func EnvironmentHasIPv6DefaultRoute() (bool, error) {
//goland:noinspection GoBoolExpressions
if runtime.GOOS != "linux" {
// TODO when implementing IPv6 for other operating systems, this should be replaced with code that determines
// whether a default route for IPv6 exists (routing.Router panics on non-linux).
return false, nil
}
router, err := routing.New()
if err != nil {
return false, err
}
routeIface, _, _, err := router.Route(netip.MustParsePrefix("::/0").Addr().AsSlice())
if err != nil {
return false, err
}
return routeIface != nil, nil
}
func TestExistsInRouteTable(t *testing.T) {
addresses, err := net.InterfaceAddrs()
if err != nil {
t.Fatal("shouldn't return error when fetching interface addresses: ", err)
}
hasV6DefaultRoute, err := EnvironmentHasIPv6DefaultRoute()
shouldIncludeV6Routes := iface.SupportsIPv6() && firewall.SupportsIPv6() && hasV6DefaultRoute && err == nil
var addressPrefixes []netip.Prefix
for _, address := range addresses {
p := netip.MustParsePrefix(address.String())
if p.Addr().Is6() {
if p.Addr().Is6() && !shouldIncludeV6Routes {
continue
}
// Windows sometimes has hidden interface link local addrs that don't turn up on any interface
@@ -307,7 +430,7 @@ func TestExistsInRouteTable(t *testing.T) {
}
}
func createWGInterface(t *testing.T, interfaceName, ipAddressCIDR string, listenPort int) *iface.WGIface {
func createWGInterface(t *testing.T, interfaceName, ipAddressCIDR string, ipAddress6CIDR string, listenPort int) *iface.WGIface {
t.Helper()
peerPrivateKey, err := wgtypes.GeneratePrivateKey()
@@ -316,7 +439,7 @@ func createWGInterface(t *testing.T, interfaceName, ipAddressCIDR string, listen
newNet, err := stdnet.NewNet()
require.NoError(t, err)
wgInterface, err := iface.NewWGIFace(interfaceName, ipAddressCIDR, listenPort, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
wgInterface, err := iface.NewWGIFace(interfaceName, ipAddressCIDR, ipAddress6CIDR, listenPort, peerPrivateKey.String(), iface.DefaultMTU, newNet, nil)
require.NoError(t, err, "should create testing WireGuard interface")
err = wgInterface.Create()
@@ -334,54 +457,67 @@ func setupTestEnv(t *testing.T) {
setupDummyInterfacesAndRoutes(t)
wgIface := createWGInterface(t, expectedVPNint, "100.64.0.1/24", 51820)
v6Addr := ""
hasV6DefaultRoute, err := EnvironmentHasIPv6DefaultRoute()
//goland:noinspection GoBoolExpressions
if !iface.SupportsIPv6() || !firewall.SupportsIPv6() || !hasV6DefaultRoute || err != nil {
t.Skip("Platform does not support IPv6, skipping IPv6 test...")
} else {
v6Addr = "2001:db8::4242:4711/128"
}
wgIface := createWGInterface(t, expectedVPNint, "100.64.0.1/24", v6Addr, 51820)
t.Cleanup(func() {
assert.NoError(t, wgIface.Close())
})
_, _, err := setupRouting(nil, wgIface)
_, _, err = setupRouting(nil, wgIface)
require.NoError(t, err, "setupRouting should not return err")
t.Cleanup(func() {
assert.NoError(t, cleanupRouting())
})
index, err := net.InterfaceByName(wgIface.Name())
require.NoError(t, err, "InterfaceByName should not return err")
intf := &net.Interface{Index: index.Index, Name: wgIface.Name()}
// default route exists in main table and vpn table
err = addVPNRoute(netip.MustParsePrefix("0.0.0.0/0"), wgIface.Name())
err = addVPNRoute(netip.MustParsePrefix("0.0.0.0/0"), intf)
require.NoError(t, err, "addVPNRoute should not return err")
t.Cleanup(func() {
err = removeVPNRoute(netip.MustParsePrefix("0.0.0.0/0"), wgIface.Name())
err = removeVPNRoute(netip.MustParsePrefix("0.0.0.0/0"), intf)
assert.NoError(t, err, "removeVPNRoute should not return err")
})
// 10.0.0.0/8 route exists in main table and vpn table
err = addVPNRoute(netip.MustParsePrefix("10.0.0.0/8"), wgIface.Name())
err = addVPNRoute(netip.MustParsePrefix("10.0.0.0/8"), intf)
require.NoError(t, err, "addVPNRoute should not return err")
t.Cleanup(func() {
err = removeVPNRoute(netip.MustParsePrefix("10.0.0.0/8"), wgIface.Name())
err = removeVPNRoute(netip.MustParsePrefix("10.0.0.0/8"), intf)
assert.NoError(t, err, "removeVPNRoute should not return err")
})
// 10.10.0.0/24 more specific route exists in vpn table
err = addVPNRoute(netip.MustParsePrefix("10.10.0.0/24"), wgIface.Name())
err = addVPNRoute(netip.MustParsePrefix("10.10.0.0/24"), intf)
require.NoError(t, err, "addVPNRoute should not return err")
t.Cleanup(func() {
err = removeVPNRoute(netip.MustParsePrefix("10.10.0.0/24"), wgIface.Name())
err = removeVPNRoute(netip.MustParsePrefix("10.10.0.0/24"), intf)
assert.NoError(t, err, "removeVPNRoute should not return err")
})
// 127.0.10.0/24 more specific route exists in vpn table
err = addVPNRoute(netip.MustParsePrefix("127.0.10.0/24"), wgIface.Name())
err = addVPNRoute(netip.MustParsePrefix("127.0.10.0/24"), intf)
require.NoError(t, err, "addVPNRoute should not return err")
t.Cleanup(func() {
err = removeVPNRoute(netip.MustParsePrefix("127.0.10.0/24"), wgIface.Name())
err = removeVPNRoute(netip.MustParsePrefix("127.0.10.0/24"), intf)
assert.NoError(t, err, "removeVPNRoute should not return err")
})
// unique route in vpn table
err = addVPNRoute(netip.MustParsePrefix("172.16.0.0/12"), wgIface.Name())
err = addVPNRoute(netip.MustParsePrefix("172.16.0.0/12"), intf)
require.NoError(t, err, "addVPNRoute should not return err")
t.Cleanup(func() {
err = removeVPNRoute(netip.MustParsePrefix("172.16.0.0/12"), wgIface.Name())
err = removeVPNRoute(netip.MustParsePrefix("172.16.0.0/12"), intf)
assert.NoError(t, err, "removeVPNRoute should not return err")
})
}
@@ -392,11 +528,17 @@ func assertWGOutInterface(t *testing.T, prefix netip.Prefix, wgIface *iface.WGIf
return
}
prefixGateway, _, err := getNextHop(prefix.Addr())
require.NoError(t, err, "getNextHop should not return err")
prefixGateway, _, err := GetNextHop(prefix.Addr())
require.NoError(t, err, "GetNextHop should not return err")
nexthop := wgIface.Address().IP.String()
if prefix.Addr().Is6() {
nexthop = wgIface.Address6().IP.String()
}
if invert {
assert.NotEqual(t, wgIface.Address().IP.String(), prefixGateway.String(), "route should not point to wireguard interface IP")
assert.NotEqual(t, nexthop, prefixGateway.String(), "route should not point to wireguard interface IP")
} else {
assert.Equal(t, wgIface.Address().IP.String(), prefixGateway.String(), "route should point to wireguard interface IP")
assert.Equal(t, nexthop, prefixGateway.String(), "route should point to wireguard interface IP")
}
}

View File

@@ -6,21 +6,57 @@ import (
"fmt"
"net"
"net/netip"
"os"
"os/exec"
"strconv"
"strings"
"sync"
"time"
log "github.com/sirupsen/logrus"
"github.com/yusufpapurcu/wmi"
"github.com/netbirdio/netbird/client/firewall/uspfilter"
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/iface"
)
type Win32_IP4RouteTable struct {
Destination string
Mask string
type MSFT_NetRoute struct {
DestinationPrefix string
NextHop string
InterfaceIndex int32
InterfaceAlias string
AddressFamily uint16
}
type Route struct {
Destination netip.Prefix
Nexthop netip.Addr
Interface *net.Interface
}
type MSFT_NetNeighbor struct {
IPAddress string
LinkLayerAddress string
State uint8
AddressFamily uint16
InterfaceIndex uint32
InterfaceAlias string
}
type Neighbor struct {
IPAddress netip.Addr
LinkLayerAddress string
State uint8
AddressFamily uint16
InterfaceIndex uint32
InterfaceAlias string
}
var prefixList []netip.Prefix
var lastUpdate time.Time
var mux = sync.Mutex{}
var routeManager *RouteManager
func setupRouting(initAddresses []net.IP, wgIface *iface.WGIface) (peer.BeforeAddPeerHookFunc, peer.AfterRemovePeerHookFunc, error) {
@@ -32,82 +68,115 @@ func cleanupRouting() error {
}
func getRoutesFromTable() ([]netip.Prefix, error) {
var routes []Win32_IP4RouteTable
query := "SELECT Destination, Mask FROM Win32_IP4RouteTable"
mux.Lock()
defer mux.Unlock()
err := wmi.Query(query, &routes)
// If many routes are added at the same time this might block for a long time (seconds to minutes), so we cache the result
if !isCacheDisabled() && time.Since(lastUpdate) < 2*time.Second {
return prefixList, nil
}
routes, err := GetRoutes()
if err != nil {
return nil, fmt.Errorf("get routes: %w", err)
}
var prefixList []netip.Prefix
prefixList = nil
for _, route := range routes {
addr, err := netip.ParseAddr(route.Destination)
if err != nil {
log.Warnf("Unable to parse route destination %s: %v", route.Destination, err)
continue
}
maskSlice := net.ParseIP(route.Mask).To4()
if maskSlice == nil {
log.Warnf("Unable to parse route mask %s", route.Mask)
continue
}
mask := net.IPv4Mask(maskSlice[0], maskSlice[1], maskSlice[2], maskSlice[3])
cidr, _ := mask.Size()
routePrefix := netip.PrefixFrom(addr, cidr)
if routePrefix.IsValid() && routePrefix.Addr().Is4() {
prefixList = append(prefixList, routePrefix)
}
prefixList = append(prefixList, route.Destination)
}
lastUpdate = time.Now()
return prefixList, nil
}
func addRoutePowershell(prefix netip.Prefix, nexthop netip.Addr, intf, intfIdx string) error {
destinationPrefix := prefix.String()
psCmd := "New-NetRoute"
func GetRoutes() ([]Route, error) {
var entries []MSFT_NetRoute
addressFamily := "IPv4"
if prefix.Addr().Is6() {
addressFamily = "IPv6"
query := `SELECT DestinationPrefix, NextHop, InterfaceIndex, InterfaceAlias, AddressFamily FROM MSFT_NetRoute`
if err := wmi.QueryNamespace(query, &entries, `ROOT\StandardCimv2`); err != nil {
return nil, fmt.Errorf("get routes: %w", err)
}
script := fmt.Sprintf(
`%s -AddressFamily "%s" -DestinationPrefix "%s" -Confirm:$False -ErrorAction Stop -PolicyStore ActiveStore`,
psCmd, addressFamily, destinationPrefix,
)
var routes []Route
for _, entry := range entries {
dest, err := netip.ParsePrefix(entry.DestinationPrefix)
if err != nil {
log.Warnf("Unable to parse route destination %s: %v", entry.DestinationPrefix, err)
continue
}
if intfIdx != "" {
script = fmt.Sprintf(
`%s -InterfaceIndex %s`, script, intfIdx,
)
} else {
script = fmt.Sprintf(
`%s -InterfaceAlias "%s"`, script, intf,
)
nexthop, err := netip.ParseAddr(entry.NextHop)
if err != nil {
log.Warnf("Unable to parse route next hop %s: %v", entry.NextHop, err)
continue
}
var intf *net.Interface
if entry.InterfaceIndex != 0 {
intf = &net.Interface{
Index: int(entry.InterfaceIndex),
Name: entry.InterfaceAlias,
}
}
routes = append(routes, Route{
Destination: dest,
Nexthop: nexthop,
Interface: intf,
})
}
if nexthop.IsValid() {
script = fmt.Sprintf(
`%s -NextHop "%s"`, script, nexthop,
)
}
out, err := exec.Command("powershell", "-Command", script).CombinedOutput()
log.Tracef("PowerShell %s: %s", script, string(out))
if err != nil {
return fmt.Errorf("PowerShell add route: %w", err)
}
return nil
return routes, nil
}
func addRouteCmd(prefix netip.Prefix, nexthop netip.Addr, _ string) error {
args := []string{"add", prefix.String(), nexthop.Unmap().String()}
func GetNeighbors() ([]Neighbor, error) {
var entries []MSFT_NetNeighbor
query := `SELECT IPAddress, LinkLayerAddress, State, AddressFamily, InterfaceIndex, InterfaceAlias FROM MSFT_NetNeighbor`
if err := wmi.QueryNamespace(query, &entries, `ROOT\StandardCimv2`); err != nil {
return nil, fmt.Errorf("failed to query MSFT_NetNeighbor: %w", err)
}
out, err := exec.Command("route", args...).CombinedOutput()
var neighbors []Neighbor
for _, entry := range entries {
addr, err := netip.ParseAddr(entry.IPAddress)
if err != nil {
log.Warnf("Unable to parse neighbor IP address %s: %v", entry.IPAddress, err)
continue
}
neighbors = append(neighbors, Neighbor{
IPAddress: addr,
LinkLayerAddress: entry.LinkLayerAddress,
State: entry.State,
AddressFamily: entry.AddressFamily,
InterfaceIndex: entry.InterfaceIndex,
InterfaceAlias: entry.InterfaceAlias,
})
}
return neighbors, nil
}
func addRouteCmd(prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error {
args := []string{"add", prefix.String()}
if nexthop.IsValid() {
args = append(args, nexthop.Unmap().String())
} else {
addr := "0.0.0.0"
if prefix.Addr().Is6() {
addr = "::"
}
args = append(args, addr)
}
if intf != nil {
args = append(args, "if", strconv.Itoa(intf.Index))
}
routeCmd := uspfilter.GetSystem32Command("route")
out, err := exec.Command(routeCmd, args...).CombinedOutput()
log.Tracef("route %s: %s", strings.Join(args, " "), out)
if err != nil {
return fmt.Errorf("route add: %w", err)
@@ -116,28 +185,29 @@ func addRouteCmd(prefix netip.Prefix, nexthop netip.Addr, _ string) error {
return nil
}
func addToRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf string) error {
var intfIdx string
if nexthop.Zone() != "" {
intfIdx = nexthop.Zone()
func addToRouteTable(prefix netip.Prefix, nexthop netip.Addr, intf *net.Interface) error {
if nexthop.Zone() != "" && intf == nil {
zone, err := strconv.Atoi(nexthop.Zone())
if err != nil {
return fmt.Errorf("invalid zone: %w", err)
}
intf = &net.Interface{Index: zone}
nexthop.WithZone("")
}
// Powershell doesn't support adding routes without an interface but allows to add interface by name
if intf != "" || intfIdx != "" {
return addRoutePowershell(prefix, nexthop, intf, intfIdx)
}
return addRouteCmd(prefix, nexthop, intf)
}
func removeFromRouteTable(prefix netip.Prefix, nexthop netip.Addr, _ string) error {
func removeFromRouteTable(prefix netip.Prefix, nexthop netip.Addr, _ *net.Interface) error {
args := []string{"delete", prefix.String()}
if nexthop.IsValid() {
nexthop.WithZone("")
args = append(args, nexthop.Unmap().String())
}
out, err := exec.Command("route", args...).CombinedOutput()
routeCmd := uspfilter.GetSystem32Command("route")
out, err := exec.Command(routeCmd, args...).CombinedOutput()
log.Tracef("route %s: %s", strings.Join(args, " "), out)
if err != nil {
@@ -145,3 +215,7 @@ func removeFromRouteTable(prefix netip.Prefix, nexthop netip.Addr, _ string) err
}
return nil
}
func isCacheDisabled() bool {
return os.Getenv("NB_DISABLE_ROUTE_CACHE") == "true"
}

View File

@@ -0,0 +1,128 @@
package routeselector
import (
"fmt"
"slices"
"strings"
"github.com/hashicorp/go-multierror"
"golang.org/x/exp/maps"
route "github.com/netbirdio/netbird/route"
)
type RouteSelector struct {
selectedRoutes map[route.NetID]struct{}
selectAll bool
}
func NewRouteSelector() *RouteSelector {
return &RouteSelector{
selectedRoutes: map[route.NetID]struct{}{},
// default selects all routes
selectAll: true,
}
}
// SelectRoutes updates the selected routes based on the provided route IDs.
func (rs *RouteSelector) SelectRoutes(routes []route.NetID, appendRoute bool, allRoutes []route.NetID) error {
if !appendRoute {
rs.selectedRoutes = map[route.NetID]struct{}{}
}
var multiErr *multierror.Error
for _, route := range routes {
if !slices.Contains(allRoutes, route) {
multiErr = multierror.Append(multiErr, fmt.Errorf("route '%s' is not available", route))
continue
}
rs.selectedRoutes[route] = struct{}{}
}
rs.selectAll = false
if multiErr != nil {
multiErr.ErrorFormat = formatError
}
return multiErr.ErrorOrNil()
}
// SelectAllRoutes sets the selector to select all routes.
func (rs *RouteSelector) SelectAllRoutes() {
rs.selectAll = true
rs.selectedRoutes = map[route.NetID]struct{}{}
}
// DeselectRoutes removes specific routes from the selection.
// If the selector is in "select all" mode, it will transition to "select specific" mode.
func (rs *RouteSelector) DeselectRoutes(routes []route.NetID, allRoutes []route.NetID) error {
if rs.selectAll {
rs.selectAll = false
rs.selectedRoutes = map[route.NetID]struct{}{}
for _, route := range allRoutes {
rs.selectedRoutes[route] = struct{}{}
}
}
var multiErr *multierror.Error
for _, route := range routes {
if !slices.Contains(allRoutes, route) {
multiErr = multierror.Append(multiErr, fmt.Errorf("route '%s' is not available", route))
continue
}
delete(rs.selectedRoutes, route)
}
if multiErr != nil {
multiErr.ErrorFormat = formatError
}
return multiErr.ErrorOrNil()
}
// DeselectAllRoutes deselects all routes, effectively disabling route selection.
func (rs *RouteSelector) DeselectAllRoutes() {
rs.selectAll = false
rs.selectedRoutes = map[route.NetID]struct{}{}
}
// IsSelected checks if a specific route is selected.
func (rs *RouteSelector) IsSelected(routeID route.NetID) bool {
if rs.selectAll {
return true
}
_, selected := rs.selectedRoutes[routeID]
return selected
}
// FilterSelected removes unselected routes from the provided map.
func (rs *RouteSelector) FilterSelected(routes route.HAMap) route.HAMap {
if rs.selectAll {
return maps.Clone(routes)
}
filtered := route.HAMap{}
for id, rt := range routes {
if rs.IsSelected(id.NetID()) {
filtered[id] = rt
}
}
return filtered
}
func formatError(es []error) string {
if len(es) == 1 {
return fmt.Sprintf("1 error occurred:\n\t* %s", es[0])
}
points := make([]string, len(es))
for i, err := range es {
points[i] = fmt.Sprintf("* %s", err)
}
return fmt.Sprintf(
"%d errors occurred:\n\t%s",
len(es), strings.Join(points, "\n\t"))
}

View File

@@ -0,0 +1,275 @@
package routeselector_test
import (
"slices"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/netbirdio/netbird/client/internal/routeselector"
"github.com/netbirdio/netbird/route"
)
func TestRouteSelector_SelectRoutes(t *testing.T) {
allRoutes := []route.NetID{"route1", "route2", "route3"}
tests := []struct {
name string
initialSelected []route.NetID
selectRoutes []route.NetID
append bool
wantSelected []route.NetID
wantError bool
}{
{
name: "Select specific routes, initial all selected",
selectRoutes: []route.NetID{"route1", "route2"},
wantSelected: []route.NetID{"route1", "route2"},
},
{
name: "Select specific routes, initial all deselected",
initialSelected: []route.NetID{},
selectRoutes: []route.NetID{"route1", "route2"},
wantSelected: []route.NetID{"route1", "route2"},
},
{
name: "Select specific routes with initial selection",
initialSelected: []route.NetID{"route1"},
selectRoutes: []route.NetID{"route2", "route3"},
wantSelected: []route.NetID{"route2", "route3"},
},
{
name: "Select non-existing route",
selectRoutes: []route.NetID{"route1", "route4"},
wantSelected: []route.NetID{"route1"},
wantError: true,
},
{
name: "Append route with initial selection",
initialSelected: []route.NetID{"route1"},
selectRoutes: []route.NetID{"route2"},
append: true,
wantSelected: []route.NetID{"route1", "route2"},
},
{
name: "Append route without initial selection",
selectRoutes: []route.NetID{"route2"},
append: true,
wantSelected: []route.NetID{"route2"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rs := routeselector.NewRouteSelector()
if tt.initialSelected != nil {
err := rs.SelectRoutes(tt.initialSelected, false, allRoutes)
require.NoError(t, err)
}
err := rs.SelectRoutes(tt.selectRoutes, tt.append, allRoutes)
if tt.wantError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
for _, id := range allRoutes {
assert.Equal(t, rs.IsSelected(id), slices.Contains(tt.wantSelected, id))
}
})
}
}
func TestRouteSelector_SelectAllRoutes(t *testing.T) {
allRoutes := []route.NetID{"route1", "route2", "route3"}
tests := []struct {
name string
initialSelected []route.NetID
wantSelected []route.NetID
}{
{
name: "Initial all selected",
wantSelected: []route.NetID{"route1", "route2", "route3"},
},
{
name: "Initial all deselected",
initialSelected: []route.NetID{},
wantSelected: []route.NetID{"route1", "route2", "route3"},
},
{
name: "Initial some selected",
initialSelected: []route.NetID{"route1"},
wantSelected: []route.NetID{"route1", "route2", "route3"},
},
{
name: "Initial all selected",
initialSelected: []route.NetID{"route1", "route2", "route3"},
wantSelected: []route.NetID{"route1", "route2", "route3"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rs := routeselector.NewRouteSelector()
if tt.initialSelected != nil {
err := rs.SelectRoutes(tt.initialSelected, false, allRoutes)
require.NoError(t, err)
}
rs.SelectAllRoutes()
for _, id := range allRoutes {
assert.Equal(t, rs.IsSelected(id), slices.Contains(tt.wantSelected, id))
}
})
}
}
func TestRouteSelector_DeselectRoutes(t *testing.T) {
allRoutes := []route.NetID{"route1", "route2", "route3"}
tests := []struct {
name string
initialSelected []route.NetID
deselectRoutes []route.NetID
wantSelected []route.NetID
wantError bool
}{
{
name: "Deselect specific routes, initial all selected",
deselectRoutes: []route.NetID{"route1", "route2"},
wantSelected: []route.NetID{"route3"},
},
{
name: "Deselect specific routes, initial all deselected",
initialSelected: []route.NetID{},
deselectRoutes: []route.NetID{"route1", "route2"},
wantSelected: []route.NetID{},
},
{
name: "Deselect specific routes with initial selection",
initialSelected: []route.NetID{"route1", "route2"},
deselectRoutes: []route.NetID{"route1", "route3"},
wantSelected: []route.NetID{"route2"},
},
{
name: "Deselect non-existing route",
initialSelected: []route.NetID{"route1", "route2"},
deselectRoutes: []route.NetID{"route1", "route4"},
wantSelected: []route.NetID{"route2"},
wantError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rs := routeselector.NewRouteSelector()
if tt.initialSelected != nil {
err := rs.SelectRoutes(tt.initialSelected, false, allRoutes)
require.NoError(t, err)
}
err := rs.DeselectRoutes(tt.deselectRoutes, allRoutes)
if tt.wantError {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
for _, id := range allRoutes {
assert.Equal(t, rs.IsSelected(id), slices.Contains(tt.wantSelected, id))
}
})
}
}
func TestRouteSelector_DeselectAll(t *testing.T) {
allRoutes := []route.NetID{"route1", "route2", "route3"}
tests := []struct {
name string
initialSelected []route.NetID
wantSelected []route.NetID
}{
{
name: "Initial all selected",
wantSelected: []route.NetID{},
},
{
name: "Initial all deselected",
initialSelected: []route.NetID{},
wantSelected: []route.NetID{},
},
{
name: "Initial some selected",
initialSelected: []route.NetID{"route1", "route2"},
wantSelected: []route.NetID{},
},
{
name: "Initial all selected",
initialSelected: []route.NetID{"route1", "route2", "route3"},
wantSelected: []route.NetID{},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rs := routeselector.NewRouteSelector()
if tt.initialSelected != nil {
err := rs.SelectRoutes(tt.initialSelected, false, allRoutes)
require.NoError(t, err)
}
rs.DeselectAllRoutes()
for _, id := range allRoutes {
assert.Equal(t, rs.IsSelected(id), slices.Contains(tt.wantSelected, id))
}
})
}
}
func TestRouteSelector_IsSelected(t *testing.T) {
rs := routeselector.NewRouteSelector()
err := rs.SelectRoutes([]route.NetID{"route1", "route2"}, false, []route.NetID{"route1", "route2", "route3"})
require.NoError(t, err)
assert.True(t, rs.IsSelected("route1"))
assert.True(t, rs.IsSelected("route2"))
assert.False(t, rs.IsSelected("route3"))
assert.False(t, rs.IsSelected("route4"))
}
func TestRouteSelector_FilterSelected(t *testing.T) {
rs := routeselector.NewRouteSelector()
err := rs.SelectRoutes([]route.NetID{"route1", "route2"}, false, []route.NetID{"route1", "route2", "route3"})
require.NoError(t, err)
routes := route.HAMap{
"route1-10.0.0.0/8": {},
"route2-192.168.0.0/16": {},
"route3-172.16.0.0/12": {},
}
filtered := rs.FilterSelected(routes)
assert.Equal(t, route.HAMap{
"route1-10.0.0.0/8": {},
"route2-192.168.0.0/16": {},
}, filtered)
}

View File

@@ -1,6 +1,7 @@
package stdnet
import (
"runtime"
"strings"
log "github.com/sirupsen/logrus"
@@ -19,7 +20,7 @@ func InterfaceFilter(disallowList []string) func(string) bool {
}
for _, s := range disallowList {
if strings.HasPrefix(iFace, s) {
if strings.HasPrefix(iFace, s) && runtime.GOOS != "ios" {
log.Tracef("ignoring interface %s - it is not allowed", iFace)
return false
}

View File

@@ -1,7 +1,7 @@
<!DOCTYPE html>
<html>
<html lang="en">
<head>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<style>
body {
display: flex;
@@ -50,16 +50,17 @@
color: black;
}
</style>
<title>NetBird Login Successful</title>
</head>
<body>
<div class="container">
<div class="logo">
<img src="https://img.mailinblue.com/6211297/images/content_library/original/64bd4ce82e1ea753e439b6a2.png">
<img alt="netbird_logo" src="https://img.mailinblue.com/6211297/images/content_library/original/64bd4ce82e1ea753e439b6a2.png">
</div>
<br>
{{ if .Error }}
<svg xmlns="http://www.w3.org/2000/svg" height="50" viewBox="0 0 100 100">
<circle cx="50" cy="50" r="45" fill="none" stroke="red" stroke-width="3"/>
<svg height="50" viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg">
<circle cx="50" cy="50" fill="none" r="45" stroke="red" stroke-width="3"/>
<path d="M30 30 L70 70 M30 70 L70 30" fill="none" stroke="red" stroke-width="3"/>
</svg>
<div class="content">
@@ -69,8 +70,8 @@
{{ .Error }}.
</div>
{{ else }}
<svg xmlns="http://www.w3.org/2000/svg" height="50" viewBox="0 0 100 100">
<circle cx="50" cy="50" r="45" fill="none" stroke="#5cb85c" stroke-width="3"/>
<svg height="50" viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg">
<circle cx="50" cy="50" fill="none" r="45" stroke="#5cb85c" stroke-width="3"/>
<path d="M30 50 L45 65 L70 35" fill="none" stroke="#5cb85c" stroke-width="5"/>
</svg>
<div class="content">

View File

@@ -1,15 +1,17 @@
package wgproxy
import "context"
type Factory struct {
wgPort int
ebpfProxy Proxy
}
func (w *Factory) GetProxy() Proxy {
func (w *Factory) GetProxy(ctx context.Context) Proxy {
if w.ebpfProxy != nil {
return w.ebpfProxy
}
return NewWGUserSpaceProxy(w.wgPort)
return NewWGUserSpaceProxy(ctx, w.wgPort)
}
func (w *Factory) Free() error {

View File

@@ -3,14 +3,16 @@
package wgproxy
import (
"context"
log "github.com/sirupsen/logrus"
)
func NewFactory(wgPort int) *Factory {
func NewFactory(ctx context.Context, wgPort int) *Factory {
f := &Factory{wgPort: wgPort}
ebpfProxy := NewWGEBPFProxy(wgPort)
err := ebpfProxy.Listen()
ebpfProxy := NewWGEBPFProxy(ctx, wgPort)
err := ebpfProxy.listen()
if err != nil {
log.Warnf("failed to initialize ebpf proxy, fallback to user space proxy: %s", err)
return f

View File

@@ -2,6 +2,8 @@
package wgproxy
func NewFactory(wgPort int) *Factory {
import "context"
func NewFactory(ctx context.Context, wgPort int) *Factory {
return &Factory{wgPort: wgPort}
}

View File

@@ -6,7 +6,7 @@ import (
// Proxy is a transfer layer between the Turn connection and the WireGuard
type Proxy interface {
AddTurnConn(urnConn net.Conn) (net.Addr, error)
AddTurnConn(turnConn net.Conn) (net.Addr, error)
CloseConn() error
Free() error
}

View File

@@ -3,6 +3,7 @@
package wgproxy
import (
"context"
"fmt"
"io"
"net"
@@ -22,7 +23,11 @@ import (
// WGEBPFProxy definition for proxy with EBPF support
type WGEBPFProxy struct {
ebpfManager ebpfMgr.Manager
ebpfManager ebpfMgr.Manager
ctx context.Context
cancel context.CancelFunc
lastUsedPort uint16
localWGListenPort int
@@ -34,7 +39,7 @@ type WGEBPFProxy struct {
}
// NewWGEBPFProxy create new WGEBPFProxy instance
func NewWGEBPFProxy(wgPort int) *WGEBPFProxy {
func NewWGEBPFProxy(ctx context.Context, wgPort int) *WGEBPFProxy {
log.Debugf("instantiate ebpf proxy")
wgProxy := &WGEBPFProxy{
localWGListenPort: wgPort,
@@ -42,11 +47,13 @@ func NewWGEBPFProxy(wgPort int) *WGEBPFProxy {
lastUsedPort: 0,
turnConnStore: make(map[uint16]net.Conn),
}
wgProxy.ctx, wgProxy.cancel = context.WithCancel(ctx)
return wgProxy
}
// Listen load ebpf program and listen the proxy
func (p *WGEBPFProxy) Listen() error {
// listen load ebpf program and listen the proxy
func (p *WGEBPFProxy) listen() error {
pl := portLookup{}
wgPorxyPort, err := pl.searchFreePort()
if err != nil {
@@ -72,7 +79,7 @@ func (p *WGEBPFProxy) Listen() error {
if err != nil {
cErr := p.Free()
if cErr != nil {
log.Errorf("failed to close the wgproxy: %s", cErr)
log.Errorf("Failed to close the wgproxy: %s", cErr)
}
return err
}
@@ -131,19 +138,27 @@ func (p *WGEBPFProxy) Free() error {
func (p *WGEBPFProxy) proxyToLocal(endpointPort uint16, remoteConn net.Conn) {
buf := make([]byte, 1500)
var err error
defer func() {
p.removeTurnConn(endpointPort)
}()
for {
n, err := remoteConn.Read(buf)
if err != nil {
if err != io.EOF {
log.Errorf("failed to read from turn conn (endpoint: :%d): %s", endpointPort, err)
}
p.removeTurnConn(endpointPort)
log.Infof("stop forward turn packages to port: %d. error: %s", endpointPort, err)
select {
case <-p.ctx.Done():
return
}
err = p.sendPkg(buf[:n], endpointPort)
if err != nil {
log.Errorf("failed to write out turn pkg to local conn: %v", err)
default:
var n int
n, err = remoteConn.Read(buf)
if err != nil {
if err != io.EOF {
log.Errorf("failed to read from turn conn (endpoint: :%d): %s", endpointPort, err)
}
return
}
err = p.sendPkg(buf[:n], endpointPort)
if err != nil {
log.Errorf("failed to write out turn pkg to local conn: %v", err)
}
}
}
}
@@ -152,23 +167,28 @@ func (p *WGEBPFProxy) proxyToLocal(endpointPort uint16, remoteConn net.Conn) {
func (p *WGEBPFProxy) proxyToRemote() {
buf := make([]byte, 1500)
for {
n, addr, err := p.conn.ReadFromUDP(buf)
if err != nil {
log.Errorf("failed to read UDP pkg from WG: %s", err)
select {
case <-p.ctx.Done():
return
}
default:
n, addr, err := p.conn.ReadFromUDP(buf)
if err != nil {
log.Errorf("failed to read UDP pkg from WG: %s", err)
return
}
p.turnConnMutex.Lock()
conn, ok := p.turnConnStore[uint16(addr.Port)]
p.turnConnMutex.Unlock()
if !ok {
log.Infof("turn conn not found by port: %d", addr.Port)
continue
}
p.turnConnMutex.Lock()
conn, ok := p.turnConnStore[uint16(addr.Port)]
p.turnConnMutex.Unlock()
if !ok {
log.Infof("turn conn not found by port: %d", addr.Port)
continue
}
_, err = conn.Write(buf[:n])
if err != nil {
log.Debugf("failed to forward local wg pkg (%d) to remote turn conn: %s", addr.Port, err)
_, err = conn.Write(buf[:n])
if err != nil {
log.Debugf("failed to forward local wg pkg (%d) to remote turn conn: %s", addr.Port, err)
}
}
}
}
@@ -266,15 +286,17 @@ func (p *WGEBPFProxy) sendPkg(data []byte, port uint16) error {
err := udpH.SetNetworkLayerForChecksum(ipH)
if err != nil {
return err
return fmt.Errorf("set network layer for checksum: %w", err)
}
layerBuffer := gopacket.NewSerializeBuffer()
err = gopacket.SerializeLayers(layerBuffer, gopacket.SerializeOptions{ComputeChecksums: true, FixLengths: true}, ipH, udpH, payload)
if err != nil {
return err
return fmt.Errorf("serialize layers: %w", err)
}
_, err = p.rawConn.WriteTo(layerBuffer.Bytes(), &net.IPAddr{IP: localhost})
return err
if _, err = p.rawConn.WriteTo(layerBuffer.Bytes(), &net.IPAddr{IP: localhost}); err != nil {
return fmt.Errorf("write to raw conn: %w", err)
}
return nil
}

View File

@@ -3,11 +3,12 @@
package wgproxy
import (
"context"
"testing"
)
func TestWGEBPFProxy_connStore(t *testing.T) {
wgProxy := NewWGEBPFProxy(1)
wgProxy := NewWGEBPFProxy(context.Background(), 1)
p, _ := wgProxy.storeTurnConn(nil)
if p != 1 {
@@ -27,7 +28,7 @@ func TestWGEBPFProxy_connStore(t *testing.T) {
}
func TestWGEBPFProxy_portCalculation_overflow(t *testing.T) {
wgProxy := NewWGEBPFProxy(1)
wgProxy := NewWGEBPFProxy(context.Background(), 1)
_, _ = wgProxy.storeTurnConn(nil)
wgProxy.lastUsedPort = 65535
@@ -43,7 +44,7 @@ func TestWGEBPFProxy_portCalculation_overflow(t *testing.T) {
}
func TestWGEBPFProxy_portCalculation_maxConn(t *testing.T) {
wgProxy := NewWGEBPFProxy(1)
wgProxy := NewWGEBPFProxy(context.Background(), 1)
for i := 0; i < 65535; i++ {
_, _ = wgProxy.storeTurnConn(nil)

View File

@@ -21,21 +21,21 @@ type WGUserSpaceProxy struct {
}
// NewWGUserSpaceProxy instantiate a user space WireGuard proxy
func NewWGUserSpaceProxy(wgPort int) *WGUserSpaceProxy {
log.Debugf("instantiate new userspace proxy")
func NewWGUserSpaceProxy(ctx context.Context, wgPort int) *WGUserSpaceProxy {
log.Debugf("Initializing new user space proxy with port %d", wgPort)
p := &WGUserSpaceProxy{
localWGListenPort: wgPort,
}
p.ctx, p.cancel = context.WithCancel(context.Background())
p.ctx, p.cancel = context.WithCancel(ctx)
return p
}
// AddTurnConn start the proxy with the given remote conn
func (p *WGUserSpaceProxy) AddTurnConn(remoteConn net.Conn) (net.Addr, error) {
p.remoteConn = remoteConn
func (p *WGUserSpaceProxy) AddTurnConn(turnConn net.Conn) (net.Addr, error) {
p.remoteConn = turnConn
var err error
p.localConn, err = nbnet.NewDialer().Dial("udp", fmt.Sprintf(":%d", p.localWGListenPort))
p.localConn, err = nbnet.NewDialer().DialContext(p.ctx, "udp", fmt.Sprintf(":%d", p.localWGListenPort))
if err != nil {
log.Errorf("failed dialing to local Wireguard port %s", err)
return nil, err

View File

@@ -2,10 +2,15 @@ package NetBirdSDK
import (
"context"
"fmt"
"net/netip"
"sort"
"strings"
"sync"
"time"
log "github.com/sirupsen/logrus"
"golang.org/x/exp/maps"
"github.com/netbirdio/netbird/client/internal"
"github.com/netbirdio/netbird/client/internal/auth"
@@ -14,6 +19,7 @@ import (
"github.com/netbirdio/netbird/client/internal/peer"
"github.com/netbirdio/netbird/client/system"
"github.com/netbirdio/netbird/formatter"
"github.com/netbirdio/netbird/route"
)
// ConnectionListener export internal Listener for mobile
@@ -38,6 +44,12 @@ type CustomLogger interface {
Error(message string)
}
type selectRoute struct {
NetID string
Network netip.Prefix
Selected bool
}
func init() {
formatter.SetLogcatFormatter(log.StandardLogger())
}
@@ -55,6 +67,7 @@ type Client struct {
onHostDnsFn func([]string)
dnsManager dns.IosDnsManager
loginComplete bool
connectClient *internal.ConnectClient
}
// NewClient instantiate a new Client
@@ -107,7 +120,9 @@ func (c *Client) Run(fd int32, interfaceName string) error {
ctx = internal.CtxInitState(ctx)
c.onHostDnsFn = func([]string) {}
cfg.WgIface = interfaceName
return internal.RunClientiOS(ctx, cfg, c.recorder, fd, c.networkChangeListener, c.dnsManager)
c.connectClient = internal.NewConnectClient(ctx, cfg, c.recorder)
return c.connectClient.RunOniOS(fd, c.networkChangeListener, c.dnsManager)
}
// Stop the internal client and free the resources
@@ -133,10 +148,29 @@ func (c *Client) GetStatusDetails() *StatusDetails {
peerInfos := make([]PeerInfo, len(fullStatus.Peers))
for n, p := range fullStatus.Peers {
var routes = RoutesDetails{}
for r := range p.GetRoutes() {
routeInfo := RoutesInfo{r}
routes.items = append(routes.items, routeInfo)
}
pi := PeerInfo{
p.IP,
p.FQDN,
p.ConnStatus.String(),
IP: p.IP,
FQDN: p.FQDN,
LocalIceCandidateEndpoint: p.LocalIceCandidateEndpoint,
RemoteIceCandidateEndpoint: p.RemoteIceCandidateEndpoint,
LocalIceCandidateType: p.LocalIceCandidateType,
RemoteIceCandidateType: p.RemoteIceCandidateType,
PubKey: p.PubKey,
Latency: formatDuration(p.Latency),
BytesRx: p.BytesRx,
BytesTx: p.BytesTx,
ConnStatus: p.ConnStatus.String(),
ConnStatusUpdate: p.ConnStatusUpdate.Format("2006-01-02 15:04:05"),
Direct: p.Direct,
LastWireguardHandshake: p.LastWireguardHandshake.String(),
Relayed: p.Relayed,
RosenpassEnabled: p.RosenpassEnabled,
Routes: routes,
}
peerInfos[n] = pi
}
@@ -223,3 +257,142 @@ func (c *Client) IsLoginComplete() bool {
func (c *Client) ClearLoginComplete() {
c.loginComplete = false
}
func (c *Client) GetRoutesSelectionDetails() (*RoutesSelectionDetails, error) {
if c.connectClient == nil {
return nil, fmt.Errorf("not connected")
}
engine := c.connectClient.Engine()
if engine == nil {
return nil, fmt.Errorf("not connected")
}
routesMap := engine.GetClientRoutesWithNetID()
routeSelector := engine.GetRouteManager().GetRouteSelector()
var routes []*selectRoute
for id, rt := range routesMap {
if len(rt) == 0 {
continue
}
route := &selectRoute{
NetID: string(id),
Network: rt[0].Network,
Selected: routeSelector.IsSelected(id),
}
routes = append(routes, route)
}
sort.Slice(routes, func(i, j int) bool {
iPrefix := routes[i].Network.Bits()
jPrefix := routes[j].Network.Bits()
if iPrefix == jPrefix {
iAddr := routes[i].Network.Addr()
jAddr := routes[j].Network.Addr()
if iAddr == jAddr {
return routes[i].NetID < routes[j].NetID
}
return iAddr.String() < jAddr.String()
}
return iPrefix < jPrefix
})
var routeSelection []RoutesSelectionInfo
for _, r := range routes {
routeSelection = append(routeSelection, RoutesSelectionInfo{
ID: r.NetID,
Network: r.Network.String(),
Selected: r.Selected,
})
}
routeSelectionDetails := RoutesSelectionDetails{items: routeSelection}
return &routeSelectionDetails, nil
}
func (c *Client) SelectRoute(id string) error {
if c.connectClient == nil {
return fmt.Errorf("not connected")
}
engine := c.connectClient.Engine()
if engine == nil {
return fmt.Errorf("not connected")
}
routeManager := engine.GetRouteManager()
routeSelector := routeManager.GetRouteSelector()
if id == "All" {
log.Debugf("select all routes")
routeSelector.SelectAllRoutes()
} else {
log.Debugf("select route with id: %s", id)
routes := toNetIDs([]string{id})
if err := routeSelector.SelectRoutes(routes, true, maps.Keys(engine.GetClientRoutesWithNetID())); err != nil {
log.Debugf("error when selecting routes: %s", err)
return fmt.Errorf("select routes: %w", err)
}
}
routeManager.TriggerSelection(engine.GetClientRoutes())
return nil
}
func (c *Client) DeselectRoute(id string) error {
if c.connectClient == nil {
return fmt.Errorf("not connected")
}
engine := c.connectClient.Engine()
if engine == nil {
return fmt.Errorf("not connected")
}
routeManager := engine.GetRouteManager()
routeSelector := routeManager.GetRouteSelector()
if id == "All" {
log.Debugf("deselect all routes")
routeSelector.DeselectAllRoutes()
} else {
log.Debugf("deselect route with id: %s", id)
routes := toNetIDs([]string{id})
if err := routeSelector.DeselectRoutes(routes, maps.Keys(engine.GetClientRoutesWithNetID())); err != nil {
log.Debugf("error when deselecting routes: %s", err)
return fmt.Errorf("deselect routes: %w", err)
}
}
routeManager.TriggerSelection(engine.GetClientRoutes())
return nil
}
func formatDuration(d time.Duration) string {
ds := d.String()
dotIndex := strings.Index(ds, ".")
if dotIndex != -1 {
// Determine end of numeric part, ensuring we stop at two decimal places or the actual end if fewer
endIndex := dotIndex + 3
if endIndex > len(ds) {
endIndex = len(ds)
}
// Find where the numeric part ends by finding the first non-digit character after the dot
unitStart := endIndex
for unitStart < len(ds) && (ds[unitStart] >= '0' && ds[unitStart] <= '9') {
unitStart++
}
// Ensures that we only take the unit characters after the numerical part
if unitStart < len(ds) {
return ds[:endIndex] + ds[unitStart:]
}
return ds[:endIndex] // In case no units are found after the digits
}
return ds
}
func toNetIDs(routes []string) []route.NetID {
var netIDs []route.NetID
for _, rt := range routes {
netIDs = append(netIDs, route.NetID(rt))
}
return netIDs
}

View File

@@ -2,9 +2,28 @@ package NetBirdSDK
// PeerInfo describe information about the peers. It designed for the UI usage
type PeerInfo struct {
IP string
FQDN string
ConnStatus string // Todo replace to enum
IP string
FQDN string
LocalIceCandidateEndpoint string
RemoteIceCandidateEndpoint string
LocalIceCandidateType string
RemoteIceCandidateType string
PubKey string
Latency string
BytesRx int64
BytesTx int64
ConnStatus string
ConnStatusUpdate string
Direct bool
LastWireguardHandshake string
Relayed bool
RosenpassEnabled bool
Routes RoutesDetails
}
// GetRoutes return with RouteDetails
func (p PeerInfo) GetRouteDetails() *RoutesDetails {
return &p.Routes
}
// PeerInfoCollection made for Java layer to get non default types as collection
@@ -16,6 +35,21 @@ type PeerInfoCollection interface {
GetIP() string
}
// RoutesInfoCollection made for Java layer to get non default types as collection
type RoutesInfoCollection interface {
Add(s string) RoutesInfoCollection
Get(i int) string
Size() int
}
type RoutesDetails struct {
items []RoutesInfo
}
type RoutesInfo struct {
Route string
}
// StatusDetails is the implementation of the PeerInfoCollection
type StatusDetails struct {
items []PeerInfo
@@ -23,6 +57,22 @@ type StatusDetails struct {
ip string
}
// Add new PeerInfo to the collection
func (array RoutesDetails) Add(s RoutesInfo) RoutesDetails {
array.items = append(array.items, s)
return array
}
// Get return an element of the collection
func (array RoutesDetails) Get(i int) *RoutesInfo {
return &array.items[i]
}
// Size return with the size of the collection
func (array RoutesDetails) Size() int {
return len(array.items)
}
// Add new PeerInfo to the collection
func (array StatusDetails) Add(s PeerInfo) StatusDetails {
array.items = append(array.items, s)

View File

@@ -71,6 +71,42 @@ func (p *Preferences) SetPreSharedKey(key string) {
p.configInput.PreSharedKey = &key
}
// SetRosenpassEnabled store if rosenpass is enabled
func (p *Preferences) SetRosenpassEnabled(enabled bool) {
p.configInput.RosenpassEnabled = &enabled
}
// GetRosenpassEnabled read rosenpass enabled from config file
func (p *Preferences) GetRosenpassEnabled() (bool, error) {
if p.configInput.RosenpassEnabled != nil {
return *p.configInput.RosenpassEnabled, nil
}
cfg, err := internal.ReadConfig(p.configInput.ConfigPath)
if err != nil {
return false, err
}
return cfg.RosenpassEnabled, err
}
// SetRosenpassPermissive store the given permissive and wait for commit
func (p *Preferences) SetRosenpassPermissive(permissive bool) {
p.configInput.RosenpassPermissive = &permissive
}
// GetRosenpassPermissive read rosenpass permissive from config file
func (p *Preferences) GetRosenpassPermissive() (bool, error) {
if p.configInput.RosenpassPermissive != nil {
return *p.configInput.RosenpassPermissive, nil
}
cfg, err := internal.ReadConfig(p.configInput.ConfigPath)
if err != nil {
return false, err
}
return cfg.RosenpassPermissive, err
}
// Commit write out the changes into config file
func (p *Preferences) Commit() error {
_, err := internal.UpdateOrCreateConfig(p.configInput)

View File

@@ -0,0 +1,36 @@
package NetBirdSDK
// RoutesSelectionInfoCollection made for Java layer to get non default types as collection
type RoutesSelectionInfoCollection interface {
Add(s string) RoutesSelectionInfoCollection
Get(i int) string
Size() int
}
type RoutesSelectionDetails struct {
All bool
Append bool
items []RoutesSelectionInfo
}
type RoutesSelectionInfo struct {
ID string
Network string
Selected bool
}
// Add new PeerInfo to the collection
func (array RoutesSelectionDetails) Add(s RoutesSelectionInfo) RoutesSelectionDetails {
array.items = append(array.items, s)
return array
}
// Get return an element of the collection
func (array RoutesSelectionDetails) Get(i int) *RoutesSelectionInfo {
return &array.items[i]
}
// Size return with the size of the collection
func (array RoutesSelectionDetails) Size() int {
return len(array.items)
}

Some files were not shown because too many files have changed in this diff Show More