SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-06-10 09:22:09 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

"Connected" hosts are not reachable with asymetric firewall policies #669

New Issue
Open
opened 2025-11-20 05:15:43 -05:00 by saavagebueno · 6 comments
saavagebueno commented 2025-11-20 05:15:43 -05:00
Owner
Copy Link

Originally created by @synfinatic on GitHub (Mar 1, 2024).

Describe the problem

Have two hosts (A & B) on two subnets (netA = 172.16.1.0/24 & netB = 172.16.3.0/24), connected to the same firewall and are running the netbird client.

Per the firewall rules:
hosts on netA can talk to anyone.
hosts on netB can only talk to the internet.

I'm finding that A & B are not always able to talk to each other, even when on connected to NetBird and showing connected. However, hosts are showing connected via P2P which is not possible per my firewall rules since B can not talk directly to A.

To Reproduce

Configure your network as defined above. Verify connected status via netbird status -d on both hosts. Try pinging the other host on the NetBird address (100.93.x.x)

Expected behavior

They should be able to talk to each other when connected and not be able to talk to each other when one or both hosts are not connected to NetBird.

Are you using NetBird Cloud?

Yes.

NetBird version

A: 0.25.7 (linux)
B: 0.25.8 (macOS)

NetBird status -d output:

Host A:

$ netbird status -d
Peers detail:
 caolila.netbird.cloud:
  NetBird IP: 100.93.171.170/32
  Public key: x3rPyqU1G5cqeFcN2N8iCV/lcrIPpklNVHBa/IjMFko=
  Status: Disconnected
  -- detail --
  Connection type: P2P
  Direct: false
  ICE candidate (Local/Remote): srflx/host
  ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814
  Last connection update: 2024-03-01 23:29:43
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/240 B

 iphone-github2-1.netbird.cloud:
  NetBird IP: 100.93.198.37/32
  Public key: hzR3AS5t7RYDY7Y1elge+rG+PwFIUg1c6s9xMJbknXQ=
  Status: Disconnected
  -- detail --
  Connection type: P2P
  Direct: false
  ICE candidate (Local/Remote): srflx/host
  ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814
  Last connection update: 2024-03-01 23:29:43
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/240 B

 rutherford.netbird.cloud:
  NetBird IP: 100.93.155.75
  Public key: FLVSVf+xiT7GlaxTr0yhniIDQrODSh8l79WtiKpSbz0=
  Status: Connected
  -- detail --
  Connection type: P2P
  Direct: true
  ICE candidate (Local/Remote): srflx/host
  ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814
  Last connection update: 2024-03-02 01:13:11
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/240 B

 desktop-5egf8eb.netbird.cloud:
  NetBird IP: 100.93.241.251
  Public key: PP3HOrxgg5VZLLSwGlfkYFqZA5o6qV3aHd83kigcR0A=
  Status: Disconnected
  -- detail --
  Connection type: P2P
  Direct: false
  ICE candidate (Local/Remote): srflx/host
  ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814
  Last connection update: -
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/240 B

Daemon version: 0.25.7
CLI version: 0.25.7
Management: Connected to https://api.netbird.io:443
Signal: Connected to https://signal.netbird.io:443
Relays:
  [stun:stun.netbird.io:5555] is Available
  [turns:turn.netbird.io:443?transport=tcp] is Available
FQDN: raspi-blue.netbird.cloud
NetBird IP: 100.93.254.165/16
Interface type: Kernel
Peers count: 1/4 Connected

$ ping 100.93.155.75
PING 100.93.155.75 (100.93.155.75) 56(84) bytes of data.
From 100.93.254.165 icmp_seq=3 Destination Host Unreachable
ping: sendmsg: Required key not available
From 100.93.254.165 icmp_seq=4 Destination Host Unreachable
ping: sendmsg: Required key not available
From 100.93.254.165 icmp_seq=5 Destination Host Unreachable
ping: sendmsg: Required key not available
From 100.93.254.165 icmp_seq=6 Destination Host Unreachable
ping: sendmsg: Required key not available
^C
--- 100.93.155.75 ping statistics ---
6 packets transmitted, 0 received, +4 errors, 100% packet loss, time 5101ms

Host B:

$ netbird status -d
Peers detail:
 caolila.netbird.cloud:
  NetBird IP: 100.93.171.170/32
  Public key: x3rPyqU1G5cqeFcN2N8iCV/lcrIPpklNVHBa/IjMFko=
  Status: Disconnected
  -- detail --
  Connection type: P2P
  Direct: false
  ICE candidate (Local/Remote): host/host
  ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820
  Last connection update: 2024-03-01 15:29:43
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/388 B

 iphone-github2-1.netbird.cloud:
  NetBird IP: 100.93.198.37/32
  Public key: hzR3AS5t7RYDY7Y1elge+rG+PwFIUg1c6s9xMJbknXQ=
  Status: Disconnected
  -- detail --
  Connection type: P2P
  Direct: false
  ICE candidate (Local/Remote): host/host
  ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820
  Last connection update: 2024-03-01 15:29:43
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/388 B

 desktop-5egf8eb.netbird.cloud:
  NetBird IP: 100.93.241.251
  Public key: PP3HOrxgg5VZLLSwGlfkYFqZA5o6qV3aHd83kigcR0A=
  Status: Disconnected
  -- detail --
  Connection type: P2P
  Direct: false
  ICE candidate (Local/Remote): host/host
  ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820
  Last connection update: -
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/388 B

 raspi-blue.netbird.cloud:
  NetBird IP: 100.93.254.165
  Public key: 35vmbpakWkYp08SIIIxpELwJcytv6nHfjQy3wcSgImU=
  Status: Connected
  -- detail --
  Connection type: P2P
  Direct: true
  ICE candidate (Local/Remote): host/host
  ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820
  Last connection update: 2024-03-01 17:14:07
  Last Wireguard handshake: -
  Transfer status (received/sent) 148 B/388 B

Daemon version: 0.25.8
CLI version: 0.25.8
Management: Connected to https://api.netbird.io:443
Signal: Connected to https://signal.netbird.io:443
Relays:
  [stun:stun.netbird.io:5555] is Available
  [turns:turn.netbird.io:443?transport=tcp] is Available
FQDN: rutherford.netbird.cloud
NetBird IP: 100.93.155.75/16
Interface type: Userspace
Peers count: 1/4 Connected

aturner@Rutherford [01:14:12]  main (bd645be) [!✓]
~> ping 100.93.254.165
PING 100.93.254.165 (100.93.254.165): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
^C
--- 100.93.254.165 ping statistics ---
5 packets transmitted, 0 packets received, 100.0% packet loss

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

If you look at the connection details for Rutherford (B) & raspi-blue (A), you'll see that they are P2P which isn't actually possible per the firewall rules since Rutherford can not talk to raspi-blue directly. Firewall in this case is pfSense.

I will say that this seemed to have worked sometimes about a week ago? But recently seems to have stopped choosing the coturn path much more consistently lately, so not sure if there was a server/backend change that explains it getting worse. I've tried disconnecting & reconnecting host B (Rutherford) multiple times and it seems to be reliably picking the invalid P2P path now.

Originally created by @synfinatic on GitHub (Mar 1, 2024). **Describe the problem** Have two hosts (A & B) on two subnets (netA = 172.16.1.0/24 & netB = 172.16.3.0/24), connected to the same firewall and are running the netbird client. Per the firewall rules: hosts on netA can talk to anyone. hosts on netB can only talk to the internet. I'm finding that A & B are not always able to talk to each other, even when on connected to NetBird and showing connected. However, hosts are showing connected via P2P which is not possible per my firewall rules since B can not talk directly to A. **To Reproduce** Configure your network as defined above. Verify connected status via `netbird status -d` on both hosts. Try pinging the other host on the NetBird address (100.93.x.x) **Expected behavior** They should be able to talk to each other when connected and not be able to talk to each other when one or both hosts are not connected to NetBird. **Are you using NetBird Cloud?** Yes. **NetBird version** A: 0.25.7 (linux) B: 0.25.8 (macOS) **NetBird status -d output:** Host A: ``` $ netbird status -d Peers detail: caolila.netbird.cloud: NetBird IP: 100.93.171.170/32 Public key: x3rPyqU1G5cqeFcN2N8iCV/lcrIPpklNVHBa/IjMFko= Status: Disconnected -- detail -- Connection type: P2P Direct: false ICE candidate (Local/Remote): srflx/host ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814 Last connection update: 2024-03-01 23:29:43 Last Wireguard handshake: - Transfer status (received/sent) 148 B/240 B iphone-github2-1.netbird.cloud: NetBird IP: 100.93.198.37/32 Public key: hzR3AS5t7RYDY7Y1elge+rG+PwFIUg1c6s9xMJbknXQ= Status: Disconnected -- detail -- Connection type: P2P Direct: false ICE candidate (Local/Remote): srflx/host ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814 Last connection update: 2024-03-01 23:29:43 Last Wireguard handshake: - Transfer status (received/sent) 148 B/240 B rutherford.netbird.cloud: NetBird IP: 100.93.155.75 Public key: FLVSVf+xiT7GlaxTr0yhniIDQrODSh8l79WtiKpSbz0= Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/host ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814 Last connection update: 2024-03-02 01:13:11 Last Wireguard handshake: - Transfer status (received/sent) 148 B/240 B desktop-5egf8eb.netbird.cloud: NetBird IP: 100.93.241.251 Public key: PP3HOrxgg5VZLLSwGlfkYFqZA5o6qV3aHd83kigcR0A= Status: Disconnected -- detail -- Connection type: P2P Direct: false ICE candidate (Local/Remote): srflx/host ICE candidate endpoints (Local/Remote): 71.202.196.214:32814/172.16.3.242:32814 Last connection update: - Last Wireguard handshake: - Transfer status (received/sent) 148 B/240 B Daemon version: 0.25.7 CLI version: 0.25.7 Management: Connected to https://api.netbird.io:443 Signal: Connected to https://signal.netbird.io:443 Relays: [stun:stun.netbird.io:5555] is Available [turns:turn.netbird.io:443?transport=tcp] is Available FQDN: raspi-blue.netbird.cloud NetBird IP: 100.93.254.165/16 Interface type: Kernel Peers count: 1/4 Connected $ ping 100.93.155.75 PING 100.93.155.75 (100.93.155.75) 56(84) bytes of data. From 100.93.254.165 icmp_seq=3 Destination Host Unreachable ping: sendmsg: Required key not available From 100.93.254.165 icmp_seq=4 Destination Host Unreachable ping: sendmsg: Required key not available From 100.93.254.165 icmp_seq=5 Destination Host Unreachable ping: sendmsg: Required key not available From 100.93.254.165 icmp_seq=6 Destination Host Unreachable ping: sendmsg: Required key not available ^C --- 100.93.155.75 ping statistics --- 6 packets transmitted, 0 received, +4 errors, 100% packet loss, time 5101ms ``` Host B: ``` $ netbird status -d Peers detail: caolila.netbird.cloud: NetBird IP: 100.93.171.170/32 Public key: x3rPyqU1G5cqeFcN2N8iCV/lcrIPpklNVHBa/IjMFko= Status: Disconnected -- detail -- Connection type: P2P Direct: false ICE candidate (Local/Remote): host/host ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820 Last connection update: 2024-03-01 15:29:43 Last Wireguard handshake: - Transfer status (received/sent) 148 B/388 B iphone-github2-1.netbird.cloud: NetBird IP: 100.93.198.37/32 Public key: hzR3AS5t7RYDY7Y1elge+rG+PwFIUg1c6s9xMJbknXQ= Status: Disconnected -- detail -- Connection type: P2P Direct: false ICE candidate (Local/Remote): host/host ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820 Last connection update: 2024-03-01 15:29:43 Last Wireguard handshake: - Transfer status (received/sent) 148 B/388 B desktop-5egf8eb.netbird.cloud: NetBird IP: 100.93.241.251 Public key: PP3HOrxgg5VZLLSwGlfkYFqZA5o6qV3aHd83kigcR0A= Status: Disconnected -- detail -- Connection type: P2P Direct: false ICE candidate (Local/Remote): host/host ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820 Last connection update: - Last Wireguard handshake: - Transfer status (received/sent) 148 B/388 B raspi-blue.netbird.cloud: NetBird IP: 100.93.254.165 Public key: 35vmbpakWkYp08SIIIxpELwJcytv6nHfjQy3wcSgImU= Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): host/host ICE candidate endpoints (Local/Remote): 172.16.3.242:51820/172.16.1.139:51820 Last connection update: 2024-03-01 17:14:07 Last Wireguard handshake: - Transfer status (received/sent) 148 B/388 B Daemon version: 0.25.8 CLI version: 0.25.8 Management: Connected to https://api.netbird.io:443 Signal: Connected to https://signal.netbird.io:443 Relays: [stun:stun.netbird.io:5555] is Available [turns:turn.netbird.io:443?transport=tcp] is Available FQDN: rutherford.netbird.cloud NetBird IP: 100.93.155.75/16 Interface type: Userspace Peers count: 1/4 Connected aturner@Rutherford [01:14:12]  main (bd645be) [!✓] ~> ping 100.93.254.165 PING 100.93.254.165 (100.93.254.165): 56 data bytes Request timeout for icmp_seq 0 Request timeout for icmp_seq 1 Request timeout for icmp_seq 2 Request timeout for icmp_seq 3 ^C --- 100.93.254.165 ping statistics --- 5 packets transmitted, 0 packets received, 100.0% packet loss ``` **Screenshots** If applicable, add screenshots to help explain your problem. **Additional context** If you look at the connection details for Rutherford (B) & raspi-blue (A), you'll see that they are P2P which isn't actually possible per the firewall rules since Rutherford can not talk to raspi-blue directly. Firewall in this case is pfSense. I will say that this seemed to have worked sometimes about a week ago? But recently seems to have stopped choosing the coturn path much more consistently lately, so not sure if there was a server/backend change that explains it getting worse. I've tried disconnecting & reconnecting host B (Rutherford) multiple times and it seems to be reliably picking the invalid P2P path now.
saavagebueno added the bugtriage-needed labels 2025-11-20 05:15:43 -05:00
saavagebueno commented 2025-11-20 05:15:44 -05:00
Author
Owner
Copy Link

@synfinatic commented on GitHub (Mar 4, 2024):

updated both nodes to 0.26.2 and problem is the same.

@synfinatic commented on GitHub (Mar 4, 2024): updated both nodes to `0.26.2` and problem is the same.
saavagebueno commented 2025-11-20 05:15:44 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 4, 2024):

@synfinatic, can you confirm if the last wireguard handshake time is empty on both peers?

Also, from your shared status, there was a difference between the last update between both peers. And in the first ping test, the ping: sendmsg: Required key not available indicated that the wireguard didn't had the public key for the other peer.

@mlsmaycon commented on GitHub (Mar 4, 2024): @synfinatic, can you confirm if the last wireguard handshake time is empty on both peers? Also, from your shared status, there was a difference between the last update between both peers. And in the first ping test, the `ping: sendmsg: Required key not available` indicated that the wireguard didn't had the public key for the other peer.
saavagebueno commented 2025-11-20 05:15:44 -05:00
Author
Owner
Copy Link

@synfinatic commented on GitHub (Mar 4, 2024):

I can confirm that the "Last Wireguard Handshake" is indeed empty on both peers. However, both peers are reporting the public key for the other side?

@synfinatic commented on GitHub (Mar 4, 2024): I can confirm that the "Last Wireguard Handshake" is indeed empty on both peers. However, both peers are reporting the public key for the other side?
saavagebueno commented 2025-11-20 05:15:45 -05:00
Author
Owner
Copy Link

@synfinatic commented on GitHub (Mar 4, 2024):

FWIW, the output from Host B showing the local/remote IP's for Host A being directly routable seem to be the big issue here. Due to the firewall rules, there is no path from 172.16.3.0/24 to 172.16.1.0/24.

@synfinatic commented on GitHub (Mar 4, 2024): FWIW, the output from Host B showing the local/remote IP's for Host A being directly routable seem to be the big issue here. Due to the firewall rules, there is no path from 172.16.3.0/24 to 172.16.1.0/24.
saavagebueno commented 2025-11-20 05:15:45 -05:00
Author
Owner
Copy Link

@nazarewk commented on GitHub (Apr 18, 2025):

@synfinatic is this still an issue for you?

@nazarewk commented on GitHub (Apr 18, 2025): @synfinatic is this still an issue for you?
saavagebueno commented 2025-11-20 05:15:45 -05:00
Author
Owner
Copy Link

@synfinatic commented on GitHub (Apr 18, 2025):

Been ages since I've tried. You can close this ticket if you can't quickly reproduce.

@synfinatic commented on GitHub (Apr 18, 2025): Been ages since I've tried. You can close this ticket if you can't quickly reproduce.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label bug triage-needed
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#669
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?