[GH-ISSUE #3648] P2P Connection Across Subnets / VLANs? #7794

Closed
opened 2026-08-05 01:14:35 -04:00 by saavagebueno · 7 comments
Owner

Originally created by @1nerdyguy on GitHub (Apr 9, 2025).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/3648

I have a site that has multiple subnets behind the same public IP address, isolated via vlans. These subnets/vlans are routed via Layer 3 switches, like you'd find in most enterprise environments.

Currently, if I have clients on the same subnet, with netbird, they will connect P2P, which is solid and I love it.

However, if a client is on a different subnet, it gets relayed.

Example:

Host A has a Local IP of 192.168.1.100/24 Gateway 192.168.1.1 Public IP 174.11.11.1/32

Host B Has a local ip of 192.168.2.100/24 Gateway 192.168.2.1 Public IP 174.11.11.1/32

Host A can ping Host B via the 192.168.x.x IP's just fine, sub 1ms latency, we're happy. No firewall exists between, no Nat, nothing.

If I install Netbird on both hosts, put them in the same group, login and do a netbird up, Host A can see Host B via it's Netbird Address, however it is reporting as Relayed.

Even more so, in the output of Netbird Status -d the relayed hosts report:

CLIENTA.netbird.selfhosted:
NetBird IP: 100.108.190.14
Public key: oxSWCOjV2/C9T1GtQIKV8UY2BQHoAikCpMC05qI3xlc=
Status: Connected
-- detail --
Connection type: Relayed
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address: rel://anon-A65NJ.domain:33080

While the non-replayed hosts report:

CLIENTB.netbird.selfhosted:
NetBird IP: 100.108.229.81
Public key: Cj+Zp/t5Ej+Zs53GcwmnrtNBjhnYtIbasoNax7oubXA=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/host
ICE candidate endpoints (Local/Remote): 192.168.1.100:51820/192.168.1.101:51820
Relay server address: rel://anon-A65NJ.domain:33080

Originally created by @1nerdyguy on GitHub (Apr 9, 2025). Original GitHub issue: https://github.com/netbirdio/netbird/issues/3648 I have a site that has multiple subnets behind the same public IP address, isolated via vlans. These subnets/vlans are routed via Layer 3 switches, like you'd find in most enterprise environments. Currently, if I have clients on the same subnet, with netbird, they will connect P2P, which is solid and I love it. However, if a client is on a different subnet, it gets relayed. Example: Host A has a Local IP of 192.168.1.100/24 Gateway 192.168.1.1 Public IP 174.11.11.1/32 Host B Has a local ip of 192.168.2.100/24 Gateway 192.168.2.1 Public IP 174.11.11.1/32 Host A can ping Host B via the 192.168.x.x IP's just fine, sub 1ms latency, we're happy. No firewall exists between, no Nat, nothing. If I install Netbird on both hosts, put them in the same group, login and do a netbird up, Host A can see Host B via it's Netbird Address, however it is reporting as Relayed. Even more so, in the output of Netbird Status -d the relayed hosts report: CLIENTA.netbird.selfhosted: NetBird IP: 100.108.190.14 Public key: oxSWCOjV2/C9T1GtQIKV8UY2BQHoAikCpMC05qI3xlc= Status: Connected -- detail -- Connection type: Relayed ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: rel://anon-A65NJ.domain:33080 While the non-replayed hosts report: CLIENTB.netbird.selfhosted: NetBird IP: 100.108.229.81 Public key: Cj+Zp/t5Ej+Zs53GcwmnrtNBjhnYtIbasoNax7oubXA= Status: Connected -- detail -- Connection type: P2P ICE candidate (Local/Remote): host/host ICE candidate endpoints (Local/Remote): 192.168.1.100:51820/192.168.1.101:51820 Relay server address: rel://anon-A65NJ.domain:33080
Author
Owner

@1nerdyguy commented on GitHub (Apr 9, 2025):

IP's sanitized for use case.

I'd just love to be able to have p2p connections for anything that's 'onsite' with each other/behind the same firewall/public IP, and I can accept relaying offsite

<!-- gh-comment-id:2790113072 --> @1nerdyguy commented on GitHub (Apr 9, 2025): IP's sanitized for use case. I'd just love to be able to have p2p connections for anything that's 'onsite' with each other/behind the same firewall/public IP, and I can accept relaying offsite
Author
Owner

@nazarewk commented on GitHub (Apr 9, 2025):

Can you enable trace logs and send us a debug bundle to analyze for both peers?
You can send it to support at netbird.io or directly to me on the Slack (kdn).

If you wanted to analyze this yourself, you should identify which Public Key belongs to which Peer and grep client.log for the public key combined with the following keywords:

  • OnNewOffer for ICE / ICE ConnectionState - start/end of the selection process
  • discovered local candidate - peer's local addressing candidates
  • OnRemoteCandidate - the remote peer's addressing candidates
  • selected candidate pair - should be there if the P2P connection succeeds, don't remember what would be the message if it fails
<!-- gh-comment-id:2790145904 --> @nazarewk commented on GitHub (Apr 9, 2025): Can you enable trace logs and send us a debug bundle to analyze for both peers? You can send it to support at netbird.io or directly to me on the Slack (kdn). If you wanted to analyze this yourself, you should identify which Public Key belongs to which Peer and grep `client.log` for the public key combined with the following keywords: - `OnNewOffer for ICE` / `ICE ConnectionState` - start/end of the selection process - `discovered local candidate` - peer's local addressing candidates - `OnRemoteCandidate` - the remote peer's addressing candidates - `selected candidate pair` - should be there if the P2P connection succeeds, don't remember what would be the message if it fails
Author
Owner

@1nerdyguy commented on GitHub (Apr 9, 2025):

@nazarewk Thank you. I have sent them via DM in slack.

<!-- gh-comment-id:2790192914 --> @1nerdyguy commented on GitHub (Apr 9, 2025): @nazarewk Thank you. I have sent them via DM in slack.
Author
Owner

@1nerdyguy commented on GitHub (Apr 9, 2025):

Reviewing those logs myself (very inexpertly), I see that both sides are sending offers, but neither side seems to be acknowledging the offer

<!-- gh-comment-id:2790213271 --> @1nerdyguy commented on GitHub (Apr 9, 2025): Reviewing those logs myself (very inexpertly), I see that both sides are sending offers, but neither side seems to be acknowledging the offer
Author
Owner

@1nerdyguy commented on GitHub (Apr 9, 2025):

So, update:

It never even clicked with me that I had a network route in place for one of the subnets. I disabled that route, and all clients now have P2P.

This does tweak my question a bit, but still remains in the same vein:

Can I have a network route for a range such as 192.168.2.x/24 AND have clients with Netbird on them in that range and get p2p with the hosts that are on netbird direct?

The vlans are currently mixed use, so not all endpoints in the 192.168.2.x would support having netbird on them, so the thought was 'put it where it can be, but the route covers the rest'

<!-- gh-comment-id:2790337253 --> @1nerdyguy commented on GitHub (Apr 9, 2025): So, update: It never even *clicked* with me that I had a network route in place for one of the subnets. I disabled that route, and all clients now have P2P. This does tweak my question a bit, but still remains in the same vein: Can I have a network route for a range such as 192.168.2.x/24 AND have clients with Netbird on them in that range and get p2p with the hosts that are on netbird direct? The vlans are currently mixed use, so not all endpoints in the 192.168.2.x would support having netbird on them, so the thought was 'put it where it can be, but the route covers the rest'
Author
Owner

@nazarewk commented on GitHub (Apr 9, 2025):

Can I have a network route for a range such as 192.168.2.x/24 AND have clients with Netbird on them in that range and get p2p with the hosts that are on netbird direct?

This would be possible on the backend if the IP addressing Posture Checks were available for free (you could filter out advertisements to the members of networks).
For now, you can try to run a netbird networks [ls|select|deselect] based script periodically to reconcile the routes.

Note that currently, the first time you deselect a route, the client will stop picking up any new routes automatically until you run select all again, see #3611

<!-- gh-comment-id:2790461438 --> @nazarewk commented on GitHub (Apr 9, 2025): > Can I have a network route for a range such as 192.168.2.x/24 AND have clients with Netbird on them in that range and get p2p with the hosts that are on netbird direct? This would be possible on the backend if the IP addressing Posture Checks were available for free (you could filter out advertisements to the members of networks). For now, you can try to run a `netbird networks [ls|select|deselect]` based script periodically to reconcile the routes. Note that currently, the first time you `deselect` a route, the client will stop picking up any new routes automatically until you run `select all` again, see #3611
Author
Owner

@1nerdyguy commented on GitHub (Apr 9, 2025):

I did confirm that if I remove the route for that IP range, I get p2p connections again. It appears working as intended.

<!-- gh-comment-id:2790560028 --> @1nerdyguy commented on GitHub (Apr 9, 2025): I did confirm that if I remove the route for that IP range, I get p2p connections again. It appears working as intended.
Sign in to join this conversation.
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#7794