SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-05-14 12:12:27 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

questions: direct mode not working? #274

New Issue
Closed
opened 2025-11-20 05:08:58 -05:00 by saavagebueno · 7 comments
saavagebueno commented 2025-11-20 05:08:58 -05:00
Owner
Copy Link

Originally created by @tarasglek on GitHub (Feb 19, 2023).

Hi I just tried netbird for first time, excited to see kernel wireguard-level perf, only to realize it doesn't work for me.
Realized that kernel wireguard connections are being proxied via userspace. And iperf3 performance is 50% lower than that of tailscale.

Would appreciate answers on some questions.

i have 3 machines: lan1,lan2 and remote1. lan1 & lan2 are on same subnet. remote1 is behind nat on another isp. All are using 0.13.

lan1, remote1 are linux
lan2 is macos.

no direct mode on same lan between lan1, lan2. Why?

connections from lan1 to lan2, remote1, show up as

  Connection type: P2P
  Direct: false

This means https://github.com/netbirdio/netbird/pull/245 is not working.
Logs shows "connected to peer" instead of "directly connected". Why is direct mode not working on lan?

Wireshark shows that p2p udp connection is established between all nodes. Why not use direct mode in this case?

This is also interesting. If We are able to use a udp connection, why does netbird not setup wireguard direct mode using hole-punched addresses?

binary size question

If netbird is not doing userspace networking, why is the binary roughly same size as tailscaled? Is it possible to make a smaller binary? My usecase for netbird was going to be to use netbird on openwrt routers that don't have the space for tailscale, but seems that netbird is not a significant win here.

Originally created by @tarasglek on GitHub (Feb 19, 2023). Hi I just tried netbird for first time, excited to see kernel wireguard-level perf, only to realize it doesn't work for me. Realized that kernel wireguard connections are being proxied via userspace. And iperf3 performance is 50% lower than that of tailscale. Would appreciate answers on some questions. i have 3 machines: lan1,lan2 and remote1. lan1 & lan2 are on same subnet. remote1 is behind nat on another isp. All are using 0.13. lan1, remote1 are linux lan2 is macos. ### no direct mode on same lan between lan1, lan2. Why? connections from lan1 to lan2, remote1, show up as ``` Connection type: P2P Direct: false ``` This means https://github.com/netbirdio/netbird/pull/245 is not working. Logs shows "connected to peer" instead of "directly connected". Why is direct mode not working on lan? ### Wireshark shows that p2p udp connection is established between all nodes. Why not use direct mode in this case? This is also interesting. If We are able to use a udp connection, why does netbird not setup wireguard direct mode using hole-punched addresses? ### binary size question If netbird is not doing userspace networking, why is the binary roughly same size as tailscaled? Is it possible to make a smaller binary? My usecase for netbird was going to be to use netbird on openwrt routers that don't have the space for tailscale, but seems that netbird is not a significant win here.
saavagebueno commented 2025-11-20 05:08:58 -05:00
Author
Owner
Copy Link

@braginini commented on GitHub (Feb 19, 2023):

Hey @tarasglekk

Would you mind joining our Slack channel? We usually respond there quicker.

Sorry to hear that NetBird doesn't work as expected. And many thanks for the detailed explanation.
Let's clarify a few things first, maybe there is something related to the OS that prevents the direct mode.

  • What OS (with version) are you running NetBird on?
  • Do you have kernel WireGuard modules installed?
  • Could you please enable debug log and restart the agent?
    just run:
sudo netbird service stop
sudo netbird service uninstall
sudo netbird service install --log-level debug
sudo netbird service start

This is also interesting. If We are able to use a udp connection, why does netbird not setup wireguard direct mode using hole-punched addresses?

Local LAN peer connections should be direct unless there is something with the firewall. The debug log will help us figure it out.
When connecting remote peers it is not always possible to configure WireGuard directly because the hole punching logic has its own allocated port X while the Kernel WireGuard has its own port Y. Separate processes. The firewall hole is punched for the port X while the traffic going to WireGuard's port Y will be blocked by the firewall. Therefore, we proxy traffic from X to Y locally. It is possible to embed the hole punching logic into the userspace implementation of WireGuard making the real direct connections.
Nevertheless, we planned to improve the connectivity layer and make it possible even with the kernel WireGuard :) Stay tuned!

If netbird is not doing userspace networking, why is the binary roughly same size as tailscaled? Is it possible to make a smaller binary? My usecase for netbird was going to be to use netbird on openwrt routers that don't have the space for tailscale, but seems that netbird is not a significant win here.

NetBird uses both userspace and kernel WireGuard implementations. The userspace one is in the binary because it is not always possible to use kernel (different platforms). Did you try one of those for your OpenWRT devices? If yours isn't supported, let me know which one do you need excactly. We figure out the build.
https://github.com/messense/openwrt-netbird/releases

@braginini commented on GitHub (Feb 19, 2023): Hey @tarasglekk Would you mind joining our [Slack channel](https://join.slack.com/t/netbirdio/shared_invite/zt-vrahf41g-ik1v7fV8du6t0RwxSrJ96A)? We usually respond there quicker. Sorry to hear that NetBird doesn't work as expected. And many thanks for the detailed explanation. Let's clarify a few things first, maybe there is something related to the OS that prevents the direct mode. - What OS (with version) are you running NetBird on? - Do you have kernel WireGuard modules installed? - Could you please enable debug log and restart the agent? just run: ``` sudo netbird service stop sudo netbird service uninstall sudo netbird service install --log-level debug sudo netbird service start ``` > This is also interesting. If We are able to use a udp connection, why does netbird not setup wireguard direct mode using hole-punched addresses? Local LAN peer connections should be direct unless there is something with the firewall. The debug log will help us figure it out. When connecting remote peers it is not always possible to configure WireGuard directly because the hole punching logic has its own allocated port X while the Kernel WireGuard has its own port Y. Separate processes. The firewall hole is punched for the port X while the traffic going to WireGuard's port Y will be blocked by the firewall. Therefore, we proxy traffic from X to Y locally. It is possible to embed the hole punching logic into the userspace implementation of WireGuard making the real direct connections. Nevertheless, we planned to improve the connectivity layer and make it possible even with the kernel WireGuard :) Stay tuned! > If netbird is not doing userspace networking, why is the binary roughly same size as tailscaled? Is it possible to make a smaller binary? My usecase for netbird was going to be to use netbird on openwrt routers that don't have the space for tailscale, but seems that netbird is not a significant win here. NetBird uses both userspace and kernel WireGuard implementations. The userspace one is in the binary because it is not always possible to use kernel (different platforms). Did you try one of those for your OpenWRT devices? If yours isn't supported, let me know which one do you need excactly. We figure out the build. https://github.com/messense/openwrt-netbird/releases
saavagebueno commented 2025-11-20 05:08:59 -05:00
Author
Owner
Copy Link

@tarasglek commented on GitHub (Feb 19, 2023):

Here is some debug:

client.log
.2 is my linux, .141 is mac

interface: wt0
  public key: 970qF+Paug+V+51YTBYp3V1gP0dzvekw6vXZ5xHTH0Y=
  private key: (hidden)
  listening port: 51820

peer: fgIc3VwixSXLdfGND7QNlphpAAZ3TaWIJXbP0rtYKGk=
  endpoint: 127.0.0.1:32961
  allowed ips: 100.109.168.196/32
  latest handshake: 24 seconds ago
  transfer: 1008 B received, 548 B sent
  persistent keepalive: every 25 seconds

peer: L8RkcRaDlvr/xUX7WlwOz0YoSWMnIZIitA5WVViB3iI=
  endpoint: 127.0.0.1:56038
  allowed ips: 100.109.156.15/32
  latest handshake: 1 minute, 40 seconds ago
  transfer: 360 B received, 684 B sent
  persistent keepalive: every 25 seconds

uname -a
Linux leader-white 5.10.146 #0 SMP Fri Oct 7 23:34:56 2022 aarch64 GNU/Linux

wireguard module is loaded, tun is unloaded

using netbird_0.13.0_linux_mips_softfloat.deb

@tarasglek commented on GitHub (Feb 19, 2023): Here is some debug: [client.log](https://github.com/netbirdio/netbird/files/10777749/client.log) .2 is my linux, .141 is mac ``` interface: wt0 public key: 970qF+Paug+V+51YTBYp3V1gP0dzvekw6vXZ5xHTH0Y= private key: (hidden) listening port: 51820 peer: fgIc3VwixSXLdfGND7QNlphpAAZ3TaWIJXbP0rtYKGk= endpoint: 127.0.0.1:32961 allowed ips: 100.109.168.196/32 latest handshake: 24 seconds ago transfer: 1008 B received, 548 B sent persistent keepalive: every 25 seconds peer: L8RkcRaDlvr/xUX7WlwOz0YoSWMnIZIitA5WVViB3iI= endpoint: 127.0.0.1:56038 allowed ips: 100.109.156.15/32 latest handshake: 1 minute, 40 seconds ago transfer: 360 B received, 684 B sent persistent keepalive: every 25 seconds ``` uname -a Linux leader-white 5.10.146 #0 SMP Fri Oct 7 23:34:56 2022 aarch64 GNU/Linux wireguard module is loaded, tun is unloaded using [netbird_0.13.0_linux_mips_softfloat.deb](https://github.com/netbirdio/netbird/releases/download/v0.13.0/netbird_0.13.0_linux_mips_softfloat.deb)
saavagebueno commented 2025-11-20 05:08:59 -05:00
Author
Owner
Copy Link

@tarasglek commented on GitHub (Feb 20, 2023):

Not sure if you guys have seen this particular hack for kernel wireguard hole-punching: https://github.com/malcolmseyd/natpunch-go

@tarasglek commented on GitHub (Feb 20, 2023): Not sure if you guys have seen this particular hack for kernel wireguard hole-punching: https://github.com/malcolmseyd/natpunch-go
saavagebueno commented 2025-11-20 05:08:59 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Feb 21, 2023):

Hello @tarasglek, thanks for sharing natpunch-go, we will have a look.

regarding your issue, I notice that you have a br-lan interface that is probably being used as your main interface. This interface name is matched by our default interface blacklist and therefore your local address is not being used for discovery.

Can you please edit the file /etc/netbird/config.json and remove the pattern br- from it and restart the agent with:

netbird down
netbird up

After that you can run:

netbird status -d

If the connection is still not being set as direct, please share the logs again.

@mlsmaycon commented on GitHub (Feb 21, 2023): Hello @tarasglek, thanks for sharing natpunch-go, we will have a look. regarding your issue, I notice that you have a `br-lan` interface that is probably being used as your main interface. This interface name is matched by our default interface blacklist and therefore your local address is not being used for discovery. Can you please edit the file /etc/netbird/config.json and remove the pattern `br-` from it and restart the agent with: ``` netbird down netbird up ``` After that you can run: ``` netbird status -d ``` If the connection is still not being set as direct, please share the logs again.
saavagebueno commented 2025-11-20 05:08:59 -05:00
Author
Owner
Copy Link

@tarasglek commented on GitHub (Feb 21, 2023):

Wow, direct mode works amazing with this fix, almost 7x faster. Thank you!

Is there a wip branch for direct mode over udp-hole punch?

@tarasglek commented on GitHub (Feb 21, 2023): Wow, direct mode works amazing with this fix, almost 7x faster. Thank you! Is there a wip branch for direct mode over udp-hole punch?
saavagebueno commented 2025-11-20 05:08:59 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Feb 21, 2023):

That's great that it fixed the issue for you.

We can share the branch once we define if we will work on integrating that solution.

@mlsmaycon commented on GitHub (Feb 21, 2023): That's great that it fixed the issue for you. We can share the branch once we define if we will work on integrating that solution.
saavagebueno commented 2025-11-20 05:08:59 -05:00
Author
Owner
Copy Link

@braginini commented on GitHub (Feb 22, 2023):

Wow, direct mode works amazing with this fix, almost 7x faster. Thank you!

@tarasglek thats cool! 7x faster comparing to what setup? NetBird with the local proxy?

@braginini commented on GitHub (Feb 22, 2023): > Wow, direct mode works amazing with this fix, almost 7x faster. Thank you! @tarasglek thats cool! 7x faster comparing to what setup? NetBird with the local proxy?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#274
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?