[GH-ISSUE #2799] User space wireguard not working? #5898

Open
opened 2026-08-05 01:05:43 -04:00 by saavagebueno · 0 comments
Owner

Originally created by @Silex on GitHub (Oct 28, 2024).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/2799

First of all, thanks a lot for this awesome software! We are in the process of migrating everything to netbird and so far the experience is great.

I already succeed in running netbird on Teltonika routers, and now I'm trying to run netbird 0.28.4 on AXIS cameras. The goal is to be able to connect to the cameras (HTTP/RTSP) from other peers.

The problem is that the user running the application (ACAP) is very limited, but someone managed to run tailscale on them (https://github.com/Mo3he/Axis_Cam_Tailscale), so I'm pretty sure it'd be feasible to run netbird too.

Apparently they do it with --tun=userspace-networking when using tailscale (https://github.com/Mo3he/Axis_Cam_Tailscale/blob/main/aarch64/app/Tailscale_VPN).

So, I made a script like this:

#!/bin/sh

PACKAGE_DIR='/usr/local/packages/netbird'
TMP_DIR="$PACKAGE_DIR/tmp"

ARGS="--config $TMP_DIR/config.json --log-file $TMP_DIR/client.log --daemon-addr unix://$TMP_DIR/netbird.sock"
NETBIRD="$PACKAGE_DIR/lib/netbird $ARGS"

export NB_WG_KERNEL_DISABLED=true
# export NB_USE_NETSTACK_MODE=true
# export NB_SOCKS5_LISTENER_PORT=30000

echo '----------------------------------------'
echo 'Starting netbird'
mkdir -p $TMP_DIR
$NETBIRD service run &

echo 'Waiting 5 seconds'
sleep 5

echo 'Joining network'
$NETBIRD up --management-url https://netbird.foo.com --setup-key 123123123

echo 'Logs'
tail -f $TMP_DIR/client.log

This script runs fine as root (I can access my cameras), but when I run it as the application user (which will is the non-negociable default in latest firmares) then it outputs this:

2024-10-28T15:59:04.396+01:00 axis-accc8ede87b2 [ INFO    ] netbird[14426]: Starting netbird
2024-10-28T15:59:04.398+01:00 axis-accc8ede87b2 [ INFO    ] netbird[14426]: Waiting 5 seconds
2024-10-28T15:59:09.397+01:00 axis-accc8ede87b2 [ INFO    ] netbird[14426]: Joining network
2024-10-28T15:59:12.398+01:00 axis-accc8ede87b2 [ INFO    ] netbird[14493]: Connected
2024-10-28T15:59:12.406+01:00 axis-accc8ede87b2 [ INFO    ] netbird[14426]: Waiting 5 seconds
2024-10-28T15:59:17.476+01:00 axis-accc8ede87b2 [ INFO    ] netbird[14863]: Error: status failed: create wg interface: operation not permitted

And while the peer shows up in the netbird ui, of course I cannot connect to it. So next if I uncomment the line with NB_USE_NETSTACK_MODE, then it connects fine and I can even ping the camera, but I cannot remotely access it.

With NB_USE_NETSTACK_MODE here is what it status prints:

stvs@axis-b8a44fb94617:/tmp/netbird/lib$ ./netbird $ARGS status
OS: linux/arm64
Daemon version: 0.28.4
CLI version: 0.28.4
Management: Connected
Signal: Connected
Relays: 2/2 Available
Nameservers: 0/0 Available
FQDN: (retracted)
NetBird IP: 100.82.41.54/16
Interface type: Userspace
Quantum resistance: false
Routes: -
Peers count: 5/9 Connected

Here's what the client.log show:

2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/cmd/service_controller.go:64: started daemon server: /usr/local/packages/netbird/tmp/netbird.sock
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:257: generated new SSH key
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:273: using default Wireguard port 51820
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:284: using default Wireguard interface wt0
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:337: filling in interface blacklist with defaults: [ wt0 wt utun tun0 zt ZeroTier wg ts Tailscale tailscale docker veth br- lo ]
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:383: using default DNS route interval 1m0s
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:02+01:00 INFO client/internal/config.go:216: new Management URL provided, updated to "https://netbird.foo.com" (old value "https://api.netbird.io:443")
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:03+01:00 WARN client/server/server.go:259: failed login: rpc error: code = InvalidArgument desc = invalid setup-key or no sso information provided, err: invalid UUID length: 0
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:05+01:00 INFO client/internal/connect.go:119: starting NetBird client version 0.28.4 on linux/arm
2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:05+01:00 INFO client/internal/login.go:130: peer has been successfully registered on Management Service
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 ERRO iface/uapi.go:15: failed to open uapi socket: remove /var/run/wireguard/wt0.sock: no such file or directory
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 ERRO iface/wg_configurer_usp.go:191: failed to open uapi listener: remove /var/run/wireguard/wt0.sock: no such file or directory
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/firewall/create_linux.go:58: no firewall manager found, trying to use userspace packet filtering firewall
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/internal/connect.go:267: Netbird engine started, the IP is: 100.82.116.219/16
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/internal/dns/host_unix.go:68: System DNS manager discovered: file
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/internal/engine.go:1479: Network monitor is disabled, not starting
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO iface/tun_netstack.go:46: create netstack tun interface
2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO signal/client/grpc.go:158: connected to the Signal Service stream
2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:07+01:00 ERRO client/internal/dns/server.go:322: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured
2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:07+01:00 INFO client/internal/acl/manager.go:52: ACL rules processed in: 208.831µs, total rules count: 2
2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:07+01:00 INFO management/client/grpc.go:164: connected to the Management Service stream
2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO    ] netbird[23262]: 2024-10-28T16:35:07+01:00 WARN client/internal/engine.go:587: running SSH server is not permitted

Can you clarify:

  1. Wether NB_USE_NETSTACK_MODE is actually a dead end and won't help me connect from outside to my camera.
  2. What rights "userspace wireguard" requires so I can ask AXIS about what I should do.
Originally created by @Silex on GitHub (Oct 28, 2024). Original GitHub issue: https://github.com/netbirdio/netbird/issues/2799 First of all, thanks a lot for this awesome software! We are in the process of migrating everything to netbird and so far the experience is great. I already succeed in running netbird on Teltonika routers, and now I'm trying to run netbird 0.28.4 on AXIS cameras. The goal is to be able to connect to the cameras (HTTP/RTSP) from other peers. The problem is that the user running the application (ACAP) is very limited, but someone managed to run tailscale on them (https://github.com/Mo3he/Axis_Cam_Tailscale), so I'm pretty sure it'd be feasible to run netbird too. Apparently they do it with `--tun=userspace-networking` when using tailscale (https://github.com/Mo3he/Axis_Cam_Tailscale/blob/main/aarch64/app/Tailscale_VPN). So, I made a script like this: ``` bash #!/bin/sh PACKAGE_DIR='/usr/local/packages/netbird' TMP_DIR="$PACKAGE_DIR/tmp" ARGS="--config $TMP_DIR/config.json --log-file $TMP_DIR/client.log --daemon-addr unix://$TMP_DIR/netbird.sock" NETBIRD="$PACKAGE_DIR/lib/netbird $ARGS" export NB_WG_KERNEL_DISABLED=true # export NB_USE_NETSTACK_MODE=true # export NB_SOCKS5_LISTENER_PORT=30000 echo '----------------------------------------' echo 'Starting netbird' mkdir -p $TMP_DIR $NETBIRD service run & echo 'Waiting 5 seconds' sleep 5 echo 'Joining network' $NETBIRD up --management-url https://netbird.foo.com --setup-key 123123123 echo 'Logs' tail -f $TMP_DIR/client.log ``` This script runs fine as root (I can access my cameras), but when I run it as the application user (which will is the non-negociable default in latest firmares) then it outputs this: ``` 2024-10-28T15:59:04.396+01:00 axis-accc8ede87b2 [ INFO ] netbird[14426]: Starting netbird 2024-10-28T15:59:04.398+01:00 axis-accc8ede87b2 [ INFO ] netbird[14426]: Waiting 5 seconds 2024-10-28T15:59:09.397+01:00 axis-accc8ede87b2 [ INFO ] netbird[14426]: Joining network 2024-10-28T15:59:12.398+01:00 axis-accc8ede87b2 [ INFO ] netbird[14493]: Connected 2024-10-28T15:59:12.406+01:00 axis-accc8ede87b2 [ INFO ] netbird[14426]: Waiting 5 seconds 2024-10-28T15:59:17.476+01:00 axis-accc8ede87b2 [ INFO ] netbird[14863]: Error: status failed: create wg interface: operation not permitted ``` And while the peer shows up in the netbird ui, of course I cannot connect to it. So next if I uncomment the line with `NB_USE_NETSTACK_MODE`, then it connects fine and I can even ping the camera, but I cannot remotely access it. With `NB_USE_NETSTACK_MODE` here is what it status prints: ``` shell stvs@axis-b8a44fb94617:/tmp/netbird/lib$ ./netbird $ARGS status OS: linux/arm64 Daemon version: 0.28.4 CLI version: 0.28.4 Management: Connected Signal: Connected Relays: 2/2 Available Nameservers: 0/0 Available FQDN: (retracted) NetBird IP: 100.82.41.54/16 Interface type: Userspace Quantum resistance: false Routes: - Peers count: 5/9 Connected ``` Here's what the client.log show: ``` 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/cmd/service_controller.go:64: started daemon server: /usr/local/packages/netbird/tmp/netbird.sock 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:257: generated new SSH key 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:273: using default Wireguard port 51820 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:284: using default Wireguard interface wt0 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:337: filling in interface blacklist with defaults: [ wt0 wt utun tun0 zt ZeroTier wg ts Tailscale tailscale docker veth br- lo ] 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:34:57+01:00 INFO client/internal/config.go:383: using default DNS route interval 1m0s 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:02+01:00 INFO client/internal/config.go:216: new Management URL provided, updated to "https://netbird.foo.com" (old value "https://api.netbird.io:443") 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:03+01:00 WARN client/server/server.go:259: failed login: rpc error: code = InvalidArgument desc = invalid setup-key or no sso information provided, err: invalid UUID length: 0 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:05+01:00 INFO client/internal/connect.go:119: starting NetBird client version 0.28.4 on linux/arm 2024-10-28T16:35:05.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:05+01:00 INFO client/internal/login.go:130: peer has been successfully registered on Management Service 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 ERRO iface/uapi.go:15: failed to open uapi socket: remove /var/run/wireguard/wt0.sock: no such file or directory 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 ERRO iface/wg_configurer_usp.go:191: failed to open uapi listener: remove /var/run/wireguard/wt0.sock: no such file or directory 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/firewall/create_linux.go:58: no firewall manager found, trying to use userspace packet filtering firewall 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/internal/connect.go:267: Netbird engine started, the IP is: 100.82.116.219/16 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/internal/dns/host_unix.go:68: System DNS manager discovered: file 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO client/internal/engine.go:1479: Network monitor is disabled, not starting 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO iface/tun_netstack.go:46: create netstack tun interface 2024-10-28T16:35:07.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:06+01:00 INFO signal/client/grpc.go:158: connected to the Signal Service stream 2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:07+01:00 ERRO client/internal/dns/server.go:322: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured 2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:07+01:00 INFO client/internal/acl/manager.go:52: ACL rules processed in: 208.831µs, total rules count: 2 2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:07+01:00 INFO management/client/grpc.go:164: connected to the Management Service stream 2024-10-28T16:35:08.314+01:00 axis-accc8ede87b2 [ INFO ] netbird[23262]: 2024-10-28T16:35:07+01:00 WARN client/internal/engine.go:587: running SSH server is not permitted ``` Can you clarify: 1) Wether `NB_USE_NETSTACK_MODE` is actually a dead end and won't help me connect from outside to my camera. 2) What rights "userspace wireguard" requires so I can ask AXIS about what I should do.
saavagebueno added the triage-needed label 2026-08-05 01:05:43 -04:00
Sign in to join this conversation.
No Label triage-needed
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#5898