SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-07-27 16:32:34 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

Linux peer cannot connect to other peers, for no apparent reason #766

New Issue
Closed
opened 2025-11-20 05:17:08 -05:00 by saavagebueno · 7 comments
saavagebueno commented 2025-11-20 05:17:08 -05:00
Owner
Copy Link

Originally created by @acarstoiu on GitHub (Apr 2, 2024).

Describe the problem

I'm using the free cloud plan of Netbird since before there were any paid plans and the serice was in its early days. Never had any connection problems with the other peers, in spite of the fact that there's no package for my distribution, Gentoo.
But after upgrading from 0.25.3 to 0.26.7 my peer is unable to connect to other peers. Initially the service complained in its log file about being unable to instantiate an "eBPF proxy" using the kernel or some user space replacement. I've read this article and configured my kernel according to the needs, which made the log file look like this (log level set to trace):

2024-04-01T19:32:45+03:00 INFO client/cmd/service_controller.go:24: starting Netbird service
2024-04-01T19:32:45+03:00 TRAC client/server/server.go:184: running client connection
2024-04-01T19:32:45+03:00 INFO client/cmd/service_controller.go:64: started daemon server: /var/run/netbird.sock
2024-04-01T19:32:45+03:00 INFO client/internal/connect.go:96: starting NetBird client version 0.26.7
2024-04-01T19:32:45+03:00 DEBG client/internal/connect.go:157: connecting to the Management service api.netbird.io:443
2024-04-01T19:32:46+03:00 DEBG client/internal/connect.go:165: connected to the Management service api.netbird.io:443
2024-04-01T19:32:47+03:00 DEBG signal/client/grpc.go:92: connected to Signal Service: signal.netbird.io:443
2024-04-01T19:32:47+03:00 DEBG client/internal/wgproxy/proxy_ebpf.go:37: instantiate ebpf proxy
2024-04-01T19:32:47+03:00 DEBG client/internal/ebpf/ebpf/wg_proxy_linux.go:11: load ebpf WG proxy
2024-04-01T19:32:47+03:00 INFO client/internal/wgproxy/proxy_ebpf.go:80: local wg proxy listening on: 3128

No sign of any malfunction, as you can see. Any ideas ❓

To Reproduce

Simply start the netbird service and notice that the local peer is not connected, both according to the GUI app and to the admin page at https://app.netbird.io/peers (which shows another peer as being online).

Expected behavior

To get connected to other peers.

Are you using NetBird Cloud?

Yes..

NetBird version

See below.

NetBird status -d output (redacted):

# netbird status -d
Peers detail:
Daemon version: 0.26.7
CLI version: 0.26.7
Management: Connected to https://api.netbird.io:443
Signal: Connected to https://signal.netbird.io:443
Relays:
Nameservers:
FQDN: {xxxx}.netbird.cloud
NetBird IP: 100.67.{xx}.{xx}/16
Interface type: Kernel
Quantum resistance: false
Routes: -
Peers count: 0/0 Connected
Originally created by @acarstoiu on GitHub (Apr 2, 2024). **Describe the problem** I'm using the free **cloud** plan of Netbird since before there were any paid plans and the serice was in its early days. Never had any connection problems with the other peers, in spite of the fact that there's no package for my distribution, [Gentoo](https://www.gentoo.org/). But after upgrading from _0.25.3_ to _0.26.7_ my peer is unable to connect to other peers. Initially the service complained in its log file about being unable to instantiate an "eBPF proxy" using the kernel or some user space replacement. I've read [this article](https://netbird.io/knowledge-hub/using-xdp-ebpf-to-share-default-dns-port-between-resolvers) and configured my kernel according to the needs, which made the log file look like this (log level set to `trace`): ``` 2024-04-01T19:32:45+03:00 INFO client/cmd/service_controller.go:24: starting Netbird service 2024-04-01T19:32:45+03:00 TRAC client/server/server.go:184: running client connection 2024-04-01T19:32:45+03:00 INFO client/cmd/service_controller.go:64: started daemon server: /var/run/netbird.sock 2024-04-01T19:32:45+03:00 INFO client/internal/connect.go:96: starting NetBird client version 0.26.7 2024-04-01T19:32:45+03:00 DEBG client/internal/connect.go:157: connecting to the Management service api.netbird.io:443 2024-04-01T19:32:46+03:00 DEBG client/internal/connect.go:165: connected to the Management service api.netbird.io:443 2024-04-01T19:32:47+03:00 DEBG signal/client/grpc.go:92: connected to Signal Service: signal.netbird.io:443 2024-04-01T19:32:47+03:00 DEBG client/internal/wgproxy/proxy_ebpf.go:37: instantiate ebpf proxy 2024-04-01T19:32:47+03:00 DEBG client/internal/ebpf/ebpf/wg_proxy_linux.go:11: load ebpf WG proxy 2024-04-01T19:32:47+03:00 INFO client/internal/wgproxy/proxy_ebpf.go:80: local wg proxy listening on: 3128 ``` No sign of any malfunction, as you can see. Any ideas ❓ **To Reproduce** Simply start the _netbird_ service and notice that the local peer is **not** connected, both according to the GUI app and to the admin page at https://app.netbird.io/peers (which shows another peer as being online). **Expected behavior** To get connected to other peers. **Are you using NetBird Cloud?** Yes.. **NetBird version** See below. **NetBird status -d output (redacted):** ``` # netbird status -d Peers detail: Daemon version: 0.26.7 CLI version: 0.26.7 Management: Connected to https://api.netbird.io:443 Signal: Connected to https://signal.netbird.io:443 Relays: Nameservers: FQDN: {xxxx}.netbird.cloud NetBird IP: 100.67.{xx}.{xx}/16 Interface type: Kernel Quantum resistance: false Routes: - Peers count: 0/0 Connected ```
saavagebueno added the clientcloud labels 2025-11-20 05:17:08 -05:00
saavagebueno commented 2025-11-20 05:17:09 -05:00
Author
Owner
Copy Link

@acarstoiu commented on GitHub (Apr 3, 2024):

✅ Downgrading to 0.25.3 makes things work again.

@acarstoiu commented on GitHub (Apr 3, 2024): ✅ Downgrading to _0.25.3_ makes things work again.
saavagebueno commented 2025-11-20 05:17:09 -05:00
Author
Owner
Copy Link

@bcmmbaga commented on GitHub (Apr 3, 2024):

Hello! Could you please provide the logs from your connection attempt using version 0.26.7? Additionally, could you share the output of netbird status -d?

@bcmmbaga commented on GitHub (Apr 3, 2024): Hello! Could you please provide the logs from your connection attempt using version 0.26.7? Additionally, could you share the output of `netbird status -d`?
saavagebueno commented 2025-11-20 05:17:09 -05:00
Author
Owner
Copy Link

@acarstoiu commented on GitHub (Apr 3, 2024):

Already did that, read above again.

@acarstoiu commented on GitHub (Apr 3, 2024): Already did that, [read above](https://github.com/netbirdio/netbird/issues/1787#issue-2221438612) again.
saavagebueno commented 2025-11-20 05:17:09 -05:00
Author
Owner
Copy Link

@bcmmbaga commented on GitHub (Apr 4, 2024):

Can you test version 0.27.0 to see if it fixes the issue?

@bcmmbaga commented on GitHub (Apr 4, 2024): Can you test version 0.27.0 to see if it fixes the issue?
saavagebueno commented 2025-11-20 05:17:10 -05:00
Author
Owner
Copy Link

@acarstoiu commented on GitHub (Apr 4, 2024):

Luckily, this brand new version works. Still, there are a few things in the log file that I do not understand:

2024-04-04T18:29:10+03:00 DEBG client/firewall/create_linux.go:51: creating an nftables firewall manager
2024-04-04T18:29:10+03:00 WARN client/firewall/nftables/route_linux.go:69: table 'filter' not found for forward rules
2024-04-04T18:29:10+03:00 DEBG iface/tun_kernel_linux.go:110: bringing up interface: wt0
2024-04-04T18:29:10+03:00 DEBG iface/tun_kernel_linux.go:130: device is ready to use: wt0
2024-04-04T18:29:10+03:00 INFO client/internal/dns/host_linux.go:68: System DNS manager discovered: file
...
2024-04-04T18:29:11+03:00 DEBG client/internal/dns/service_listener.go:69: starting dns on 100.67.198.217:53
2024-04-04T18:29:11+03:00 ERRO client/internal/dns/server.go:323: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured
  1. What's with the table 'filter' not found for forward rules ?
  2. What's a nameserver group with all domains configured ?
    My /etc/resolv.conf is managed by dhcpcd.

Thank you.

@acarstoiu commented on GitHub (Apr 4, 2024): Luckily, this brand new version **works**. Still, there are a few things in the log file that I do not understand: ``` 2024-04-04T18:29:10+03:00 DEBG client/firewall/create_linux.go:51: creating an nftables firewall manager 2024-04-04T18:29:10+03:00 WARN client/firewall/nftables/route_linux.go:69: table 'filter' not found for forward rules 2024-04-04T18:29:10+03:00 DEBG iface/tun_kernel_linux.go:110: bringing up interface: wt0 2024-04-04T18:29:10+03:00 DEBG iface/tun_kernel_linux.go:130: device is ready to use: wt0 2024-04-04T18:29:10+03:00 INFO client/internal/dns/host_linux.go:68: System DNS manager discovered: file ... 2024-04-04T18:29:11+03:00 DEBG client/internal/dns/service_listener.go:69: starting dns on 100.67.198.217:53 2024-04-04T18:29:11+03:00 ERRO client/internal/dns/server.go:323: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured ``` 1. What's with the `table 'filter' not found for forward rules` ? 2. What's `a nameserver group with all domains configured` ? My `/etc/resolv.conf` is managed by _dhcpcd_. Thank you.
saavagebueno commented 2025-11-20 05:17:10 -05:00
Author
Owner
Copy Link

@wbarnard81 commented on GitHub (Nov 18, 2024):

I know this topic has been open for a while, but I would like to note that I am experiencing the same. So far I have only noticed this on my exit note.

netstat when I know it is not working.

OS: linux/amd64
Daemon version: 0.31.1
CLI version: 0.31.1
Management: Connected to https://netbird.mydomain.co.za:33073
Signal: Connected to http://netbird.mydomain.co.za:10000
Relays:
  [stun:netbird.mydomain.co.za:3478] is Available
  [turn:netbird.mydomain.co.za:3478?transport=udp] is Available
  [rel://netbird.mydomain.co.za:33080] is Available
Nameservers:
  [1.1.1.1:53, 1.0.0.1:53] for [.] is Available
FQDN: netbird-exit-node.netbird.local
NetBird IP: 100.84.169.9/16
Interface type: Kernel
Quantum resistance: false
Routes: 102.xxx.xxx.0/23, mydomain.co.za
Peers count: 0/9 Connected

netbird service restart

Daemon version: 0.31.1
CLI version: 0.31.1
Management: Connected to https://netbird.mydomain.co.za:33073
Signal: Connected to http://netbird.mydomain.co.za:10000
Relays:
  [rel://netbird.mydomain.co.za:33080] is Available
Nameservers:
  [1.1.1.1:53, 1.0.0.1:53] for [.] is Available
FQDN: netbird-exit-node.netbird.local
NetBird IP: 100.84.169.9/16
Interface type: Kernel
Quantum resistance: false
Routes: 102.xxx.xxx.0/23, mydomain.co.za
Peers count: 0/10 Connected

I also noticed a new version was out, so I updated and restarted and now the exit node is working again.

Daemon version: 0.32.0
CLI version: 0.32.0
Management: Connected to https://netbird.mydomain:33073
Signal: Connected to http://netbird.mydomain:10000
Relays:
  [stun:netbird.mydomain:3478] is Available
  [turn:netbird.mydomain:3478?transport=udp] is Available
  [rel://netbird.mydomain:33080] is Available
Nameservers:
  [1.1.1.1:53, 1.0.0.1:53] for [.] is Available
FQDN: netbird-exit-node.netbird.local
NetBird IP: 100.84.169.9/16
Interface type: Kernel
Quantum resistance: false
Routes: 102.xxx.xxx.0/23, mydomain
Peers count: 1/11 Connected

I suspect it will be down again tomorrow, so will report back then.

@wbarnard81 commented on GitHub (Nov 18, 2024): I know this topic has been open for a while, but I would like to note that I am experiencing the same. So far I have only noticed this on my exit note. netstat when I know it is not working. ``` OS: linux/amd64 Daemon version: 0.31.1 CLI version: 0.31.1 Management: Connected to https://netbird.mydomain.co.za:33073 Signal: Connected to http://netbird.mydomain.co.za:10000 Relays: [stun:netbird.mydomain.co.za:3478] is Available [turn:netbird.mydomain.co.za:3478?transport=udp] is Available [rel://netbird.mydomain.co.za:33080] is Available Nameservers: [1.1.1.1:53, 1.0.0.1:53] for [.] is Available FQDN: netbird-exit-node.netbird.local NetBird IP: 100.84.169.9/16 Interface type: Kernel Quantum resistance: false Routes: 102.xxx.xxx.0/23, mydomain.co.za Peers count: 0/9 Connected ``` netbird service restart ```OS: linux/amd64 Daemon version: 0.31.1 CLI version: 0.31.1 Management: Connected to https://netbird.mydomain.co.za:33073 Signal: Connected to http://netbird.mydomain.co.za:10000 Relays: [rel://netbird.mydomain.co.za:33080] is Available Nameservers: [1.1.1.1:53, 1.0.0.1:53] for [.] is Available FQDN: netbird-exit-node.netbird.local NetBird IP: 100.84.169.9/16 Interface type: Kernel Quantum resistance: false Routes: 102.xxx.xxx.0/23, mydomain.co.za Peers count: 0/10 Connected ``` I also noticed a new version was out, so I updated and restarted and now the exit node is working again. ```OS: linux/amd64 Daemon version: 0.32.0 CLI version: 0.32.0 Management: Connected to https://netbird.mydomain:33073 Signal: Connected to http://netbird.mydomain:10000 Relays: [stun:netbird.mydomain:3478] is Available [turn:netbird.mydomain:3478?transport=udp] is Available [rel://netbird.mydomain:33080] is Available Nameservers: [1.1.1.1:53, 1.0.0.1:53] for [.] is Available FQDN: netbird-exit-node.netbird.local NetBird IP: 100.84.169.9/16 Interface type: Kernel Quantum resistance: false Routes: 102.xxx.xxx.0/23, mydomain Peers count: 1/11 Connected ``` I suspect it will be down again tomorrow, so will report back then.
saavagebueno commented 2025-11-20 05:17:10 -05:00
Author
Owner
Copy Link

@wbarnard81 commented on GitHub (Nov 20, 2024):

So far, it seems switching to v0.32.0 has solved my problem.

@wbarnard81 commented on GitHub (Nov 20, 2024): So far, it seems switching to v0.32.0 has solved my problem.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label client cloud
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#766
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?