[GH-ISSUE #3328] Connection issues during PC bootup #7162

Open
opened 2026-08-05 01:11:50 -04:00 by saavagebueno · 10 comments
Owner

Originally created by @mawanasad on GitHub (Feb 15, 2025).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/3328

Originally assigned to: @pappz on GitHub.

Describe the problem

I am connected to a Linux exit node on my Windows machine. I have enabled Netbird to start on system startup. Every time I boot up the machine, internet breaks, and I am unable to ping Netbird IP of my exit node. I have to disconnect exit node, disconnect Netbird, connect it back, and then enable exit node to make my internet work through the exit node

This is a repeated behavior that I have to do every time. Not been a single time when it worked straight from the get go after turning my PC on

Is there a solution? What can I provide to further debug?

To Reproduce

Steps to reproduce the behavior:

  1. Connect to an exit node
  2. Turn off your PC
  3. Turn it back on
  4. Check internet status. It does not work.
Originally created by @mawanasad on GitHub (Feb 15, 2025). Original GitHub issue: https://github.com/netbirdio/netbird/issues/3328 Originally assigned to: @pappz on GitHub. **Describe the problem** I am connected to a Linux exit node on my Windows machine. I have enabled Netbird to start on system startup. Every time I boot up the machine, internet breaks, and I am unable to ping Netbird IP of my exit node. I have to disconnect exit node, disconnect Netbird, connect it back, and then enable exit node to make my internet work through the exit node **_This is a repeated behavior that I have to do every time. Not been a single time when it worked straight from the get go after turning my PC on_** Is there a solution? What can I provide to further debug? **To Reproduce** Steps to reproduce the behavior: 1. Connect to an exit node 2. Turn off your PC 3. Turn it back on 4. Check internet status. It does not work.
saavagebueno added the triage-needed label 2026-08-05 01:11:50 -04:00
Author
Owner

@pappz commented on GitHub (Mar 9, 2025):

Hello!

Could you send me the log bundle package?
Open a PowerShell terminal and apply these commands:

  1. Enable debug log level in a privileged PowerShell terminal
[Environment]::SetEnvironmentVariable("NB_LOG_LEVEL", "debug", "Machine")
  1. Restart your machine to reproduce the issue
  2. Collect the logs with the debug command
netbird debug bundle -AS
<!-- gh-comment-id:2708982879 --> @pappz commented on GitHub (Mar 9, 2025): Hello! Could you send me the log bundle package? Open a PowerShell terminal and apply these commands: 1. Enable debug log level in a privileged PowerShell terminal ``` [Environment]::SetEnvironmentVariable("NB_LOG_LEVEL", "debug", "Machine") ``` 2. Restart your machine to reproduce the issue 3. Collect the logs with the debug command ``` netbird debug bundle -AS ```
Author
Owner

@gamerslouis commented on GitHub (Jul 2, 2025):

Hi,

Are there any updates?
We’re also encountering this issue when deploying Netbird on an Ubuntu server and rebooting.

Even after flushing nftable ip netbird and deleting the IP rule for Netbird, the problem still persists.

The behavior we’ve observed is:
ARP works normally, but we’re unable to access the external network.
Additionally, we can ping 127.0.0.1, but we can’t ping any of the IPs bound to other interfaces

Currently, the only way to restore the network is by restarting the Netbird service.

<!-- gh-comment-id:3027139802 --> @gamerslouis commented on GitHub (Jul 2, 2025): Hi, Are there any updates? We’re also encountering this issue when deploying Netbird on an Ubuntu server and rebooting. Even after flushing nftable ip netbird and deleting the IP rule for Netbird, the problem still persists. The behavior we’ve observed is: ARP works normally, but we’re unable to access the external network. Additionally, we can ping 127.0.0.1, but we can’t ping any of the IPs bound to other interfaces Currently, the only way to restore the network is by restarting the Netbird service.
Author
Owner

@nazarewk commented on GitHub (Jul 2, 2025):

@gamerslouis could you do the following?

  1. add below to /etc/sysconfig/netbird,

    NB_LOG_LEVEL=trace
    NB_LOG_MAX_SIZE_MB=50
    
  2. reboot the machine

  3. upload a debug bundle netbird debug bundle -SU

  4. restart the service once to make it work again

  5. upload a debug bundle netbird debug bundle -SU

  6. (optionally) remove/clear/comment out /etc/sysconfig/netbird

it will help us identify a problem in your specific case

<!-- gh-comment-id:3027157271 --> @nazarewk commented on GitHub (Jul 2, 2025): @gamerslouis could you do the following? 1. add below to `/etc/sysconfig/netbird`, NB_LOG_LEVEL=trace NB_LOG_MAX_SIZE_MB=50 2. reboot the machine 3. upload a debug bundle `netbird debug bundle -SU` 4. restart the service once to make it work again 5. upload a debug bundle `netbird debug bundle -SU` 6. (optionally) remove/clear/comment out `/etc/sysconfig/netbird` it will help us identify a problem in your specific case
Author
Owner

@gamerslouis commented on GitHub (Jul 2, 2025):

Hi,
We have identified the issue as being related to the IP rule for advanced routing.
When we delete the IP rule "100: from all lookup main suppress_prefixlength 0", the network returns to normal.
OS: Ubuntu 24.04.2, kernel version 6.8.0-58-generic.

<!-- gh-comment-id:3029994218 --> @gamerslouis commented on GitHub (Jul 2, 2025): Hi, We have identified the issue as being related to the IP rule for advanced routing. When we delete the IP rule "100: from all lookup main suppress_prefixlength 0", the network returns to normal. OS: Ubuntu 24.04.2, kernel version 6.8.0-58-generic.
Author
Owner

@gamerslouis commented on GitHub (Jul 4, 2025):

We found that this is an known issue since we use Cilium in our environment.
So the problem is still exists with the new versions of Cilium and netbird

<!-- gh-comment-id:3034663026 --> @gamerslouis commented on GitHub (Jul 4, 2025): We found that this is an known issue since we use Cilium in our environment. So the problem is still exists with the new versions of Cilium and netbird
Author
Owner

@nazarewk commented on GitHub (Jul 4, 2025):

@gamerslouis could you please open another bug report about your findings?

<!-- gh-comment-id:3035276847 --> @nazarewk commented on GitHub (Jul 4, 2025): @gamerslouis could you please open another bug report about your findings?
Author
Owner

@codablock commented on GitHub (Aug 4, 2025):

This seems to be the same issue as reported here: https://github.com/netbirdio/netbird/issues/2926. I'm also running into it right now.

The suggested fix is to apply https://gist.github.com/dkrhodes/040778116eedafe857ddc34a1ccdcfec and compile + run your own build of Netbird. I wonder if this patch has ever been tried to get upstream? If not, I can create a PR with that patch cherry-picked.

<!-- gh-comment-id:3149439171 --> @codablock commented on GitHub (Aug 4, 2025): This seems to be the same issue as reported here: https://github.com/netbirdio/netbird/issues/2926. I'm also running into it right now. The suggested fix is to apply https://gist.github.com/dkrhodes/040778116eedafe857ddc34a1ccdcfec and compile + run your own build of Netbird. I wonder if this patch has ever been tried to get upstream? If not, I can create a PR with that patch cherry-picked.
Author
Owner

@lixmal commented on GitHub (Aug 4, 2025):

@codablock what is your ip rules? I'd like to see what cilium puts there. This doesn't seem to happen on a default cilium installation(?)

<!-- gh-comment-id:3149562208 --> @lixmal commented on GitHub (Aug 4, 2025): @codablock what is your `ip rule`s? I'd like to see what cilium puts there. This doesn't seem to happen on a default cilium installation(?)
Author
Owner

@codablock commented on GitHub (Aug 4, 2025):

First, it looks like I mixed up issues here. I meant to post my comment to a different issue but somehow got into the wrong tab 🙈 But now I can't find the other issue anymore where I wanted to post this...sorry about the confusion. We can continue this discussion in https://github.com/netbirdio/netbird/issues/2926 if you want.

@lixmal This is what I got right now on a machine where I deleted the netbird rules to get it work for me:

$ ip rule
9:	from all fwmark 0x200/0xf00 lookup 2004
100:	from all lookup local
110:	not from all fwmark 0x1bd00 lookup netbird
32766:	from all lookup main
32767:	from all lookup default
$ ip -6 rule
0:	from all lookup local
110:	not from all fwmark 0x1bd00 lookup netbird
32766:	from all lookup main

I assume the 100: from all lookup local is the conflicting one, but I'm not familiar enough about this part of linux networking to be confident enough.

<!-- gh-comment-id:3149636742 --> @codablock commented on GitHub (Aug 4, 2025): First, it looks like I mixed up issues here. I meant to post my comment to a different issue but somehow got into the wrong tab 🙈 But now I can't find the other issue anymore where I wanted to post this...sorry about the confusion. We can continue this discussion in https://github.com/netbirdio/netbird/issues/2926 if you want. @lixmal This is what I got right now on a machine where I deleted the netbird rules to get it work for me: ```shell $ ip rule 9: from all fwmark 0x200/0xf00 lookup 2004 100: from all lookup local 110: not from all fwmark 0x1bd00 lookup netbird 32766: from all lookup main 32767: from all lookup default $ ip -6 rule 0: from all lookup local 110: not from all fwmark 0x1bd00 lookup netbird 32766: from all lookup main ``` I assume the `100: from all lookup local` is the conflicting one, but I'm not familiar enough about this part of linux networking to be confident enough.
Author
Owner

@gamerslouis commented on GitHub (Aug 22, 2025):

#4273 Fix this problem. It works in my env with netbird 0.54.2

<!-- gh-comment-id:3213358913 --> @gamerslouis commented on GitHub (Aug 22, 2025): #4273 Fix this problem. It works in my env with netbird 0.54.2
Sign in to join this conversation.
No Label triage-needed
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#7162