Routes not being created #857

Closed
opened 2025-11-20 05:18:44 -05:00 by saavagebueno · 18 comments
Owner

Originally created by @Icare-github on GitHub (May 7, 2024).

Describe the problem

Once connected, routes aren't being created on workstation to different environments.

Expected behavior

All routes created, once connected.

NetBird version

Netbird v0.27.4

vpn-2.netbird.selfhosted:
NetBird IP: [redacted]
Public key: [redacted]
Status: Connected
-- detail --
Connection type: P2P
Direct: true
ICE candidate (Local/Remote): srflx/srflx
ICE candidate endpoints (Local/Remote): [redacted]
Last connection update: 2 minutes, 12 seconds ago
Last WireGuard handshake: 7 seconds ago
Transfer status (received/sent) 216 B/520 B
Quantum resistance: false
Routes: -
Latency: 20.7595ms

vpn-3.netbird.selfhosted:
NetBird IP: [redacted]
Public key: [redacted]
Status: Connected
-- detail --
Connection type: P2P
Direct: true
ICE candidate (Local/Remote): srflx/srflx
ICE candidate endpoints (Local/Remote): [redacted]
Last connection update: 2 minutes, 12 seconds ago
Last WireGuard handshake: 2 seconds ago
Transfer status (received/sent) 364 B/760 B
Quantum resistance: false
Routes: -
Latency: 26.5064ms

vpn-1.netbird.selfhosted:
NetBird IP: [redacted]
Public key: [redacted]
Status: Connected
-- detail --
Connection type: P2P
Direct: true
ICE candidate (Local/Remote): srflx/srflx
ICE candidate endpoints (Local/Remote): [redacted]
Last connection update: 2 minutes, 12 seconds ago
Last WireGuard handshake: 7 seconds ago
Transfer status (received/sent) 216 B/520 B
Quantum resistance: false
Routes: -
Latency: 25.9289ms

OS: windows/amd64
Daemon version: 0.27.4
CLI version: 0.27.4
Management: Connected to https://[redacted]
Signal: Connected to https://[redacted]
Relays:
[stun:[redacted]:3478] is Available
[turn:[redacted]:3478?transport=udp] is Available
Nameservers:
[9.9.9.9:53, 149.112.112.112:53] for [.] is Available
[10.0.0.2:53] for [test.internal] is Unavailable, reason: 1 error occurred:
* read udp 192.168.0.132:52965->10.0.0.2:53: i/o timeout
[10.0.10.2:53] for [acc.internal] is Unavailable, reason: 1 error occurred:
* read udp 192.168.0.132:52966->10.0.10.2:53: i/o timeout
[10.0.30.2:53] for [eu.prod.internal] is Unavailable, reason: 1 error occurred:
* read udp 192.168.0.132:52963->10.0.30.2:53: i/o timeout
FQDN: prec55404.netbird.selfhosted
NetBird IP: [redacted]
Interface type: Userspace
Quantum resistance: false
Routes: -
Peers count: 3/3 Connected

Screenshots

Client.log

2024-05-06T09:28:56+02:00 ERRO client/internal/routemanager/systemops.go:385: Failed to add route reference: adding route reference: failed to add route for prefix [redacted]: add route to table: route add: exec: "route": cannot run executable found relative to current directory
2024-05-06T09:28:56+02:00 ERRO client/internal/routemanager/systemops.go:385: Failed to add route reference: adding route reference: failed to add route for prefix [redacted]: add route to table: route add: exec: "route": cannot run executable found relative to current directory

Originally created by @Icare-github on GitHub (May 7, 2024). **Describe the problem** Once connected, routes aren't being created on workstation to different environments. **Expected behavior** All routes created, once connected. **NetBird version** Netbird v0.27.4 vpn-2.netbird.selfhosted: NetBird IP: [redacted] Public key: [redacted] Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): [redacted] Last connection update: 2 minutes, 12 seconds ago Last WireGuard handshake: 7 seconds ago Transfer status (received/sent) 216 B/520 B Quantum resistance: false Routes: - Latency: 20.7595ms vpn-3.netbird.selfhosted: NetBird IP: [redacted] Public key: [redacted] Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): [redacted] Last connection update: 2 minutes, 12 seconds ago Last WireGuard handshake: 2 seconds ago Transfer status (received/sent) 364 B/760 B Quantum resistance: false Routes: - Latency: 26.5064ms vpn-1.netbird.selfhosted: NetBird IP: [redacted] Public key: [redacted] Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): [redacted] Last connection update: 2 minutes, 12 seconds ago Last WireGuard handshake: 7 seconds ago Transfer status (received/sent) 216 B/520 B Quantum resistance: false Routes: - Latency: 25.9289ms OS: windows/amd64 Daemon version: 0.27.4 CLI version: 0.27.4 Management: Connected to https://[redacted] Signal: Connected to https://[redacted] Relays: [stun:[redacted]:3478] is Available [turn:[redacted]:3478?transport=udp] is Available Nameservers: [9.9.9.9:53, 149.112.112.112:53] for [.] is Available [10.0.0.2:53] for [test.internal] is Unavailable, reason: 1 error occurred: * read udp 192.168.0.132:52965->10.0.0.2:53: i/o timeout [10.0.10.2:53] for [acc.internal] is Unavailable, reason: 1 error occurred: * read udp 192.168.0.132:52966->10.0.10.2:53: i/o timeout [10.0.30.2:53] for [eu.prod.internal] is Unavailable, reason: 1 error occurred: * read udp 192.168.0.132:52963->10.0.30.2:53: i/o timeout FQDN: prec55404.netbird.selfhosted NetBird IP: [redacted] Interface type: Userspace Quantum resistance: false Routes: - Peers count: 3/3 Connected **Screenshots** Client.log 2024-05-06T09:28:56+02:00 ERRO client/internal/routemanager/systemops.go:385: Failed to add route reference: adding route reference: failed to add route for prefix [redacted]: add route to table: route add: exec: "route": cannot run executable found relative to current directory 2024-05-06T09:28:56+02:00 ERRO client/internal/routemanager/systemops.go:385: Failed to add route reference: adding route reference: failed to add route for prefix [redacted]: add route to table: route add: exec: "route": cannot run executable found relative to current directory
saavagebueno added the triage-needed label 2025-11-20 05:18:44 -05:00
Author
Owner

@Icare-github commented on GitHub (May 7, 2024):

FYI - working correctly in 0.27.3

@Icare-github commented on GitHub (May 7, 2024): FYI - working correctly in 0.27.3
Author
Owner

@lixmal commented on GitHub (May 7, 2024):

Please share your windows + patch version.

It's very unusual that route is missing or not the in the PATH env var. Can you check if it exists in your system, and/or PATH is set correctly?

E.g. https://www.computerhope.com/issues/ch000549.htm

@lixmal commented on GitHub (May 7, 2024): Please share your windows + patch version. It's very unusual that `route` is missing or not the in the `PATH` env var. Can you check if it exists in your system, and/or `PATH` is set correctly? E.g. https://www.computerhope.com/issues/ch000549.htm
Author
Owner

@Icare-github commented on GitHub (May 7, 2024):

Route cmd is not missing. Route can be executed in cmd.

OS Windows
OS name Microsoft Windows 11 Pro (x64)
OS version 10.0.22631
OS full version Version 23H2 (OS Build 22631.3447)
Service pack N/A
Build version 22631

@Icare-github commented on GitHub (May 7, 2024): Route cmd is not missing. Route can be executed in cmd. OS Windows OS name Microsoft Windows 11 Pro (x64) OS version 10.0.22631 OS full version Version 23H2 (OS Build 22631.3447) Service pack N/A Build version 22631
Author
Owner

@myevit commented on GitHub (May 7, 2024):

Same 0.27.4 - issues with routes
0.27.3 - working
windows.

@myevit commented on GitHub (May 7, 2024): Same 0.27.4 - issues with routes 0.27.3 - working windows.
Author
Owner

@mlsmaycon commented on GitHub (May 7, 2024):

@myevit @Icare-github does the route command work on both cmd and powershell?

Is there any customization with the system about its default shell?

@mlsmaycon commented on GitHub (May 7, 2024): @myevit @Icare-github does the route command work on both cmd and powershell? Is there any customization with the system about its default shell?
Author
Owner

@myevit commented on GitHub (May 7, 2024):

@myevit @Icare-github does the route command work on both cmd and powershell?

Is there any customization with the system about its default shell?

route command works in cmd and pwsh
No customizations. Just rolled back to 0.7.3 and all working as intended.
I can speculate: I have multiple routes that are using same name. Could be bug there?

@myevit commented on GitHub (May 7, 2024): > @myevit @Icare-github does the route command work on both cmd and powershell? > > Is there any customization with the system about its default shell? route command works in cmd and pwsh No customizations. Just rolled back to 0.7.3 and all working as intended. I can speculate: I have multiple routes that are using same name. Could be bug there?
Author
Owner

@myevit commented on GitHub (May 7, 2024):

https://github.com/netbirdio/netbird/issues/1925 possible the same issue

@myevit commented on GitHub (May 7, 2024): https://github.com/netbirdio/netbird/issues/1925 possible the same issue
Author
Owner

@mlsmaycon commented on GitHub (May 7, 2024):

The issue on #1925 is a different one.

@myevit can you share your logs too? https://docs.netbird.io/how-to/troubleshooting-client#getting-client-logs

If you didn't clean the logs, you can just copy the current file, as it will contain errors from the 0.27.4 version.

@mlsmaycon commented on GitHub (May 7, 2024): The issue on #1925 is a different one. @myevit can you share your logs too? https://docs.netbird.io/how-to/troubleshooting-client#getting-client-logs If you didn't clean the logs, you can just copy the current file, as it will contain errors from the 0.27.4 version.
Author
Owner

@myevit commented on GitHub (May 7, 2024):

here are logs from 0.27.4 and 0.27.4

0.7.3.log
0 7 3
0.7.4.log
0 7 4

  • I have deleted .log file before each install.
    P.S. my setup is self hosted
@myevit commented on GitHub (May 7, 2024): here are logs from 0.27.4 and 0.27.4 [0.7.3.log](https://github.com/netbirdio/netbird/files/15239963/0.7.3.log) <img width="581" alt="0 7 3" src="https://github.com/netbirdio/netbird/assets/13375246/f19787ad-e28c-4028-b7ec-fe2fd7be349d"> [0.7.4.log](https://github.com/netbirdio/netbird/files/15239964/0.7.4.log) <img width="572" alt="0 7 4" src="https://github.com/netbirdio/netbird/assets/13375246/b9c21d9c-a4d9-46bc-b57d-c4a004441b7e"> * I have deleted .log file before each install. P.S. my setup is self hosted
Author
Owner

@karstennilsen commented on GitHub (May 8, 2024):

I think you should do route print to get more useful info for this ticket.
Just route gives just a manual of the command.

@karstennilsen commented on GitHub (May 8, 2024): I think you should do `route print` to get more useful info for this ticket. Just `route` gives just a manual of the command.
Author
Owner

@Icare-github commented on GitHub (May 8, 2024):

"route" command is working in cmd and powershell. No custimization.
nb1
nb2

@Icare-github commented on GitHub (May 8, 2024): "route" command is working in cmd and powershell. No custimization. ![nb1](https://github.com/netbirdio/netbird/assets/126246289/79a348e1-b501-4783-9ffd-398a1abb2fd2) ![nb2](https://github.com/netbirdio/netbird/assets/126246289/b1efc3fd-b5db-4fc0-acc2-2ae293a1d79d)
Author
Owner

@mlsmaycon commented on GitHub (May 8, 2024):

Hello @Icare-github @myevit can you please test a development version from PR #1946?

To do that you need to download the artifact https://github.com/netbirdio/netbird/actions/runs/8999323479/artifacts/1483471589, then you need to stop the netbird service, extract the artifact and move the file into the C:\Program Files\NetBird path.

Here is how you stop the service using an elevated PowerShell:

netbird service stop
# after copying the file
netbird service start

Just a heads up, the binary is not signed and the machine's antivirus might complain.

Please share the logs after running the client as well. It will print the PATH value our agent is getting from these nodes.

@mlsmaycon commented on GitHub (May 8, 2024): Hello @Icare-github @myevit can you please test a development version from PR #1946? To do that you need to download the artifact https://github.com/netbirdio/netbird/actions/runs/8999323479/artifacts/1483471589, then you need to stop the netbird service, extract the artifact and move the file into the C:\Program Files\NetBird path. Here is how you stop the service using an elevated PowerShell: ```shell netbird service stop # after copying the file netbird service start ``` Just a heads up, the binary is not signed and the machine's antivirus might complain. Please share the logs after running the client as well. It will print the PATH value our agent is getting from these nodes.
Author
Owner

@myevit commented on GitHub (May 8, 2024):

I think you should do route print to get more useful info for this ticket. Just route gives just a manual of the command.

My point was that route command is working. Just that.

@myevit commented on GitHub (May 8, 2024): > I think you should do `route print` to get more useful info for this ticket. Just `route` gives just a manual of the command. My point was that route command is working. Just that.
Author
Owner

@myevit commented on GitHub (May 8, 2024):

Hello @Icare-github @myevit can you please test a development version from PR #1946?

To do that you need to download the artifact https://github.com/netbirdio/netbird/actions/runs/8999323479/artifacts/1483471589, then you need to stop the netbird service, extract the artifact and move the file into the C:\Program Files\NetBird path.

Here is how you stop the service using an elevated PowerShell:

netbird service stop
# after copying the file
netbird service start

Just a heads up, the binary is not signed and the machine's antivirus might complain.

Please share the logs after running the client as well. It will print the PATH value our agent is getting from these nodes.

Yep. That fix is working for me. So far no issues. All running as expected.

@myevit commented on GitHub (May 8, 2024): > Hello @Icare-github @myevit can you please test a development version from PR #1946? > > To do that you need to download the artifact https://github.com/netbirdio/netbird/actions/runs/8999323479/artifacts/1483471589, then you need to stop the netbird service, extract the artifact and move the file into the C:\Program Files\NetBird path. > > Here is how you stop the service using an elevated PowerShell: > > ```shell > netbird service stop > # after copying the file > netbird service start > ``` > > Just a heads up, the binary is not signed and the machine's antivirus might complain. > > Please share the logs after running the client as well. It will print the PATH value our agent is getting from these nodes. Yep. That fix is working for me. So far no issues. All running as expected.
Author
Owner

@mlsmaycon commented on GitHub (May 8, 2024):

@myevit thanks for confirming. Could you please share the logs from your peer too?

@mlsmaycon commented on GitHub (May 8, 2024): @myevit thanks for confirming. Could you please share the logs from your peer too?
Author
Owner

@myevit commented on GitHub (May 8, 2024):

client.log
here is the logs from PR

@myevit commented on GitHub (May 8, 2024): [client.log](https://github.com/netbirdio/netbird/files/15252713/client.log) here is the logs from PR
Author
Owner

@mlsmaycon commented on GitHub (May 8, 2024):

Thanks for sharing the logs @myevit we found another issue with the paths and added a fix.

We should have a release with the fix this week.

@mlsmaycon commented on GitHub (May 8, 2024): Thanks for sharing the logs @myevit we found another issue with the paths and added a fix. We should have a release with the fix this week.
Author
Owner

@mlsmaycon commented on GitHub (May 9, 2024):

@myevit I have a new version with better path handling. Can you run the last test? https://github.com/netbirdio/netbird/actions/runs/9006903302/artifacts/1485301075

@mlsmaycon commented on GitHub (May 9, 2024): @myevit I have a new version with better path handling. Can you run the last test? https://github.com/netbirdio/netbird/actions/runs/9006903302/artifacts/1485301075
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#857