SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-04-29 21:01:56 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

After upgrading a node from 0.13.0 to 0.14.2 I stop being able to connect to it #283

New Issue
Closed
opened 2025-11-20 05:09:04 -05:00 by saavagebueno · 6 comments
saavagebueno commented 2025-11-20 05:09:04 -05:00
Owner
Copy Link

Originally created by @rgl on GitHub (Mar 4, 2023).

Describe the problem

After upgrading a ubuntu machine from 0.13.0 to 0.14.2 I stop being able to connect to it tru the netbird/wireguard IP address. If I downgrade to 0.13.0 it works again.

The ubuntu machine is inside an unknown corporate network. So I do not really known what is going on between it and my windows home machine (using netbird 0.14.2).

Any help trying to troubleshoot this would be welcome.

Expected behavior

I expected to keep using netbird/wireguard network after upgrade.

NetBird status -d output:

With 0.13.0:

Peers detail:
 home.netbird.cloud:
  NetBird IP: 100.X.X.123
  Public key: XXX
  Status: Connected
  -- detail --
  Connection type: Relayed
  Direct: false
  ICE candidate (Local/Remote): relay/prflx
  Last connection update: 2023-03-04 12:38:51
...
Daemon version: 0.13.0
CLI version: 0.13.0
Daemon status: Connected
Management: Connected to https://api.wiretrustee.com:443
Signal: Connected to https://signal2.wiretrustee.com:443
Domain: xxx.netbird.cloud
NetBird IP: 100.X.X.124/16
Interface type: Kernel
Peers count: 2/5 Connected

With 0.14.2, it never leaves the Connecting state:

Peers detail:
 home.netbird.cloud:
  NetBird IP: 100.X.X.123
  Public key: XXX
  Status: Connecting
  -- detail --
  Connection type:
  Direct: false
  ICE candidate (Local/Remote): -/-
  Last connection update: 2023-03-04 12:53:13
Originally created by @rgl on GitHub (Mar 4, 2023). **Describe the problem** After upgrading a ubuntu machine from 0.13.0 to 0.14.2 I stop being able to connect to it tru the netbird/wireguard IP address. If I downgrade to 0.13.0 it works again. The ubuntu machine is inside an unknown corporate network. So I do not really known what is going on between it and my windows home machine (using netbird 0.14.2). Any help trying to troubleshoot this would be welcome. **Expected behavior** I expected to keep using netbird/wireguard network after upgrade. **NetBird status -d output:** With 0.13.0: ``` Peers detail: home.netbird.cloud: NetBird IP: 100.X.X.123 Public key: XXX Status: Connected -- detail -- Connection type: Relayed Direct: false ICE candidate (Local/Remote): relay/prflx Last connection update: 2023-03-04 12:38:51 ... Daemon version: 0.13.0 CLI version: 0.13.0 Daemon status: Connected Management: Connected to https://api.wiretrustee.com:443 Signal: Connected to https://signal2.wiretrustee.com:443 Domain: xxx.netbird.cloud NetBird IP: 100.X.X.124/16 Interface type: Kernel Peers count: 2/5 Connected ``` With 0.14.2, it never leaves the `Connecting` state: ```plain Peers detail: home.netbird.cloud: NetBird IP: 100.X.X.123 Public key: XXX Status: Connecting -- detail -- Connection type: Direct: false ICE candidate (Local/Remote): -/- Last connection update: 2023-03-04 12:53:13 ```
saavagebueno commented 2025-11-20 05:09:05 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 4, 2023):

Hello @rgl,did you update the dashboard as well?

is there any change to it's status in the UI? Is the peer listed as expired?

@mlsmaycon commented on GitHub (Mar 4, 2023): Hello @rgl,did you update the dashboard as well? is there any change to it's status in the UI? Is the peer listed as expired?
saavagebueno commented 2025-11-20 05:09:05 -05:00
Author
Owner
Copy Link

@rgl commented on GitHub (Mar 4, 2023):

@mlsmaycon, what do you mean by "update the dashboard"? Not sure you are referring to the new client expiration feature, if you are, I did not touch/change it. I just upgraded to the new binary and installed it. Didn't touch anything else.

The peer appears as Online and with a "expiration disabled" tag.

I did not mention it before, but I'm using the public instance at https://app.netbird.io/peers. I'm not using the self-hosted edition.

@rgl commented on GitHub (Mar 4, 2023): @mlsmaycon, what do you mean by "update the dashboard"? Not sure you are referring to the new client expiration feature, if you are, I did not touch/change it. I just upgraded to the new binary and installed it. Didn't touch anything else. The peer appears as Online and with a "expiration disabled" tag. I did not mention it before, but I'm using the public instance at https://app.netbird.io/peers. I'm not using the self-hosted edition.
saavagebueno commented 2025-11-20 05:09:05 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 4, 2023):

Hello @rgl thanks for clarifying that. If you have a out of band access to the ubuntu machine, can you run the following commands and share the logs with us?

sudo netbird service stop
sudo bash -c 'NB_LOG_LEVEL=debug PIONS_LOG_DEBUG=all netbird up -F' > client.log 2>&1
# after 40 seconds stop execution with:
CTRL + c
@mlsmaycon commented on GitHub (Mar 4, 2023): Hello @rgl thanks for clarifying that. If you have a out of band access to the ubuntu machine, can you run the following commands and share the logs with us? ```shell sudo netbird service stop sudo bash -c 'NB_LOG_LEVEL=debug PIONS_LOG_DEBUG=all netbird up -F' > client.log 2>&1 # after 40 seconds stop execution with: CTRL + c ```
saavagebueno commented 2025-11-20 05:09:06 -05:00
Author
Owner
Copy Link

@rgl commented on GitHub (Mar 4, 2023):

@mlsmaycon, is there a way to anonymize the log?

in the mean time, some strange things that I've identified, which I'm not sure are normal.

There are a bunch of Host not found errors, which do not happen when I dig them. So I'm not sure why these are here:

turnc DEBUG: 18:15:23.363484 client.go:108: resolving turn.netbird.io:5555
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host
...
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host
ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: stun.netbird.io:5555: lookup stun.netbird.io on 10.53.53.53:53: no such host

One thing to note, this network does not allow outbound access to the Relay (TURN) service ports; tho that does not prevent 0.13.0 from working, because outbound to port 443 is allowed.

But maybe, those warnings messages are expected, because it seems to have succeeded with port 443:

ice DEBUG: 18:15:24.760319 udp_mux.go:260: Registered 10.53.53.53:44200 for AtNJodWjCvHxrbZIturns:turneutls.netbird.io:443?transport=tcp
ice DEBUG: 18:15:24.760394 udp_mux.go:260: Registered 10.53.53.53:44200 for AtNJodWjCvHxrbZIturns:turneutls.netbird.io:443?transport=tcp

I was not expecting it to touch that 10.53.53.53 interface, that one should be treated like localhost, is there a way for doing that?.

And these:

turnc DEBUG: 18:15:23.380070 client.go:114: turnServ: 18.198.13.240:5555
ice WARNING: 2023/03/04 18:15:23 No Username attribute in STUN message from 52.16.140.95:443
ice WARNING: 2023/03/04 18:15:23 No Username attribute in STUN message from 52.16.140.95:443
ice WARNING: 2023/03/04 18:15:23 No Username attribute in STUN message from 52.16.140.95:443

And these:

ice WARNING: 2023/03/04 18:15:28 could not get server reflexive address udp4 turn:turn.netbird.io:5555?transport=udp: timeout while waiting for XORMappedAddrice WARNING: 2023/03/04 18:15:28 could not get server reflexive address udp4 stun:stun.netbird.io:5555: timeout while waiting for XORMappedAddr
@rgl commented on GitHub (Mar 4, 2023): @mlsmaycon, is there a way to anonymize the log? in the mean time, some strange things that I've identified, which I'm not sure are normal. There are a bunch of `Host not found` errors, which do not happen when I `dig` them. So I'm not sure why these are here: ``` turnc DEBUG: 18:15:23.363484 client.go:108: resolving turn.netbird.io:5555 ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host ... ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turn.netbird.io:5555: lookup turn.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: turneutls.netbird.io:443: lookup turneutls.netbird.io on 10.53.53.53:53: no such host ice WARNING: 2023/03/04 18:15:23 failed to resolve stun host: stun.netbird.io:5555: lookup stun.netbird.io on 10.53.53.53:53: no such host ``` One thing to note, this network does not allow outbound access to the [Relay (TURN) service ports](https://netbird.io/docs/FAQ/firewall-ports); tho that does not prevent 0.13.0 from working, because outbound to port 443 is allowed. But maybe, those warnings messages are expected, because it seems to have succeeded with port 443: ``` ice DEBUG: 18:15:24.760319 udp_mux.go:260: Registered 10.53.53.53:44200 for AtNJodWjCvHxrbZIturns:turneutls.netbird.io:443?transport=tcp ice DEBUG: 18:15:24.760394 udp_mux.go:260: Registered 10.53.53.53:44200 for AtNJodWjCvHxrbZIturns:turneutls.netbird.io:443?transport=tcp ``` I was not expecting it to touch that 10.53.53.53 interface, that one should be treated like localhost, is there a way for doing that?. And these: ``` turnc DEBUG: 18:15:23.380070 client.go:114: turnServ: 18.198.13.240:5555 ice WARNING: 2023/03/04 18:15:23 No Username attribute in STUN message from 52.16.140.95:443 ice WARNING: 2023/03/04 18:15:23 No Username attribute in STUN message from 52.16.140.95:443 ice WARNING: 2023/03/04 18:15:23 No Username attribute in STUN message from 52.16.140.95:443 ``` And these: ``` ice WARNING: 2023/03/04 18:15:28 could not get server reflexive address udp4 turn:turn.netbird.io:5555?transport=udp: timeout while waiting for XORMappedAddrice WARNING: 2023/03/04 18:15:28 could not get server reflexive address udp4 stun:stun.netbird.io:5555: timeout while waiting for XORMappedAddr ```
saavagebueno commented 2025-11-20 05:09:06 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 5, 2023):

Hello @rgl thanks for the logs.

@mlsmaycon, is there a way to anonymize the log?

You can use sed to replace public and private addresses, see example:

sed -i 's/8.8.8.8/NET_PUBLIC_IP/g' client.log
sed -i 's/10.0.0.1/NET_PRIVATE_IP1/g' client.log

There are a bunch of Host not found errors, which do not happen when I dig them. So I'm not sure why these are here:

These are from IPV6 lookup, we will be enabling it in the next coming weeks. they shouldn't influence the connection status

One thing to note, this network does not allow outbound access to the Relay (TURN) service ports; tho that does not prevent 0.13.0 from working, because outbound to port 443 is allowed.

Both versions should be able to talk to our TCP turn servers on port 443 (turneutls.netbird.io:443)

I was not expecting it to touch that 10.53.53.53 interface, that one should be treated like localhost, is there a way for doing that?.

You can edit the /etc/netbird/config.json and add the interface from 10.53.53.53 Ip into the black list.

No Username attribute in STUN message

Is normal, we can ignore it

timeout while waiting for XORMappedAddr

As the port 5555 is blocked, this is expected.

Can you run the sed command and share all the logs? You can also send me via slack if you are concern in sharing it here.

@mlsmaycon commented on GitHub (Mar 5, 2023): Hello @rgl thanks for the logs. >@mlsmaycon, is there a way to anonymize the log? You can use `sed` to replace public and private addresses, see example: ```shell sed -i 's/8.8.8.8/NET_PUBLIC_IP/g' client.log sed -i 's/10.0.0.1/NET_PRIVATE_IP1/g' client.log ``` > There are a bunch of Host not found errors, which do not happen when I dig them. So I'm not sure why these are here: These are from IPV6 lookup, we will be enabling it in the next coming weeks. they shouldn't influence the connection status > One thing to note, this network does not allow outbound access to the [Relay (TURN) service ports](https://netbird.io/docs/FAQ/firewall-ports); tho that does not prevent 0.13.0 from working, because outbound to port 443 is allowed. Both versions should be able to talk to our TCP turn servers on port 443 (turneutls.netbird.io:443) > I was not expecting it to touch that 10.53.53.53 interface, that one should be treated like localhost, is there a way for doing that?. You can edit the /etc/netbird/config.json and add the interface from 10.53.53.53 Ip into the black list. > No Username attribute in STUN message Is normal, we can ignore it > timeout while waiting for XORMappedAddr As the port 5555 is blocked, this is expected. Can you run the `sed` command and share all the logs? You can also send me via slack if you are concern in sharing it here.
saavagebueno commented 2025-11-20 05:09:06 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 5, 2023):

Thanks @rgl for sharing he logs in slack. The issue was caused by https://github.com/pion/ice/pull/539, as your peer is running on restricted NAT, it requires TLS relay, but the issue from the link prevented the connection to work.

@mlsmaycon commented on GitHub (Mar 5, 2023): Thanks @rgl for sharing he logs in slack. The issue was caused by https://github.com/pion/ice/pull/539, as your peer is running on restricted NAT, it requires TLS relay, but the issue from the link prevented the connection to work.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#283
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?