SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-04-20 08:22:27 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

Netbird should connect to peers before setting up DNS #899

New Issue
Open
opened 2025-11-20 05:19:33 -05:00 by saavagebueno · 4 comments
saavagebueno commented 2025-11-20 05:19:33 -05:00
Owner
Copy Link

Originally created by @Thunderbottom on GitHub (May 16, 2024).

Describe the problem

In the latest version, netbird tries to resolve DNS before connecting to the peers. This causes the DNS resolution to fail in case where the DNS being used is a private DNS behind a routing peer. This further causes netbird to wait for the DNS resolution to timeout before connecting to peers on the network, and hence, it takes at least 15 seconds in our case to connect to the first peer.

After the peer connects, the DNS resolution works perfectly fine. But this delay in most cases is unbearable and causes usability issues for a lot of people.

Logs:

May 16 22:31:13 hades netbird[1405]: 2024-05-16T22:31:13+05:30 INFO signal/client/grpc.go:158: connected to the Signal Service stream
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:59667->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:41074->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:37390->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN client/internal/dns/upstream.go:265: Upstream resolving is Disabled for 30s
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 INFO [nameservers: [{192.168.0.2 udp 53}]] client/internal/dns/server.go:504: Temporarily deactivating nameservers group due to timeout
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:59543->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:56609->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [upstream: 192.168.0.2:53, error: read udp 192.168.69.100:45333->192.168.0.2:53: i/o timeout] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 INFO client/internal/dns/resolvconf_linux.go:73: added 2 search domains. Search list: [local.netbird lan]
May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 WARN [error: read udp 192.168.69.100:46766->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 WARN [error: read udp 192.168.69.100:57562->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream
May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout
May 16 22:31:17 hades netbird[1405]: 2024-05-16T22:31:17+05:30 INFO management/client/grpc.go:147: connected to the Management Service stream
May 16 22:31:17 hades netbird[1405]: 2024-05-16T22:31:17+05:30 WARN client/internal/routemanager/client.go:154: the network 192.168.0.0/19 has not been assigned a routing peer as no peers from the list [<LIST>] are currently connected
May 16 22:31:18 hades netbird[1405]: 2024-05-16T22:31:18+05:30 INFO client/internal/routemanager/client.go:165: new chosen route is <ROUTE> with peer <PEER-ID> with score 2.974409 for network 192.168.0.2/32
May 16 22:31:18 hades netbird[1405]: 2024-05-16T22:31:18+05:30 INFO client/internal/dns/upstream.go:241: upstreams [192.168.0.2:53] are responsive again. Adding them back to system

In the logs it took a few seconds to connect, but usually on netbird up, this takes at least 10-15 seconds to connect.

To Reproduce

Steps to reproduce the behavior:

  1. Set up a private DNS on netbird using routing peer.
  2. Connect to netbird.
  3. Notice that netbird tries to resolve DNS and fails before trying to connect to the peers.
  4. See error.

Expected behavior

The DNS resolution should take place after the peer connections are initialized. There's no need for netbird to replace and resolve DNS before connecting to peers.

Are you using NetBird Cloud?

Self-hosted NetBird's control plane.

NetBird version

netbird version: 0.27.7

Originally created by @Thunderbottom on GitHub (May 16, 2024). **Describe the problem** In the latest version, netbird tries to resolve DNS before connecting to the peers. This causes the DNS resolution to fail in case where the DNS being used is a private DNS behind a routing peer. This further causes netbird to wait for the DNS resolution to timeout before connecting to peers on the network, and hence, it takes at least 15 seconds in our case to connect to the first peer. After the peer connects, the DNS resolution works perfectly fine. But this delay in most cases is unbearable and causes usability issues for a lot of people. Logs: ``` May 16 22:31:13 hades netbird[1405]: 2024-05-16T22:31:13+05:30 INFO signal/client/grpc.go:158: connected to the Signal Service stream May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:59667->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:41074->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:37390->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN client/internal/dns/upstream.go:265: Upstream resolving is Disabled for 30s May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 INFO [nameservers: [{192.168.0.2 udp 53}]] client/internal/dns/server.go:504: Temporarily deactivating nameservers group due to timeout May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:59543->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [error: read udp 192.168.69.100:56609->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 WARN [upstream: 192.168.0.2:53, error: read udp 192.168.69.100:45333->192.168.0.2:53: i/o timeout] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:15 hades netbird[1405]: 2024-05-16T22:31:15+05:30 INFO client/internal/dns/resolvconf_linux.go:73: added 2 search domains. Search list: [local.netbird lan] May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 WARN [error: read udp 192.168.69.100:46766->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 WARN [error: read udp 192.168.69.100:57562->192.168.0.2:53: i/o timeout, upstream: 192.168.0.2:53] client/internal/dns/upstream.go:102: got an error while connecting to upstream May 16 22:31:16 hades netbird[1405]: 2024-05-16T22:31:16+05:30 ERRO client/internal/dns/upstream.go:134: all queries to the upstream nameservers failed with timeout May 16 22:31:17 hades netbird[1405]: 2024-05-16T22:31:17+05:30 INFO management/client/grpc.go:147: connected to the Management Service stream May 16 22:31:17 hades netbird[1405]: 2024-05-16T22:31:17+05:30 WARN client/internal/routemanager/client.go:154: the network 192.168.0.0/19 has not been assigned a routing peer as no peers from the list [<LIST>] are currently connected May 16 22:31:18 hades netbird[1405]: 2024-05-16T22:31:18+05:30 INFO client/internal/routemanager/client.go:165: new chosen route is <ROUTE> with peer <PEER-ID> with score 2.974409 for network 192.168.0.2/32 May 16 22:31:18 hades netbird[1405]: 2024-05-16T22:31:18+05:30 INFO client/internal/dns/upstream.go:241: upstreams [192.168.0.2:53] are responsive again. Adding them back to system ``` In the logs it took a few seconds to connect, but usually on `netbird up`, this takes at least 10-15 seconds to connect. **To Reproduce** Steps to reproduce the behavior: 1. Set up a private DNS on netbird using routing peer. 2. Connect to netbird. 3. Notice that netbird tries to resolve DNS and fails before trying to connect to the peers. 4. See error. **Expected behavior** The DNS resolution should take place after the peer connections are initialized. There's no need for netbird to replace and resolve DNS before connecting to peers. **Are you using NetBird Cloud?** Self-hosted NetBird's control plane. **NetBird version** `netbird version`: `0.27.7`
saavagebueno added the bugclientdns labels 2025-11-20 05:19:33 -05:00
saavagebueno commented 2025-11-20 05:19:34 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (May 17, 2024):

Thanks for opening this bug @Thunderbottom , the current behavior is the following:

We configure DNS and test it right the way, it should fail faster for this initial test. Decreasing timeout will help but not setting it up is the best approach.

@mlsmaycon commented on GitHub (May 17, 2024): Thanks for opening this bug @Thunderbottom , the current behavior is the following: We configure DNS and test it right the way, it should fail faster for this initial test. Decreasing timeout will help but not setting it up is the best approach.
saavagebueno commented 2025-11-20 05:19:34 -05:00
Author
Owner
Copy Link

@LeszekBlazewski commented on GitHub (Aug 16, 2024):

Looking forward for the fix for this as well.

My use case is that I have a inbound route53 private hosted zone resolver which sits within a private VPC subnet and access to it is allowed from the range of IP addresses in which a netbird routing peer runs (so without sending the traffic via the peer, the DNS won't resolve)

As mentioned in the issue, the current behaviour on latest version (0.28.7) is that it works sometimes and it doesn't in other cases (happens randomly for both linux and macos).

@LeszekBlazewski commented on GitHub (Aug 16, 2024): Looking forward for the fix for this as well. My use case is that I have a inbound route53 private hosted zone resolver which sits within a private VPC subnet and access to it is allowed from the range of IP addresses in which a netbird routing peer runs (so without sending the traffic via the peer, the DNS won't resolve) As mentioned in the issue, the current behaviour on latest version (0.28.7) is that it works sometimes and it doesn't in other cases (happens randomly for both linux and macos).
saavagebueno commented 2025-11-20 05:19:34 -05:00
Author
Owner
Copy Link

@galiev commented on GitHub (Apr 2, 2025):

sorry, if you can help i'll be very grateful, i also have private dns server, will it be a solution if i declare public dns servers for zone (*.example.com) where my peers, relay, signal etc are and declare my private dns servers for all other zones?

@galiev commented on GitHub (Apr 2, 2025): sorry, if you can help i'll be very grateful, i also have private dns server, will it be a solution if i declare public dns servers for zone (*.example.com) where my peers, relay, signal etc are and declare my private dns servers for all other zones?
saavagebueno commented 2025-11-20 05:19:34 -05:00
Author
Owner
Copy Link

@nazarewk commented on GitHub (Apr 2, 2025):

Just to give you an update: we're planning to slightly revamp and expand the Nameserver later this year, it should be touching on this issue too.

@nazarewk commented on GitHub (Apr 2, 2025): Just to give you an update: we're planning to slightly revamp and expand the Nameserver later this year, it should be touching on this issue too.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label bug client dns
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#899
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?