SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-08-04 19:55:09 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

DNS resolution with Nameserver behind bastion host stopped working #1988

New Issue
Closed
opened 2025-11-20 06:10:43 -05:00 by saavagebueno · 3 comments
saavagebueno commented 2025-11-20 06:10:43 -05:00
Owner
Copy Link

Originally created by @Jachimoo on GitHub (Jun 20, 2025).

Describe the problem

After updating client on ubuntu 22.04, the DNS resolution stopped working. The DNS nameservers for domain are configured and accessible but client

To Reproduce

Steps to reproduce the behavior:

  1. Have a selfhosted netbird instance working as bastion host, with DNS servers that extends our public domain to internal servers
  2. Configure Nameservers for client group to be set as internal servers for our domain (We use FreeIPA)
  3. Client v0.46 works fine
  4. Client v0.48 in unable to use internal DNS (Logs says that there is a timeout, but if we check manually the DNS server is accessible and resolves names)

Expected behavior

The nameservers are used without any problems

Are you using NetBird Cloud?

No I'm using selfhosted

NetBird version
Server:

  • Control plane: latest as of 20.06.2025
  • Client: 0.45.1

Client:
Works: 0.46.0
Broken: 0.48.0

Is any other VPN software installed?

Nope

Debug output (domain obfustated)
client 0.46:

Peers detail:
  <some-client>anon-ZmCZH.domain:
  NetBird IP: 100.125.60.209/32
  Public key: VZMrvHq28JZD2xB7G+EuJ2QpME+vQVgOFfpjBrQRumI=
  Status: Idle
  -- detail --
  Connection type: 
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: 
  Last connection update: 1 minute, 34 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 0 B/0 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

 <some-client>.anon-ZmCZH.domain:
  NetBird IP: 100.125.36.234/32
  Public key: 0NguuRsxno4LMfvlkljqOhkFdXLlEPAigLiKOP3C308=
  Status: Idle
  -- detail --
  Connection type: 
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: 
  Last connection update: 1 minute, 34 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 0 B/0 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

 <routing_peer/control_plane>.anon-ZmCZH.domain:
  NetBird IP: 100.125.37.253
  Public key: cl2RJM9i3oqDb0422dzOa1SOU+9OXXV5zHKZrGR9u1c=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rel://netbird.anon-ZmCZH.domain:33080
  Last connection update: 1 minute, 33 seconds ago
  Last WireGuard handshake: 1 minute, 28 seconds ago
  Transfer status (received/sent) 9.9 KiB/8.5 KiB
  Quantum resistance: false
  Networks: 10.10.0.0/16, 10.100.0.0/16, 198.51.100.0/32
  Latency: 0s

 <some-cient>anon-ZmCZH.domain:
  NetBird IP: 100.125.51.179
  Public key: Dygbc4m2GoCyJClp6mCEs7tMg3e4gSnDgE8B+HKA9AY=
  Status: Connected
  -- detail --
  Connection type: P2P
  ICE candidate (Local/Remote): host/srflx
  ICE candidate endpoints (Local/Remote): 192.168.122.1:51820/198.51.100.1:16135
  Relay server address: rel://netbird.anon-ZmCZH.domain:33080
  Last connection update: 1 minute, 33 seconds ago
  Last WireGuard handshake: 1 minute, 28 seconds ago
  Transfer status (received/sent) 424 B/364 B
  Quantum resistance: false
  Networks: -
  Latency: 14.349299ms

 <routing_peer>.anon-ZmCZH.domain:
  NetBird IP: 100.125.111.182
  Public key: kOOSzYtKp3kNB7vbgM6ge06DdIHPCds3a3YepEJFR2Q=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rel://netbird.anon-ZmCZH.domain:33080
  Last connection update: 1 minute, 4 seconds ago
  Last WireGuard handshake: 41 seconds ago
  Transfer status (received/sent) 572 B/488 B
  Quantum resistance: false
  Networks: 172.16.0.0/16, anon-LNzcq.domain
  Latency: 29.403567ms

Events:
  [WARNING] DNS (6560e27f-0c7b-441a-bd86-aba3c2b48691)
    Message: All upstream servers failed (probe failed)
    Time: 1 minute, 34 seconds ago
    Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53
  [WARNING] DNS (1f12b277-6c07-4982-9947-40584e5d2080)
    Message: All upstream servers failed (probe failed)
    Time: 1 minute, 34 seconds ago
    Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53
  [WARNING] DNS (e8058d77-f32e-4d00-82bf-09f142e2b958)
    Message: All upstream servers failed (probe failed)
    Time: 1 minute, 34 seconds ago
    Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53
  [INFO] SYSTEM (11ddcb9b-4c1f-41f7-8a8e-bb1f35a8a926)
    Message: Network map updated
    Time: 1 minute, 34 seconds ago
OS: linux/amd64
Daemon version: 0.46.0
CLI version: 0.46.0
Management: Connected to https://netbird.anon-ZmCZH.domain:443
Signal: Connected to https://netbird.anon-ZmCZH.domain:443
Relays: 
  [stun:netbird.anon-ZmCZH.domain:3478] is Available
  [turn:netbird.anon-ZmCZH.domain:3478?transport=udp] is Available
  [rel://netbird.anon-ZmCZH.domain:33080] is Available
Nameservers: 
  [10.10.20.253:53, 10.10.30.253:53] for [anon-ZmCZH.domain, anon-LNzcq.domain, anon-Xm8Aa.domain] is Available
FQDN: <client1>.anon-ZmCZH.domain
NetBird IP: 100.125.212.195/16
Interface type: Kernel
Quantum resistance: false
Lazy connection: false
Networks: -
Forwarding rules: 0
Peers count: 3/5 Connected


dig  <some-app>.anon-ZmCZH.domain                                                                                                                                                                

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> <some-app>.anon-ZmCZH.domain
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36373
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;<some-app>.anon-ZmCZH.domain.		IN	A

;; ANSWER SECTION:
<some-app>.anon-ZmCZH.domain.	1200	IN	A	10.100.x.x //obfustated cuz dont need

;; AUTHORITY SECTION:
anon-ZmCZH.domain		86400	IN	NS	<FreeIpa server>
anon-ZmCZH.domain		86400	IN	NS	<FreeIpa server 2>

;; ADDITIONAL SECTION:
<FreeIpa server>	1200	IN	A	10.10.20.253
<FreeIpa server 2>	1200	IN	A	10.10.30.253

;; Query time: 32 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Fri Jun 20 12:24:18 CEST 2025
;; MSG SIZE  rcvd: 133

client 0.48.:

Peers detail:
 <some-client>.anon-w4kgQ.domain:
  NetBird IP: 100.125.60.209/32
  Public key: VZMrvHq28JZD2xB7G+EuJ2QpME+vQVgOFfpjBrQRumI=
  Status: Idle
  -- detail --
  Connection type: 
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: 
  Last connection update: 24 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 0 B/0 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

 <some-client>.anon-w4kgQ.domain:
  NetBird IP: 100.125.36.234/32
  Public key: 0NguuRsxno4LMfvlkljqOhkFdXLlEPAigLiKOP3C308=
  Status: Idle
  -- detail --
  Connection type: 
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: 
  Last connection update: 24 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 0 B/0 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

  <routing_peer/control_plane>.anon-w4kgQ.domain:
  NetBird IP: 100.125.37.253
  Public key: cl2RJM9i3oqDb0422dzOa1SOU+9OXXV5zHKZrGR9u1c=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rel://netbird.anon-w4kgQ.domain:33080
  Last connection update: 24 seconds ago
  Last WireGuard handshake: 18 seconds ago
  Transfer status (received/sent) 7.2 KiB/5.9 KiB
  Quantum resistance: false
  Networks: 10.10.0.0/16, 10.100.0.0/16, 198.51.100.0/32
  Latency: 0s

 <some-client>.anon-w4kgQ.domain:
  NetBird IP: 100.125.51.179
  Public key: Dygbc4m2GoCyJClp6mCEs7tMg3e4gSnDgE8B+HKA9AY=
  Status: Connected
  -- detail --
  Connection type: P2P
  ICE candidate (Local/Remote): host/prflx
  ICE candidate endpoints (Local/Remote): 192.168.122.1:51820/198.51.100.1:4055
  Relay server address: rel://netbird.anon-w4kgQ.domain:33080
  Last connection update: 22 seconds ago
  Last WireGuard handshake: 18 seconds ago
  Transfer status (received/sent) 328 B/364 B
  Quantum resistance: false
  Networks: -
  Latency: 13.074887ms

 <routing_peer>.anon-w4kgQ.domain:
  NetBird IP: 100.125.111.182
  Public key: kOOSzYtKp3kNB7vbgM6ge06DdIHPCds3a3YepEJFR2Q=
  Status: Connected
  -- detail --
  Connection type: P2P
  ICE candidate (Local/Remote): host/prflx
  ICE candidate endpoints (Local/Remote): 198.51.100.2:51820/198.51.100.0:51820
  Relay server address: rel://netbird.anon-w4kgQ.domain:33080
  Last connection update: 22 seconds ago
  Last WireGuard handshake: 23 seconds ago
  Transfer status (received/sent) 180 B/272 B
  Quantum resistance: false
  Networks: 172.16.0.0/16, anon-xfIRx.domain
  Latency: 33.40652ms

Events:
  [WARNING] DNS (1e0ef040-2d12-4c99-9b29-5a9d0d5ceef8)
    Message: All upstream servers failed (probe failed)
    Time: 24 seconds ago
    Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53
  [WARNING] DNS (783bb17c-2959-44a6-91e2-4d8448fa278f)
    Message: All upstream servers failed (probe failed)
    Time: 24 seconds ago
    Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53
  [WARNING] DNS (5e08b96c-38ba-483f-b945-867d74721478)
    Message: All upstream servers failed (probe failed)
    Time: 24 seconds ago
    Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53
  [INFO] SYSTEM (2b6f86a9-90e7-42d9-a67a-572a542bb26e)
    Message: Network map updated
    Time: 24 seconds ago
OS: linux/amd64
Daemon version: 0.48.0
CLI version: 0.48.0
Management: Connected to https://netbird.anon-w4kgQ.domain:443
Signal: Connected to https://netbird.anon-w4kgQ.domain:443
Relays: 
  [stun:netbird.anon-w4kgQ.domain:3478] is Unavailable, reason: dial: failed to listen: d.Dialer.DialContext: dial udp: lookup netbird.anon-w4kgQ.domain on 127.0.0.53:53: no such host
  [turn:netbird.anon-w4kgQ.domain:3478?transport=udp] is Unavailable, reason: create client: lookup netbird.anon-w4kgQ.domain on 127.0.0.53:53: no such host
  [rel://netbird.anon-w4kgQ.domain:33080] is Available
Nameservers: 
  [10.10.20.253:53, 10.10.30.253:53] for [anon-w4kgQ.domain, anon-xfIRx.domain, anon-OQqQX.domain] is Available
FQDN: gabriel-xps-13-7390.anon-w4kgQ.domain
NetBird IP: 100.125.212.195/16
Interface type: Kernel
Quantum resistance: false
Lazy connection: false
Networks: -
Forwarding rules: 0
Peers count: 3/5 Connected

dig  <some-app>.anon-ZmCZH.domain                                                                                                                                                                  

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>>  <some-app>.anon-ZmCZH.domain
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 47495
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
; <some-app>.anon-ZmCZH.domain		IN	A

;; Query time: 1 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Fri Jun 20 13:07:23 CEST 2025
;; MSG SIZE  rcvd: 45

dig @10.10.20.253 <some-app>.anon-ZmCZH.domain                                                                                                                                                     

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> @10.10.20.253 <some-app>.anon-ZmCZH.domain
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24850
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 1a4fe68f062fd27233556262685540fbea66c3affa429eeb (good)
;; QUESTION SECTION:
;<some-app>.anon-ZmCZH.domain.	IN	A

;; ANSWER SECTION:
<some-app>.anon-ZmCZH.domain	1200	IN	A	10.100.x.x //obfustated cuz dont need

;; AUTHORITY SECTION:
anon-ZmCZH.domain		86400	IN	NS	<FreeIpa server>
anon-ZmCZH.domain		86400	IN	NS	<FreeIpa server 2>

;; ADDITIONAL SECTION:
<FreeIpa server>	1200	IN	A	10.10.20.253
<FreeIpa server 2>	1200	IN	A	10.10.30.253

;; Query time: 33 msec
;; SERVER: 10.10.20.253#53(10.10.20.253) (UDP)
;; WHEN: Fri Jun 20 13:07:39 CEST 2025
;; MSG SIZE  rcvd: 161

The resolution is broken on client level, just for sure I've added a netbird.anon-ZmCZH.domain. record on my DNS server (thats just a NS record for Cloudflare domain):

dig netbird.anon-ZmCZH.domain.                                                                                                                                                               

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> netbird.anon-ZmCZH.domain.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 40036
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;netbird.anon-ZmCZH.domain..		IN	A

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Fri Jun 20 13:21:37 CEST 2025
;; MSG SIZE  rcvd: 47


dig @10.10.20.253 netbird.anon-ZmCZH.domain.                                                                                                                                                  

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> @10.10.20.253 netbird.anon-ZmCZH.domain.
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65409
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: bcbecf53da95a288f5add07568554492eba8f773f270ccaa (good)
;; QUESTION SECTION:
;netbird.anon-ZmCZH.domain.		IN	A

;; ANSWER SECTION:
netbird.anon-ZmCZH.domain.	275	IN	A	<NETBIRD PUBLIC IP>

;; AUTHORITY SECTION:
netbird.anon-ZmCZH.domain.	86400	IN	NS	rose.ns.cloudflare.com.
netbird.anon-ZmCZH.domain.	86400	IN	NS	sid.ns.cloudflare.com.

;; Query time: 34 msec
;; SERVER: 10.10.20.253#53(10.10.20.253) (UDP)
;; WHEN: Fri Jun 20 13:22:58 CEST 2025
;; MSG SIZE  rcvd: 142

Debug Files

netbird.debug-0.46.zip
netbird.debug-0.48.zip

Additional context

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • Disabled other VPN software
  • Checked firewall settings
Originally created by @Jachimoo on GitHub (Jun 20, 2025). **Describe the problem** After updating client on ubuntu 22.04, the DNS resolution stopped working. The DNS nameservers for domain are configured and accessible but client **To Reproduce** Steps to reproduce the behavior: 1. Have a selfhosted netbird instance working as bastion host, with DNS servers that extends our public domain to internal servers 2. Configure Nameservers for client group to be set as internal servers for our domain (We use FreeIPA) 3. Client v0.46 works fine 4. Client v0.48 in unable to use internal DNS (Logs says that there is a timeout, but if we check manually the DNS server is accessible and resolves names) **Expected behavior** The nameservers are used without any problems **Are you using NetBird Cloud?** No I'm using selfhosted **NetBird version** Server: - Control plane: latest as of 20.06.2025 - Client: 0.45.1 Client: Works: 0.46.0 Broken: 0.48.0 **Is any other VPN software installed?** Nope **Debug output (domain obfustated)** client 0.46: ``` Peers detail: <some-client>anon-ZmCZH.domain: NetBird IP: 100.125.60.209/32 Public key: VZMrvHq28JZD2xB7G+EuJ2QpME+vQVgOFfpjBrQRumI= Status: Idle -- detail -- Connection type: ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: Last connection update: 1 minute, 34 seconds ago Last WireGuard handshake: - Transfer status (received/sent) 0 B/0 B Quantum resistance: false Networks: - Latency: 0s <some-client>.anon-ZmCZH.domain: NetBird IP: 100.125.36.234/32 Public key: 0NguuRsxno4LMfvlkljqOhkFdXLlEPAigLiKOP3C308= Status: Idle -- detail -- Connection type: ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: Last connection update: 1 minute, 34 seconds ago Last WireGuard handshake: - Transfer status (received/sent) 0 B/0 B Quantum resistance: false Networks: - Latency: 0s <routing_peer/control_plane>.anon-ZmCZH.domain: NetBird IP: 100.125.37.253 Public key: cl2RJM9i3oqDb0422dzOa1SOU+9OXXV5zHKZrGR9u1c= Status: Connected -- detail -- Connection type: Relayed ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: rel://netbird.anon-ZmCZH.domain:33080 Last connection update: 1 minute, 33 seconds ago Last WireGuard handshake: 1 minute, 28 seconds ago Transfer status (received/sent) 9.9 KiB/8.5 KiB Quantum resistance: false Networks: 10.10.0.0/16, 10.100.0.0/16, 198.51.100.0/32 Latency: 0s <some-cient>anon-ZmCZH.domain: NetBird IP: 100.125.51.179 Public key: Dygbc4m2GoCyJClp6mCEs7tMg3e4gSnDgE8B+HKA9AY= Status: Connected -- detail -- Connection type: P2P ICE candidate (Local/Remote): host/srflx ICE candidate endpoints (Local/Remote): 192.168.122.1:51820/198.51.100.1:16135 Relay server address: rel://netbird.anon-ZmCZH.domain:33080 Last connection update: 1 minute, 33 seconds ago Last WireGuard handshake: 1 minute, 28 seconds ago Transfer status (received/sent) 424 B/364 B Quantum resistance: false Networks: - Latency: 14.349299ms <routing_peer>.anon-ZmCZH.domain: NetBird IP: 100.125.111.182 Public key: kOOSzYtKp3kNB7vbgM6ge06DdIHPCds3a3YepEJFR2Q= Status: Connected -- detail -- Connection type: Relayed ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: rel://netbird.anon-ZmCZH.domain:33080 Last connection update: 1 minute, 4 seconds ago Last WireGuard handshake: 41 seconds ago Transfer status (received/sent) 572 B/488 B Quantum resistance: false Networks: 172.16.0.0/16, anon-LNzcq.domain Latency: 29.403567ms Events: [WARNING] DNS (6560e27f-0c7b-441a-bd86-aba3c2b48691) Message: All upstream servers failed (probe failed) Time: 1 minute, 34 seconds ago Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53 [WARNING] DNS (1f12b277-6c07-4982-9947-40584e5d2080) Message: All upstream servers failed (probe failed) Time: 1 minute, 34 seconds ago Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53 [WARNING] DNS (e8058d77-f32e-4d00-82bf-09f142e2b958) Message: All upstream servers failed (probe failed) Time: 1 minute, 34 seconds ago Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53 [INFO] SYSTEM (11ddcb9b-4c1f-41f7-8a8e-bb1f35a8a926) Message: Network map updated Time: 1 minute, 34 seconds ago OS: linux/amd64 Daemon version: 0.46.0 CLI version: 0.46.0 Management: Connected to https://netbird.anon-ZmCZH.domain:443 Signal: Connected to https://netbird.anon-ZmCZH.domain:443 Relays: [stun:netbird.anon-ZmCZH.domain:3478] is Available [turn:netbird.anon-ZmCZH.domain:3478?transport=udp] is Available [rel://netbird.anon-ZmCZH.domain:33080] is Available Nameservers: [10.10.20.253:53, 10.10.30.253:53] for [anon-ZmCZH.domain, anon-LNzcq.domain, anon-Xm8Aa.domain] is Available FQDN: <client1>.anon-ZmCZH.domain NetBird IP: 100.125.212.195/16 Interface type: Kernel Quantum resistance: false Lazy connection: false Networks: - Forwarding rules: 0 Peers count: 3/5 Connected dig <some-app>.anon-ZmCZH.domain ; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> <some-app>.anon-ZmCZH.domain ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36373 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;<some-app>.anon-ZmCZH.domain. IN A ;; ANSWER SECTION: <some-app>.anon-ZmCZH.domain. 1200 IN A 10.100.x.x //obfustated cuz dont need ;; AUTHORITY SECTION: anon-ZmCZH.domain 86400 IN NS <FreeIpa server> anon-ZmCZH.domain 86400 IN NS <FreeIpa server 2> ;; ADDITIONAL SECTION: <FreeIpa server> 1200 IN A 10.10.20.253 <FreeIpa server 2> 1200 IN A 10.10.30.253 ;; Query time: 32 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP) ;; WHEN: Fri Jun 20 12:24:18 CEST 2025 ;; MSG SIZE rcvd: 133 ``` client 0.48.: ``` Peers detail: <some-client>.anon-w4kgQ.domain: NetBird IP: 100.125.60.209/32 Public key: VZMrvHq28JZD2xB7G+EuJ2QpME+vQVgOFfpjBrQRumI= Status: Idle -- detail -- Connection type: ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: Last connection update: 24 seconds ago Last WireGuard handshake: - Transfer status (received/sent) 0 B/0 B Quantum resistance: false Networks: - Latency: 0s <some-client>.anon-w4kgQ.domain: NetBird IP: 100.125.36.234/32 Public key: 0NguuRsxno4LMfvlkljqOhkFdXLlEPAigLiKOP3C308= Status: Idle -- detail -- Connection type: ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: Last connection update: 24 seconds ago Last WireGuard handshake: - Transfer status (received/sent) 0 B/0 B Quantum resistance: false Networks: - Latency: 0s <routing_peer/control_plane>.anon-w4kgQ.domain: NetBird IP: 100.125.37.253 Public key: cl2RJM9i3oqDb0422dzOa1SOU+9OXXV5zHKZrGR9u1c= Status: Connected -- detail -- Connection type: Relayed ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: rel://netbird.anon-w4kgQ.domain:33080 Last connection update: 24 seconds ago Last WireGuard handshake: 18 seconds ago Transfer status (received/sent) 7.2 KiB/5.9 KiB Quantum resistance: false Networks: 10.10.0.0/16, 10.100.0.0/16, 198.51.100.0/32 Latency: 0s <some-client>.anon-w4kgQ.domain: NetBird IP: 100.125.51.179 Public key: Dygbc4m2GoCyJClp6mCEs7tMg3e4gSnDgE8B+HKA9AY= Status: Connected -- detail -- Connection type: P2P ICE candidate (Local/Remote): host/prflx ICE candidate endpoints (Local/Remote): 192.168.122.1:51820/198.51.100.1:4055 Relay server address: rel://netbird.anon-w4kgQ.domain:33080 Last connection update: 22 seconds ago Last WireGuard handshake: 18 seconds ago Transfer status (received/sent) 328 B/364 B Quantum resistance: false Networks: - Latency: 13.074887ms <routing_peer>.anon-w4kgQ.domain: NetBird IP: 100.125.111.182 Public key: kOOSzYtKp3kNB7vbgM6ge06DdIHPCds3a3YepEJFR2Q= Status: Connected -- detail -- Connection type: P2P ICE candidate (Local/Remote): host/prflx ICE candidate endpoints (Local/Remote): 198.51.100.2:51820/198.51.100.0:51820 Relay server address: rel://netbird.anon-w4kgQ.domain:33080 Last connection update: 22 seconds ago Last WireGuard handshake: 23 seconds ago Transfer status (received/sent) 180 B/272 B Quantum resistance: false Networks: 172.16.0.0/16, anon-xfIRx.domain Latency: 33.40652ms Events: [WARNING] DNS (1e0ef040-2d12-4c99-9b29-5a9d0d5ceef8) Message: All upstream servers failed (probe failed) Time: 24 seconds ago Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53 [WARNING] DNS (783bb17c-2959-44a6-91e2-4d8448fa278f) Message: All upstream servers failed (probe failed) Time: 24 seconds ago Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53 [WARNING] DNS (5e08b96c-38ba-483f-b945-867d74721478) Message: All upstream servers failed (probe failed) Time: 24 seconds ago Metadata: upstreams: 10.10.20.253:53, 10.10.30.253:53 [INFO] SYSTEM (2b6f86a9-90e7-42d9-a67a-572a542bb26e) Message: Network map updated Time: 24 seconds ago OS: linux/amd64 Daemon version: 0.48.0 CLI version: 0.48.0 Management: Connected to https://netbird.anon-w4kgQ.domain:443 Signal: Connected to https://netbird.anon-w4kgQ.domain:443 Relays: [stun:netbird.anon-w4kgQ.domain:3478] is Unavailable, reason: dial: failed to listen: d.Dialer.DialContext: dial udp: lookup netbird.anon-w4kgQ.domain on 127.0.0.53:53: no such host [turn:netbird.anon-w4kgQ.domain:3478?transport=udp] is Unavailable, reason: create client: lookup netbird.anon-w4kgQ.domain on 127.0.0.53:53: no such host [rel://netbird.anon-w4kgQ.domain:33080] is Available Nameservers: [10.10.20.253:53, 10.10.30.253:53] for [anon-w4kgQ.domain, anon-xfIRx.domain, anon-OQqQX.domain] is Available FQDN: gabriel-xps-13-7390.anon-w4kgQ.domain NetBird IP: 100.125.212.195/16 Interface type: Kernel Quantum resistance: false Lazy connection: false Networks: - Forwarding rules: 0 Peers count: 3/5 Connected dig <some-app>.anon-ZmCZH.domain ; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> <some-app>.anon-ZmCZH.domain ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 47495 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ; <some-app>.anon-ZmCZH.domain IN A ;; Query time: 1 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP) ;; WHEN: Fri Jun 20 13:07:23 CEST 2025 ;; MSG SIZE rcvd: 45 dig @10.10.20.253 <some-app>.anon-ZmCZH.domain ; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> @10.10.20.253 <some-app>.anon-ZmCZH.domain ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24850 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ; COOKIE: 1a4fe68f062fd27233556262685540fbea66c3affa429eeb (good) ;; QUESTION SECTION: ;<some-app>.anon-ZmCZH.domain. IN A ;; ANSWER SECTION: <some-app>.anon-ZmCZH.domain 1200 IN A 10.100.x.x //obfustated cuz dont need ;; AUTHORITY SECTION: anon-ZmCZH.domain 86400 IN NS <FreeIpa server> anon-ZmCZH.domain 86400 IN NS <FreeIpa server 2> ;; ADDITIONAL SECTION: <FreeIpa server> 1200 IN A 10.10.20.253 <FreeIpa server 2> 1200 IN A 10.10.30.253 ;; Query time: 33 msec ;; SERVER: 10.10.20.253#53(10.10.20.253) (UDP) ;; WHEN: Fri Jun 20 13:07:39 CEST 2025 ;; MSG SIZE rcvd: 161 ``` The resolution is broken on client level, just for sure I've added a netbird.anon-ZmCZH.domain. record on my DNS server (thats just a NS record for Cloudflare domain): ``` dig netbird.anon-ZmCZH.domain. ; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> netbird.anon-ZmCZH.domain. ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 40036 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;netbird.anon-ZmCZH.domain.. IN A ;; Query time: 0 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP) ;; WHEN: Fri Jun 20 13:21:37 CEST 2025 ;; MSG SIZE rcvd: 47 dig @10.10.20.253 netbird.anon-ZmCZH.domain. ; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> @10.10.20.253 netbird.anon-ZmCZH.domain. ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65409 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ; COOKIE: bcbecf53da95a288f5add07568554492eba8f773f270ccaa (good) ;; QUESTION SECTION: ;netbird.anon-ZmCZH.domain. IN A ;; ANSWER SECTION: netbird.anon-ZmCZH.domain. 275 IN A <NETBIRD PUBLIC IP> ;; AUTHORITY SECTION: netbird.anon-ZmCZH.domain. 86400 IN NS rose.ns.cloudflare.com. netbird.anon-ZmCZH.domain. 86400 IN NS sid.ns.cloudflare.com. ;; Query time: 34 msec ;; SERVER: 10.10.20.253#53(10.10.20.253) (UDP) ;; WHEN: Fri Jun 20 13:22:58 CEST 2025 ;; MSG SIZE rcvd: 142 ``` **Debug Files** [netbird.debug-0.46.zip](https://github.com/user-attachments/files/20835719/netbird.debug-0.46.zip) [netbird.debug-0.48.zip](https://github.com/user-attachments/files/20835720/netbird.debug-0.48.zip) **Additional context** **Have you tried these troubleshooting steps?** - [x] Reviewed [client troubleshooting](https://docs.netbird.io/how-to/troubleshooting-client) (if applicable) - [x] Checked for newer NetBird versions - [x] Searched for similar issues on GitHub (including closed ones) - [x] Restarted the NetBird client - [x] Disabled other VPN software - [x] Checked firewall settings
saavagebueno added the triage-needed label 2025-11-20 06:10:43 -05:00
saavagebueno commented 2025-11-20 06:10:44 -05:00
Author
Owner
Copy Link

@Jachimoo commented on GitHub (Jun 23, 2025):

Note, in ubuntu we made workaround by adding:

"CustomDNSAddress": "10.10.20.253:53",

into /etc/netbird/config.json (idk, if we can add second DNS here).
Unfortunatly the issue is also present in MacOS. Where DNS resolution stop working and it doesn't matter which client version is installed, but maybe this is due to server update that we done? so right now i am not sure what causes the problems

@Jachimoo commented on GitHub (Jun 23, 2025): Note, in ubuntu we made workaround by adding: ``` "CustomDNSAddress": "10.10.20.253:53", ``` into `/etc/netbird/config.json` (idk, if we can add second DNS here). Unfortunatly the issue is also present in MacOS. Where DNS resolution stop working and it doesn't matter which client version is installed, but maybe this is due to server update that we done? so right now i am not sure what causes the problems
saavagebueno commented 2025-11-20 06:10:44 -05:00
Author
Owner
Copy Link

@lixmal commented on GitHub (Jun 23, 2025):

@Jachimoo you're routing the internal dns domain somewhere else:

[10.10.20.253:53, 10.10.30.253:53] for [anon-Ke6km.domain, ...] is Available
FQDN: gabriel-xps-13-7390.anon-Ke6km.domain

Either don't use it in match domains or change the internal domain to something else, a subdomain of what you have configured right now will also do.

<v0.48.0 lacked this validation unfortunately

@lixmal commented on GitHub (Jun 23, 2025): @Jachimoo you're routing the internal dns domain somewhere else: > [10.10.20.253:53, 10.10.30.253:53] for [anon-Ke6km.domain, ...] is Available > FQDN: gabriel-xps-13-7390.anon-Ke6km.domain Either don't use it in match domains or change the internal domain to something else, a subdomain of what you have configured right now will also do. <v0.48.0 lacked this validation unfortunately
saavagebueno commented 2025-11-20 06:10:44 -05:00
Author
Owner
Copy Link

@Jachimoo commented on GitHub (Jun 23, 2025):

Indeed, changing internal domain, and disabling match domain resolved the issue, big thanks!

@Jachimoo commented on GitHub (Jun 23, 2025): Indeed, changing internal domain, and disabling match domain resolved the issue, big thanks!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label triage-needed
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#1988
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?