DNS Resolution Debian 11 and 12 not working as expected - Name or Service not known #1043

Closed
opened 2025-11-20 05:22:59 -05:00 by saavagebueno · 10 comments
Owner

Originally created by @VanLampe on GitHub (Jul 5, 2024).

Hi together,

I am currently trying the selfhosted installation of netbird. So far everything is working properly, but I have some issues with the dns resolution.

It seems, that the dns resolver is not resolving the fqdn of connected peers under debian VMs.

I have some peers added to the netbird mesh but I am not able to reach a peer by its fqdn from the linux / debian hosts. On Android and Windows, I can call the peers via fqdn.

ping app.netbird.selfhosted
ping: app.netbird.selfhosted: Name or service not known

ssh app.netbird.selfhosted
ssh: Could not resolve hostname app.netbird.selfhosted: Name or service not known

Connecting via the overlay IP is working without issues:

ping 100.69.131.193
PING 100.69.131.193 (100.69.131.193) 56(84) bytes of data.
64 bytes from 100.69.131.193: icmp_seq=1 ttl=64 time=0.351 ms
64 bytes from 100.69.131.193: icmp_seq=2 ttl=64 time=0.262 ms
^C
--- 100.69.131.193 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1029ms

Also using dig, the fqdn seems to resolve correctly:

dig app.netbird.selfhosted

; <<>> DiG 9.18.24-1-Debian <<>> app.netbird.selfhosted
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4865
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;app.netbird.selfhosted.                IN      A

;; ANSWER SECTION:
app.netbird.selfhosted. 300     IN      A       100.69.131.193

;; Query time: 3 msec
;; SERVER: 100.69.179.157#53(100.69.179.157) (UDP)
;; WHEN: Fri Jul 05 12:00:49 UTC 2024
;; MSG SIZE  rcvd: 78

/etc/resolv.conf

# Generated by NetBird
# If needed you can restore the original file by copying back /etc/resolv.conf.original.netbird

options timeout:4 attempts:1
search netbird.selfhosted local
nameserver 100.69.179.157
nameserver <ip of my local dns server>
nameserver 8.8.8.8

I recognized, that the resolv.conf only gets updated, if I reconnect the netbird agent after rebooting the VM (netbird down and up). I think this is caused by the proxmox hypervisor, that is applying the global dns settings for my proxmox hosts.

Can anyone help me to solve this issue?

Thanks and best regards,
Frank

Originally created by @VanLampe on GitHub (Jul 5, 2024). Hi together, I am currently trying the selfhosted installation of netbird. So far everything is working properly, but I have some issues with the dns resolution. It seems, that the dns resolver is not resolving the fqdn of connected peers under debian VMs. I have some peers added to the netbird mesh but I am not able to reach a peer by its fqdn from the linux / debian hosts. On Android and Windows, I can call the peers via fqdn. ``` ping app.netbird.selfhosted ping: app.netbird.selfhosted: Name or service not known ssh app.netbird.selfhosted ssh: Could not resolve hostname app.netbird.selfhosted: Name or service not known ``` Connecting via the overlay IP is working without issues: ``` ping 100.69.131.193 PING 100.69.131.193 (100.69.131.193) 56(84) bytes of data. 64 bytes from 100.69.131.193: icmp_seq=1 ttl=64 time=0.351 ms 64 bytes from 100.69.131.193: icmp_seq=2 ttl=64 time=0.262 ms ^C --- 100.69.131.193 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1029ms ``` Also using `dig`, the fqdn seems to resolve correctly: ``` dig app.netbird.selfhosted ; <<>> DiG 9.18.24-1-Debian <<>> app.netbird.selfhosted ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4865 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;app.netbird.selfhosted. IN A ;; ANSWER SECTION: app.netbird.selfhosted. 300 IN A 100.69.131.193 ;; Query time: 3 msec ;; SERVER: 100.69.179.157#53(100.69.179.157) (UDP) ;; WHEN: Fri Jul 05 12:00:49 UTC 2024 ;; MSG SIZE rcvd: 78 ``` `/etc/resolv.conf` ``` # Generated by NetBird # If needed you can restore the original file by copying back /etc/resolv.conf.original.netbird options timeout:4 attempts:1 search netbird.selfhosted local nameserver 100.69.179.157 nameserver <ip of my local dns server> nameserver 8.8.8.8 ``` I recognized, that the `resolv.conf` only gets updated, if I reconnect the netbird agent after rebooting the VM (netbird down and up). I think this is caused by the proxmox hypervisor, that is applying the global dns settings for my proxmox hosts. Can anyone help me to solve this issue? Thanks and best regards, Frank
saavagebueno added the dns label 2025-11-20 05:22:59 -05:00
Author
Owner

@pappz commented on GitHub (Jul 8, 2024):

@VanLampe If you configure the nameserver in the NetBird Dashboard then it should work well.

@pappz commented on GitHub (Jul 8, 2024): @VanLampe If you configure the nameserver in the NetBird Dashboard then it should work well.
Author
Owner

@VanLampe commented on GitHub (Jul 8, 2024):

Hi @pappz,

I already tried to configure a nameserver for all peers, but that did not work:

image

@VanLampe commented on GitHub (Jul 8, 2024): Hi @pappz, I already tried to configure a nameserver for all peers, but that did not work: ![image](https://github.com/netbirdio/netbird/assets/55480780/518d6d9e-a042-4325-b741-3e15a322034f)
Author
Owner

@itoffshore commented on GitHub (Jul 9, 2024):

  • to get my *.netbird.cloud hosts to resolve under systemd-resolved I set the following
  • where DNS is the ip address of my wt0 interface:
# /etc/systemd/network/20-eth0.network

[Match]
Name=eth0

[Network]
DHCP=ipv4
DNS=100.x.x.x
Domains=~netbird.cloud

[DHCPv4]
UseDNS=false

This solution works:

  • running netbird in privileged podman containers
  • running netbird as an RPM package in MicroOS
@itoffshore commented on GitHub (Jul 9, 2024): * to get my `*.netbird.cloud` hosts to resolve under `systemd-resolved` I set the following * where `DNS` is the ip address of my `wt0` interface: ``` # /etc/systemd/network/20-eth0.network [Match] Name=eth0 [Network] DHCP=ipv4 DNS=100.x.x.x Domains=~netbird.cloud [DHCPv4] UseDNS=false ``` This solution works: - running `netbird` in privileged `podman` containers - running `netbird` as an **RPM** package in MicroOS
Author
Owner

@mlsmaycon commented on GitHub (Jul 10, 2024):

@VanLampe can you share your original /etc/resolv.conf with us? we have a known bug with debian's resolvconf and it seems to be the case with your setup.

@mlsmaycon commented on GitHub (Jul 10, 2024): @VanLampe can you share your original /etc/resolv.conf with us? we have a known bug with debian's resolvconf and it seems to be the case with your setup.
Author
Owner

@Marcus1Pierce commented on GitHub (Jul 22, 2024):

@mlsmaycon I have same error when i ping to other client from debian (Ex. ping ldap.overlay.netbird i got Name or service not known.

Looks like netbird not update my /etc/resolv.conf

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "resolvectl status" to see details about the actual nameservers.

nameserver 1.1.1.2

And i got error on /var/log/netbird/client.log

2024-07-23T09:19:54+07:00 ERRO client/internal/dns/server.go:196: removing resolvconf configuration for wt0 interface, error: exit status 99
2024-07-23T09:19:54+07:00 ERRO iface/bind/udp_mux_universal.go:93: error while reading packet: shared socked stopped
2024-07-23T09:19:56+07:00 WARN client/internal/dns/unclean_shutdown_unix.go:27: detected unclean shutdown, file /var/lib/netbird/resolv.conf exists
2024-07-23T09:19:56+07:00 WARN client/internal/dns/unclean_shutdown_unix.go:45: restoring unclean shutdown dns settings via previously detected manager: resolvconf
2024-07-23T09:19:56+07:00 ERRO client/internal/connect.go:124: checking unclean shutdown error: restore unclean shutdown backup: restoring dns for interface wt0: removing resolvconf configuration for wt0 interface, error: exit status 99
2024-07-23T09:19:58+07:00 WARN client/internal/engine.go:602: running SSH server is not permitted
2024-07-23T09:19:58+07:00 ERRO client/internal/dns/server.go:322: apply config: applying resolvconf configuration for wt0 interface, error: exit status 99
2024-07-23T09:19:58+07:00 WARN client/internal/dns/upstream.go:196: probing upstream nameserver 100.111.134.235:53: write udp 100.111.114.92:35236->100.111.134.235:53: write: required key not available
2024-07-23T09:19:58+07:00 WARN client/internal/dns/upstream.go:275: Upstream resolving is Disabled for 30s
2024-07-23T09:19:58+07:00 ERRO [nameservers: [{100.111.134.235 udp 53}]] client/internal/dns/server.go:525: Failed to apply nameserver deactivation on the host: apply config: applying resolvconf configuration for wt0 interface, error: exit status 99
2024-07-23T09:20:08+07:00 ERRO [nameservers: [{100.111.134.235 udp 53}], error: apply config: applying resolvconf configuration for wt0 interface, error: exit status 99] client/internal/dns/server.go:555: reactivate temporary disabled nameserver group, DNS update apply

And config from /var/lib/netbird/resolv.conf

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "resolvectl status" to see details about the actual nameservers.

nameserver 1.1.1.2

Nevermind. I fix it with remove resolvconf package and restart the client.

@Marcus1Pierce commented on GitHub (Jul 22, 2024): @mlsmaycon I have same error when i ping to other client from debian (Ex. `ping ldap.overlay.netbird` i got Name or service not known. Looks like netbird not update my /etc/resolv.conf ``` # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN # 127.0.0.53 is the systemd-resolved stub resolver. # run "resolvectl status" to see details about the actual nameservers. nameserver 1.1.1.2 ``` And i got error on /var/log/netbird/client.log ``` 2024-07-23T09:19:54+07:00 ERRO client/internal/dns/server.go:196: removing resolvconf configuration for wt0 interface, error: exit status 99 2024-07-23T09:19:54+07:00 ERRO iface/bind/udp_mux_universal.go:93: error while reading packet: shared socked stopped 2024-07-23T09:19:56+07:00 WARN client/internal/dns/unclean_shutdown_unix.go:27: detected unclean shutdown, file /var/lib/netbird/resolv.conf exists 2024-07-23T09:19:56+07:00 WARN client/internal/dns/unclean_shutdown_unix.go:45: restoring unclean shutdown dns settings via previously detected manager: resolvconf 2024-07-23T09:19:56+07:00 ERRO client/internal/connect.go:124: checking unclean shutdown error: restore unclean shutdown backup: restoring dns for interface wt0: removing resolvconf configuration for wt0 interface, error: exit status 99 2024-07-23T09:19:58+07:00 WARN client/internal/engine.go:602: running SSH server is not permitted 2024-07-23T09:19:58+07:00 ERRO client/internal/dns/server.go:322: apply config: applying resolvconf configuration for wt0 interface, error: exit status 99 2024-07-23T09:19:58+07:00 WARN client/internal/dns/upstream.go:196: probing upstream nameserver 100.111.134.235:53: write udp 100.111.114.92:35236->100.111.134.235:53: write: required key not available 2024-07-23T09:19:58+07:00 WARN client/internal/dns/upstream.go:275: Upstream resolving is Disabled for 30s 2024-07-23T09:19:58+07:00 ERRO [nameservers: [{100.111.134.235 udp 53}]] client/internal/dns/server.go:525: Failed to apply nameserver deactivation on the host: apply config: applying resolvconf configuration for wt0 interface, error: exit status 99 2024-07-23T09:20:08+07:00 ERRO [nameservers: [{100.111.134.235 udp 53}], error: apply config: applying resolvconf configuration for wt0 interface, error: exit status 99] client/internal/dns/server.go:555: reactivate temporary disabled nameserver group, DNS update apply ``` And config from /var/lib/netbird/resolv.conf ``` # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN # 127.0.0.53 is the systemd-resolved stub resolver. # run "resolvectl status" to see details about the actual nameservers. nameserver 1.1.1.2 ``` **Nevermind. I fix it with remove resolvconf package and restart the client.**
Author
Owner

@fruworg commented on GitHub (Aug 10, 2024):

@VanLampe can you share your original /etc/resolv.conf with us? we have a known bug with debian's resolvconf and it seems to be the case with your setup.

Hi! I have the same problem.

$ ping server-1.netbird.cloud
ping: server-1.netbird.cloud: Name or service not known
$ cat /etc/resolv.conf
# Generated by NetBird
# If needed you can restore the original file by copying back /etc/resolv.conf.original.netbird

options timeout:4 attempts:1
search netbird.cloud .
nameserver <machine-netbird-ip>
nameserver 169.254.169.254

изображение

@fruworg commented on GitHub (Aug 10, 2024): > @VanLampe can you share your original /etc/resolv.conf with us? we have a known bug with debian's resolvconf and it seems to be the case with your setup. Hi! I have the same problem. ``` $ ping server-1.netbird.cloud ping: server-1.netbird.cloud: Name or service not known ``` ``` $ cat /etc/resolv.conf # Generated by NetBird # If needed you can restore the original file by copying back /etc/resolv.conf.original.netbird options timeout:4 attempts:1 search netbird.cloud . nameserver <machine-netbird-ip> nameserver 169.254.169.254 ``` ![изображение](https://github.com/user-attachments/assets/79e6b746-99eb-4ad2-af60-f9ee8c324bb9)
Author
Owner

@eadwu commented on GitHub (Sep 9, 2024):

Under systemd-resolved the DNS configuration doesn't really work.

Link X ()
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR +mDNS +DNSOverTLS
                    DNSSEC=no/unsupported
Current DNS Server: ...
       DNS Servers: ...
        DNS Domain: netbird.cloud

It only configures the search domain but no routing domain ~netbird.cloud

@eadwu commented on GitHub (Sep 9, 2024): Under systemd-resolved the DNS configuration doesn't really work. ``` Link X () Current Scopes: DNS Protocols: +DefaultRoute -LLMNR +mDNS +DNSOverTLS DNSSEC=no/unsupported Current DNS Server: ... DNS Servers: ... DNS Domain: netbird.cloud ``` It only configures the search domain but no routing domain `~netbird.cloud`
Author
Owner

@itoffshore commented on GitHub (Sep 10, 2024):

Configuring networkd as I do - if resolved or networkd is enabled in Linux - would make DNS a bit more reliable out of the box (resolved reads networkd configuration)

A standard configuration that works everywhere systemd exists (without any extra tools) - would means a lot less time spent on DNS issues.

Netbird DNS configured this way works side by side with my internal custom DNS provided by Knot DNS containers across 2 x clusters (configured as custom nameservers in netbird)

In my situation a single secondary authoritative custom DNS in each cluster is good enough (as resolved is caching DNS) - this setup has been completely reliable & survives daily reboots under MicroOS

@itoffshore commented on GitHub (Sep 10, 2024): Configuring `networkd` [as I do](https://github.com/netbirdio/netbird/issues/2239#issuecomment-2218970703) - if `resolved` or `networkd` is enabled in Linux - would make DNS a bit more reliable out of the box (`resolved` reads `networkd` configuration) A standard configuration that works everywhere `systemd` exists (without any extra tools) - would means a lot less time spent on DNS issues. Netbird DNS configured this way works side by side with my internal custom DNS provided by Knot DNS containers across 2 x clusters (configured as custom nameservers in netbird) In my situation a single secondary authoritative custom DNS in each cluster is good enough (as `resolved` is caching DNS) - this setup has been completely reliable & survives daily reboots under MicroOS
Author
Owner

@VanLampe commented on GitHub (Oct 7, 2024):

Hi together,

last weekend I completly reinstalled netbird and still experiencing the same DNS problems within my proxmox VMs.

I have also installed the netbird client on an external VPS outside my Proxmox hypervisor as a test. There the DNS resolution works wihtout issues (also debian os).

So it must be something with my proxmox DNS configuration.

I will play arround with the configuration and post an update here later.

@VanLampe commented on GitHub (Oct 7, 2024): Hi together, last weekend I completly reinstalled netbird and still experiencing the same DNS problems within my proxmox VMs. I have also installed the netbird client on an external VPS outside my Proxmox hypervisor as a test. There the DNS resolution works wihtout issues (also debian os). So it must be something with my proxmox DNS configuration. I will play arround with the configuration and post an update here later.
Author
Owner

@VanLampe commented on GitHub (Oct 7, 2024):

After reinstalling netbird, I did not set a global nameserver for my peers again. After setting this, dns resolution is working fine now with the latest agents.

The resolv.conf is now also configured properly by the netbird agents. I have no idea, why it was working for my remote VPS without setting global nameserver - but nevermind.

@VanLampe commented on GitHub (Oct 7, 2024): After reinstalling netbird, I did not set a global nameserver for my peers again. After setting this, dns resolution is working fine now with the latest agents. The `resolv.conf` is now also configured properly by the netbird agents. I have no idea, why it was working for my remote VPS without setting global nameserver - but nevermind.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#1043