The new Netbird DNS Resolver Forced activation cause more problems that the ones that solve #2133

@buzzzo commented on GitHub (Jul 30, 2025):

To add a just discovered other side effect.
Netbird add its own domain BEFORE the system one (eg: vpn.foo.org is netbird domain and foo.org is domain).
So if BEFORE i could reach mickey.foo.org just typing mickey AFTER the netbird addition result in automatically resolver mickey in michey.vpn.foo.org).

This is fine for desktop system but totally wrong for servers.
Please make this feature as selectable.

@buzzzo commented on GitHub (Jul 30, 2025): To add a just discovered other side effect. Netbird add its own domain BEFORE the system one (eg: vpn.foo.org is netbird domain and foo.org is domain). So if BEFORE i could reach mickey.foo.org just typing mickey AFTER the netbird addition result in automatically resolver mickey in michey.vpn.foo.org). This is fine for desktop system but totally wrong for servers. Please make this feature as selectable.

saavagebueno commented

@buzzzo commented on GitHub (Jul 30, 2025):

Seems that 0.52.1 fixed the possibility to disable dns management with peer group.

@buzzzo commented on GitHub (Jul 30, 2025): Seems that 0.52.1 fixed the possibility to disable dns management with peer group.

saavagebueno commented

@LordAnchemis commented on GitHub (Jul 30, 2025):

TLDR: 0.52.1 has buggy DNS resolution

For me it was apt update failing as DNS couldn't resolve debian.org (lol)

@LordAnchemis commented on GitHub (Jul 30, 2025): TLDR: 0.52.1 has buggy DNS resolution For me it was apt update failing as DNS couldn't resolve debian.org (lol)

saavagebueno commented

@Pamalosebi commented on GitHub (Jul 30, 2025):

When netbird 0.52.1 is activated on my systems I get stuff like

dig github.com
dig: parse of /etc/resolv.conf failed

And It resolves when I disable netbird
sudo netbird down

@Pamalosebi commented on GitHub (Jul 30, 2025): When netbird 0.52.1 is activated on my systems I get stuff like ``` dig github.com dig: parse of /etc/resolv.conf failed ``` And It resolves when I disable netbird `sudo netbird down`

saavagebueno commented

@nazarewk commented on GitHub (Jul 30, 2025):

Can I ask you to enable trace logs ("permanently", so it starts with those) and gather debug bundles? You can try netbird debug bundle -U, but feel free to mail those to support@netbird.io if the upload fails.

See https://docs.netbird.io/how-to/troubleshooting-client#on-other-linux-and-mac-os for instructions on changing log levels

@nazarewk commented on GitHub (Jul 30, 2025): Can I ask you to enable `trace` logs ("permanently", so it starts with those) and gather debug bundles? You can try `netbird debug bundle -U`, but feel free to mail those to support@netbird.io if the upload fails. See https://docs.netbird.io/how-to/troubleshooting-client#on-other-linux-and-mac-os for instructions on changing log levels

saavagebueno commented

@buzzzo commented on GitHub (Jul 30, 2025):

When netbird 0.52.1 is activated on my systems I get stuff like
dig github.com
dig: parse of /etc/resolv.conf failed
And It resolves when I disable netbird sudo netbird down

This is because netbird creates a bogus config and the system (linux itself) cannot parse it correctly. The solution would be to upgrade netbird to the latest version AND add the linux peers to the dns exclusion. This seems (at least on my systems) to ignore completely the netbird dns rules and revert to the original resolv.conf.

You could also try to create a dns rules (on the dashboard) with your internal dns servers matching group "all". In this way you are telling to netbird to redirect ALL dns request to your dns server, but in any case netbird rewrite the resolv.conf of your system adding as nameserver its netbird peer ip.

@buzzzo commented on GitHub (Jul 30, 2025): > When netbird 0.52.1 is activated on my systems I get stuff like > > ``` > dig github.com > dig: parse of /etc/resolv.conf failed > ``` > > And It resolves when I disable netbird `sudo netbird down` This is because netbird creates a bogus config and the system (linux itself) cannot parse it correctly. The solution would be to upgrade netbird to the latest version AND add the linux peers to the dns exclusion. This seems (at least on my systems) to ignore completely the netbird dns rules and revert to the original resolv.conf. You could also try to create a dns rules (on the dashboard) with your internal dns servers matching group "all". In this way you are telling to netbird to redirect ALL dns request to your dns server, but in any case netbird rewrite the resolv.conf of your system adding as nameserver its netbird peer ip.

saavagebueno commented

@Pamalosebi commented on GitHub (Jul 30, 2025):

My system was unusable so I reverted to an older version... That did work.
I did put netbird in my package managers (pacman) exclusion list for now.

sudo podman -U /var/cache/pacman/pkg/netbird-0.51.2-1-x86_64.pkg.tar.zst
sudo systemctl restart netbird.service
sudo netbird up

@Pamalosebi commented on GitHub (Jul 30, 2025): My system was unusable so I reverted to an older version... That did work. I did put netbird in my package managers (pacman) exclusion list for now. ``` sudo podman -U /var/cache/pacman/pkg/netbird-0.51.2-1-x86_64.pkg.tar.zst sudo systemctl restart netbird.service sudo netbird up ```

saavagebueno commented

@LordAnchemis commented on GitHub (Jul 30, 2025):

Can I ask you to enable trace logs ("permanently", so it starts with those) and gather debug bundles? You can try netbird debug bundle -U, but feel free to mail those to support@netbird.io if the upload fails.

See https://docs.netbird.io/how-to/troubleshooting-client#on-other-linux-and-mac-os for instructions on changing log levels

DNS name resolution is broken

Netbird forces its own nameservers (100.x.x.x) into /etc/resolv.conf
Although it does backup the original file (with the old name servers), you cannot edit resolv.conf while netbird is up

If you haven't defined a name server in the dashboard and your client relies on IPv4 DNS - domain names are no longer resolved system-wide - it will try to look up the name server at 100.x.x.x, which then points to nothing = fail

You can bypass the issue by using either IPv6 (if your system/network/ISP supports it) or set up a custom name server (in the dashboard) that points to your 'old' DNS resolver (ie. the router/gateway IP address)

But this is only a 'temporary' fix - as netbird should NOT break core system functionality (like DNS out of the box)

Users should be allowed to choose which DNS server they want (ie. router/gateway or netbird's) and what priority etc. - or at least fall back to the default/old DNS if there is no custom name server defined in the dashboard

Don't need logs - as a few of us (@buzzzo) have already done the digging around and diagnosed the issue this morning

@LordAnchemis commented on GitHub (Jul 30, 2025): > Can I ask you to enable `trace` logs ("permanently", so it starts with those) and gather debug bundles? You can try `netbird debug bundle -U`, but feel free to mail those to [support@netbird.io](mailto:support@netbird.io) if the upload fails. > > See https://docs.netbird.io/how-to/troubleshooting-client#on-other-linux-and-mac-os for instructions on changing log levels DNS name resolution is broken Netbird forces its own nameservers (100.x.x.x) into /etc/resolv.conf Although it does backup the original file (with the old name servers), you cannot edit resolv.conf while netbird is up If you haven't defined a name server in the dashboard and your client relies on IPv4 DNS - domain names are no longer resolved system-wide - it will try to look up the name server at 100.x.x.x, which then points to nothing = fail You can bypass the issue by using either IPv6 (if your system/network/ISP supports it) or set up a custom name server (in the dashboard) that points to your 'old' DNS resolver (ie. the router/gateway IP address) But this is only a 'temporary' fix - as netbird should NOT break core system functionality (like DNS out of the box) Users should be allowed to choose which DNS server they want (ie. router/gateway or netbird's) and what priority etc. - or at least fall back to the default/old DNS if there is no custom name server defined in the dashboard Don't need logs - as a few of us (@buzzzo) have already done the digging around and diagnosed the issue this morning

saavagebueno commented

@lixmal commented on GitHub (Jul 30, 2025):

Don't need logs - as a few of us (@buzzzo) have already done the digging around and diagnosed the issue this morning

Can you describe what needs to be fixed or provide a PR? I haven't figured out what's broken yet without logs

@lixmal commented on GitHub (Jul 30, 2025): >Don't need logs - as a few of us (@buzzzo) have already done the digging around and diagnosed the issue this morning Can you describe what needs to be fixed or provide a PR? I haven't figured out what's broken yet without logs

saavagebueno commented

@braginini commented on GitHub (Jul 30, 2025):

Don't need logs - as a few of us (@buzzzo) have already done the digging around and diagnosed the issue this morning

Logs help fixing things as they contain information that may be specific to certain environments and just speed things up when debugging. Not everyone has the issue. Would be great if you could share the logs

@braginini commented on GitHub (Jul 30, 2025): >Don't need logs - as a few of us (@buzzzo) have already done the digging around and diagnosed the issue this morning Logs help fixing things as they contain information that may be specific to certain environments and just speed things up when debugging. Not everyone has the issue. Would be great if you could share the logs

saavagebueno commented

@LordAnchemis commented on GitHub (Jul 30, 2025):

2025-07-30T17:38:26Z TRAC client/internal/dns/handler_chain.go:166: DNS request domain=deb.debian.org., handlers (3):
  - pattern: domain=78.100.in-addr.arpa. original: domain=78.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100
  - pattern: domain=netbird.cloud. original: domain=netbird.cloud. wildcard=false match_subdomain=true priority=100
  - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100
2025-07-30T17:38:26Z TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=deb.debian.org. -> pattern=. wildcard=false match_subdomain=true priority=-100
2025-07-30T17:38:26Z TRAC [request_id: 4e843190] client/internal/dns/upstream.go:114: received upstream question: domain=deb.debian.org. type=1 class=1
2025-07-30T17:38:28Z WARN [request_id: 4e843190] client/internal/dns/upstream.go:138: upstream [fd78:94bf:7df8::1:53]:53 timed out for question domain=deb.debian.org.
2025-07-30T17:38:28Z ERRO [request_id: 4e843190] client/internal/dns/upstream.go:161: all queries to the upstream [[fd78:94bf:7df8::1:53]:53] failed for question domain=deb.debian.org.
2025-07-30T17:38:28Z WARN [request_id: d6e0f964] client/internal/dns/upstream.go:138: upstream [fd78:94bf:7df8::1:53]:53 timed out for question domain=deb.debian.org.
2025-07-30T17:38:28Z ERRO [request_id: d6e0f964] client/internal/dns/upstream.go:161: all queries to the upstream [[fd78:94bf:7df8::1:53]:53] failed for question domain=deb.debian.org.

@LordAnchemis commented on GitHub (Jul 30, 2025): ``` 2025-07-30T17:38:26Z TRAC client/internal/dns/handler_chain.go:166: DNS request domain=deb.debian.org., handlers (3): - pattern: domain=78.100.in-addr.arpa. original: domain=78.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100 - pattern: domain=netbird.cloud. original: domain=netbird.cloud. wildcard=false match_subdomain=true priority=100 - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100 2025-07-30T17:38:26Z TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=deb.debian.org. -> pattern=. wildcard=false match_subdomain=true priority=-100 2025-07-30T17:38:26Z TRAC [request_id: 4e843190] client/internal/dns/upstream.go:114: received upstream question: domain=deb.debian.org. type=1 class=1 2025-07-30T17:38:28Z WARN [request_id: 4e843190] client/internal/dns/upstream.go:138: upstream [fd78:94bf:7df8::1:53]:53 timed out for question domain=deb.debian.org. 2025-07-30T17:38:28Z ERRO [request_id: 4e843190] client/internal/dns/upstream.go:161: all queries to the upstream [[fd78:94bf:7df8::1:53]:53] failed for question domain=deb.debian.org. 2025-07-30T17:38:28Z WARN [request_id: d6e0f964] client/internal/dns/upstream.go:138: upstream [fd78:94bf:7df8::1:53]:53 timed out for question domain=deb.debian.org. 2025-07-30T17:38:28Z ERRO [request_id: d6e0f964] client/internal/dns/upstream.go:161: all queries to the upstream [[fd78:94bf:7df8::1:53]:53] failed for question domain=deb.debian.org. ```

saavagebueno commented

@buzzzo commented on GitHub (Jul 30, 2025):

IMHO the point on this issue is not related to find a possible bug or not but on the opportunity to force a configuration on a system which could potentially (and it did) create severe disruption of service.

From what i've seen, let's imagine the following case:

no dns policy non netbird server side

So imagine to upgrade netbird on host A, as soon as netbird gets upgraded the newest service starts to do the following things on host A:

rewrite the system resolver config of host A pretending to set as nameserver the netbird peer ip of host A
if no dns policy found then use the ones presents in resolv.conf of host A

From now on the life lets get started complicated:
imagine that for whatever reason the netbird service is unable to create a valid config on host A:

it creates a resolv.conf file which a line starting like: nameserver BROKEN_ip (yes i've find in my resolv.conf something like that)
I can't modify the resolv.conf to restore a working one because the netbird service actively monitoring it: as soon as i will try to modify resolv.conf it wil restore its own file (the broken one).

So in the end i will have a broken dns resolver, with all the relative consequences.

Some of the problems found with this approach:

nobody ask netbird to enable by default this "feature", which could create dns resolution issues also with a good config applied.
netbird does not check if the config works: resulting in the worst case in a total broken resolver config.
adding in the search domains also the netbird domain could cause misconfiguration in dns resolving (ping hostA gives host A netbird peer ip instead of host A real ip)
Bogus dns resolver (aka: invalid rbl lookup) in the fortunate cases that the resolver config is applied correctly.

So my first concern is the approach that Netbird's Team has used in implementing a breaking (in all senses) change feature.

POSSIBLE WORKAROUNDS:

create a GLOBAL switch on netbird server to enable/disable the auto-replacement of peer's resolver config
create 1) as false by default in the netbird config. (something like ENABLE_DNS_RESOLVER_MODIFICATION=false)
create a GLOBAL flag for disable/enable search netbird vpn domain in peer's resolver config

@buzzzo commented on GitHub (Jul 30, 2025): IMHO the point on this issue is not related to find a possible bug or not but on the opportunity to force a configuration on a system which could potentially (and it did) create severe disruption of service. From what i've seen, let's imagine the following case: 1) no dns policy non netbird server side So imagine to upgrade netbird on host A, as soon as netbird gets upgraded the newest service starts to do the following things on host A: 1) rewrite the system resolver config of host A pretending to set as nameserver the netbird peer ip of host A 2) if no dns policy found then use the ones presents in resolv.conf of host A From now on the life lets get started complicated: imagine that for whatever reason the netbird service is unable to create a valid config on host A: 1) it creates a resolv.conf file which a line starting like: nameserver BROKEN_ip (yes i've find in my resolv.conf something like that) 2) I can't modify the resolv.conf to restore a working one because the netbird service actively monitoring it: as soon as i will try to modify resolv.conf it wil restore its own file (the broken one). So in the end i will have a broken dns resolver, with all the relative consequences. Some of the problems found with this approach: 1) nobody ask netbird to enable by default this "feature", which could create dns resolution issues also with a good config applied. 2) netbird does not check if the config works: resulting in the worst case in a total broken resolver config. 3) adding in the search domains also the netbird domain could cause misconfiguration in dns resolving (ping hostA gives host A netbird peer ip instead of host A real ip) 4) Bogus dns resolver (aka: invalid rbl lookup) in the fortunate cases that the resolver config is applied correctly. So my first concern is the approach that Netbird's Team has used in implementing a breaking (in all senses) change feature. POSSIBLE WORKAROUNDS: 1) create a GLOBAL switch on netbird server to enable/disable the auto-replacement of peer's resolver config 2) create 1) as false by default in the netbird config. (something like ENABLE_DNS_RESOLVER_MODIFICATION=false) 3) create a GLOBAL flag for disable/enable search netbird vpn domain in peer's resolver config

saavagebueno commented

@LordAnchemis commented on GitHub (Jul 30, 2025):

I agree with @buzzzo

Or at least install the DNS resolver to another file like /etc/netbird/nameservers or something - rather than hijack the system resolv.conf (which is naughty)

And give the user ability to select which DNS resolver to use

@LordAnchemis commented on GitHub (Jul 30, 2025): I agree with @buzzzo Or at least install the DNS resolver to another file like /etc/netbird/nameservers or something - rather than hijack the system resolv.conf (which is naughty) And give the user ability to select which DNS resolver to use

saavagebueno commented

@buzzzo commented on GitHub (Jul 30, 2025):

I agree with @buzzzo

Or at least install the DNS resolver to another file like /etc/netbird/nameservers or something - rather than hijack the system resolv.conf (which is naughty)

And give the user ability to select which DNS resolver to use

If it done properly the config should work but imho the choice of enabling/disabling should be an admin responsability.

@buzzzo commented on GitHub (Jul 30, 2025): > I agree with [@buzzzo](https://github.com/buzzzo) > > Or at least install the DNS resolver to another file like /etc/netbird/nameservers or something - rather than hijack the system resolv.conf (which is naughty) > > And give the user ability to select which DNS resolver to use If it done properly the config should work but imho the choice of enabling/disabling should be an admin responsability.

saavagebueno commented

@lixmal commented on GitHub (Jul 30, 2025):

https://github.com/netbirdio/netbird/releases/tag/v0.52.2 is on the way

@lixmal commented on GitHub (Jul 30, 2025): https://github.com/netbirdio/netbird/releases/tag/v0.52.2 is on the way

saavagebueno commented

@buzzzo commented on GitHub (Jul 30, 2025):

https://github.com/netbirdio/netbird/releases/tag/v0.52.2 is on the way

Is it supposed to fix some of the DNS management issues ?

@buzzzo commented on GitHub (Jul 30, 2025): > https://github.com/netbirdio/netbird/releases/tag/v0.52.2 is on the way Is it supposed to fix some of the DNS management issues ?

saavagebueno commented

@Blackclaws commented on GitHub (Jul 31, 2025):

My guess is that the upstream resolvers are not picked up correctly by netbird, hence resolution is failing.

@buzzzo
it creates a resolv.conf file which a line starting like: nameserver BROKEN_ip (yes i've find in my resolv.conf something like that)

What is the BROKEN_ip here? isn't it the netbird peer local address?

@Blackclaws commented on GitHub (Jul 31, 2025): My guess is that the upstream resolvers are not picked up correctly by netbird, hence resolution is failing. @buzzzo it creates a resolv.conf file which a line starting like: nameserver BROKEN_ip (yes i've find in my resolv.conf something like that) What is the BROKEN_ip here? isn't it the netbird peer local address?

saavagebueno commented

@buzzzo commented on GitHub (Jul 31, 2025):

My guess is that the upstream resolvers are not picked up correctly by netbird, hence resolution is failing.

@buzzzo it creates a resolv.conf file which a line starting like: nameserver BROKEN_ip (yes i've find in my resolv.conf something like that)

What is the BROKEN_ip here? isn't it the netbird peer local address?

It's certainly a race condition: but BEFORE commiting a conf on a vital file on the system imho netbird should be verify that the conf works.

@buzzzo commented on GitHub (Jul 31, 2025): > My guess is that the upstream resolvers are not picked up correctly by netbird, hence resolution is failing. > > [@buzzzo](https://github.com/buzzzo) it creates a resolv.conf file which a line starting like: nameserver BROKEN_ip (yes i've find in my resolv.conf something like that) > > What is the BROKEN_ip here? isn't it the netbird peer local address? It's certainly a race condition: but BEFORE commiting a conf on a vital file on the system imho netbird should be verify that the conf works.

saavagebueno commented

@hurricanehrndz commented on GitHub (Jul 31, 2025):

I think most ppl would like to know what the purpose of the change or more specifically this PR is. Also if you are going to be the system resolver it would be best that you ensure you are adhering to the RFCs and the standard:
https://github.com/netbirdio/netbird/issues/4242

I am in agreeance with everyone else here though, on linux it should be the admin's job to decide what system configurations get applied.

@hurricanehrndz commented on GitHub (Jul 31, 2025): I think most ppl would like to know what the purpose of the change or more specifically this [PR](https://github.com/netbirdio/netbird/commit/cb85d3f2fc8d6836544013fd1764d93d8a02351d) is. Also if you are going to be the system resolver it would be best that you ensure you are adhering to the RFCs and the standard: https://github.com/netbirdio/netbird/issues/4242 I am in agreeance with everyone else here though, on linux it should be the admin's job to decide what system configurations get applied.

saavagebueno commented

@hurricanehrndz commented on GitHub (Jul 31, 2025):

Does anyone know what problems the change was trying to fix

@hurricanehrndz commented on GitHub (Jul 31, 2025): Does anyone know what problems the change was trying to fix

saavagebueno commented

@andreascschmidt commented on GitHub (Aug 1, 2025):

I have 0.52.2 and the whole DNS is still fubar. But I also note that on all nodes since the 0.52 update, the routing itself seems broken .. Like it's actually not picking the netbird network at all hence cant resolve the "custom" DNS.

@andreascschmidt commented on GitHub (Aug 1, 2025): I have 0.52.2 and the whole DNS is still fubar. But I also note that on all nodes since the 0.52 update, the routing itself seems broken .. Like it's actually not picking the netbird network at all hence cant resolve the "custom" DNS.

saavagebueno commented

@hurricanehrndz commented on GitHub (Aug 3, 2025):

I have not seen this, and also I understand now what the team was trying to
fix. I am sure that they will solve the issue.

On Fri, Aug 1, 2025 at 1:37 PM Andreas @.***> wrote:

andreascschmidt left a comment (netbirdio/netbird#4250)
https://github.com/netbirdio/netbird/issues/4250#issuecomment-3145581551

I have 0.52.2 and the whole DNS is still fubar. But I also note that on
all nodes since the update, the routing itself seems broken .. Like it's
actually not picking the netbird network at all hence cant resolve the
"custom" DNS.

—
Reply to this email directly, view it on GitHub
https://github.com/netbirdio/netbird/issues/4250#issuecomment-3145581551,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABMJBTJKCAEOGQ73AV3IYAL3LO6WBAVCNFSM6AAAAACCWE2YCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCNBVGU4DCNJVGE
.
You are receiving this because you commented.Message ID:
@.***>

@hurricanehrndz commented on GitHub (Aug 3, 2025): I have not seen this, and also I understand now what the team was trying to fix. I am sure that they will solve the issue. On Fri, Aug 1, 2025 at 1:37 PM Andreas ***@***.***> wrote: > *andreascschmidt* left a comment (netbirdio/netbird#4250) > <https://github.com/netbirdio/netbird/issues/4250#issuecomment-3145581551> > > I have 0.52.2 and the whole DNS is still fubar. But I also note that on > all nodes since the update, the routing itself seems broken .. Like it's > actually not picking the netbird network at all hence cant resolve the > "custom" DNS. > > — > Reply to this email directly, view it on GitHub > <https://github.com/netbirdio/netbird/issues/4250#issuecomment-3145581551>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABMJBTJKCAEOGQ73AV3IYAL3LO6WBAVCNFSM6AAAAACCWE2YCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCNBVGU4DCNJVGE> > . > You are receiving this because you commented.Message ID: > ***@***.***> >

saavagebueno commented