SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-03-31 06:34:19 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

Glibc DNS Resolver (non systemd) fails to resolve CNAME record to local CGNAT IP #2208

New Issue
Open
opened 2025-11-20 07:05:50 -05:00 by saavagebueno · 0 comments
saavagebueno commented 2025-11-20 07:05:50 -05:00
Owner
Copy Link

Originally created by @TomGudman on GitHub (Aug 19, 2025).

Describe the problem

When using NetBird 0.53.0 on a Gentoo or MX Linux that doesn't use systemd but the glibc resolver, we cannot resolve a CNAME pointing to a NetBird DNS name to its CGNAT IP. The issue could be around using a DNS name from the managed NetBird zone (.netbird.selfhosted) as the target of the CNAME.

In the below example you would expect wiki.vpn.domain.tld to resolve to 100.93.192.221 but it doesn't whereas querying wiki01.netbird.selfhosted directly does return the IP.

root@mx:~# host wiki.vpn.domain.tld
wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted.

root@mx:~# host wiki01.netbird.selfhosted
wiki01.infra.iob has address 100.93.192.221

What is normally expected is something like this (e.g: via systemd), the resolver resolves the CNAME and then the A record for wiki01.netbird.selfhosted

root@mx:~# host wiki.vpn.domain.tld
wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted.
wiki01.netbird.selfhosted has address 100.93.192.221

To Reproduce

Steps to reproduce the behavior:

  1. Install MX Linux in a VM
  2. Install NetBird
  3. Set up public DNS entries (assuming netbird.selfhosted is NetBird's automatic DNS)
    • wiki.vpn.domain.tld IN CNAME wiki01.netbird.selfhosted
  4. Where wiki01.netbird.selfhosted is a node on the NetBird mesh network with the IP 100.93.192.221 (for illustrative purpose, any IP will do)
  5. Try to resolve: host wiki.vpn.domain.tld
  6. The IP is not returned.
root@mx:~# host wiki.vpn.domain.tld
wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted.

I suspect NetBird's resolver sees that wiki.vpn.domain.tld is not 'local', so it tries to resolve it by recursing from the root level, gets the CNAME but then doesn't realise the target is actually a 'local' hostname wiki01.netbird.selfhosted and doesn't resolve via its own authoritative DNS whereas it could :)

Expected behavior
I would expect NetBird with Glibc resolver to work the same than on a systemd system.
The right answer to the DNS query is:

root@mx:~# host wiki.vpn.domain.tld
wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted.
wiki01.netbird.selfhosted has address 100.93.192.221

Are you using NetBird Cloud?
No. Selfhosted.

NetBird version

netbird version
0.53.0

Is any other VPN software installed?
No.

Debug output
Unfortunately I tore apart my test scenario so I cannot provide a debug output. I abandoned the idea of the CNAME to the NetBird internal domain and point the public name directly to the CGNAT IP. While not ideal, it's a workaround to get us going. However this issue remains an unexpected behaviour.

Screenshots
N/A

Additional context
I increased debug logging.

oot@mx:~# host wiki.vpn.domain.tld

2025-08-08T14:48:36+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki.vpn.domain.tld., handlers (3):
  - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100
  - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100
  - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100
2025-08-08T14:48:36+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki.vpn.domain.tld. -> pattern=. wildcard=false match_subdomain=true priority=-100
2025-08-08T14:48:36+10:00 TRAC [request_id: 5af76651] client/internal/dns/upstream.go:114: received upstream question: domain=wiki.vpn.domain.tld. type=1 class=1
2025-08-08T14:48:37+10:00 TRAC [request_id: 5af76651] client/internal/dns/upstream.go:151: took 301.380076ms to query the upstream 192.168.1.254:53 for question domain=wiki.vpn.domain.tld.
Host wiki.vpn.domain.tld not found: 3(NXDOMAIN)


root@mx:~# host wiki.vpn.domain.tld

2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki.vpn.domain.tld., handlers (3):
  - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100
  - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100
  - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100
2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki.vpn.domain.tld. -> pattern=. wildcard=false match_subdomain=true priority=-100
2025-08-08T14:52:17+10:00 TRAC [request_id: 9d2b7861] client/internal/dns/upstream.go:114: received upstream question: domain=wiki.vpn.domain.tld. type=1 class=1
2025-08-08T14:52:17+10:00 TRAC [request_id: 9d2b7861] client/internal/dns/upstream.go:151: took 1.176969ms to query the upstream 192.168.1.254:53 for question domain=wiki.vpn.domain.tld.
wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted.

2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki01.netbird.selfhosted., handlers (3):
  - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100
  - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100
  - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100
2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki01.netbird.selfhosted. -> pattern=netbird.selfhosted. wildcard=false match_subdomain=true priority=100
2025-08-08T14:52:17+10:00 TRAC client/internal/dns/local/local.go:58: received local question: domain=wiki01.netbird.selfhosted. type=28 class=1

2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki01.netbird.selfhosted., handlers (3):
  - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100
  - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100
  - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100
2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki01.netbird.selfhosted. -> pattern=netbird.selfhosted. wildcard=false match_subdomain=true priority=100
2025-08-08T14:52:17+10:00 TRAC client/internal/dns/local/local.go:58: received local question: domain=wiki01.netbird.selfhosted. type=15 class=1

Observations
So based on QTYPES values from Wikipedia, it requests an MX record first (type=15) followed by a AAAA (type=28)... but not the A record (type=1) for the CGNAT IPv4 address we are after.

Unfortunately I can't tell why it's requesting for a AAAA record (ipv6) instead of an A record (ipv4) or even both.
Actually, the MX record query could be the mistake, why does it request the MX (type=15) instead of A (type=1)

Ref:

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions (at the time 0.53 was the latest)
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • [N/A] Disabled other VPN software
  • Checked firewall settings
Originally created by @TomGudman on GitHub (Aug 19, 2025). **Describe the problem** When using NetBird 0.53.0 on a Gentoo or MX Linux that doesn't use systemd but the glibc resolver, we cannot resolve a CNAME pointing to a NetBird DNS name to its CGNAT IP. The issue could be around using a DNS name from the managed NetBird zone (`.netbird.selfhosted`) as the target of the CNAME. In the below example you would expect `wiki.vpn.domain.tld` to resolve to `100.93.192.221` but it doesn't whereas querying `wiki01.netbird.selfhosted` directly does return the IP. ``` root@mx:~# host wiki.vpn.domain.tld wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted. root@mx:~# host wiki01.netbird.selfhosted wiki01.infra.iob has address 100.93.192.221 ``` What is normally expected is something like this (e.g: via systemd), the resolver resolves the CNAME and then the A record for `wiki01.netbird.selfhosted` ``` root@mx:~# host wiki.vpn.domain.tld wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted. wiki01.netbird.selfhosted has address 100.93.192.221 ``` **To Reproduce** Steps to reproduce the behavior: 1. Install MX Linux in a VM 2. Install NetBird 3. Set up public DNS entries (assuming `netbird.selfhosted` is NetBird's automatic DNS) - `wiki.vpn.domain.tld IN CNAME wiki01.netbird.selfhosted` 4. Where `wiki01.netbird.selfhosted` is a node on the NetBird mesh network with the IP `100.93.192.221` (for illustrative purpose, any IP will do) 4. Try to resolve: `host wiki.vpn.domain.tld` 5. The IP is not returned. ``` root@mx:~# host wiki.vpn.domain.tld wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted. ``` I suspect NetBird's resolver sees that `wiki.vpn.domain.tld` is not 'local', so it tries to resolve it by recursing from the root level, gets the CNAME but then doesn't _realise_ the target is actually a 'local' hostname `wiki01.netbird.selfhosted` and doesn't resolve via its own authoritative DNS whereas it could :) **Expected behavior** I would expect NetBird with Glibc resolver to work the same than on a systemd system. The right answer to the DNS query is: ``` root@mx:~# host wiki.vpn.domain.tld wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted. wiki01.netbird.selfhosted has address 100.93.192.221 ``` **Are you using NetBird Cloud?** No. Selfhosted. **NetBird version** ``` netbird version 0.53.0 ``` **Is any other VPN software installed?** No. **Debug output** Unfortunately I tore apart my test scenario so I cannot provide a debug output. I abandoned the idea of the CNAME to the NetBird internal domain and point the public name directly to the CGNAT IP. While not ideal, it's a workaround to get us going. However this issue remains an unexpected behaviour. **Screenshots** N/A **Additional context** I increased debug logging. ``` oot@mx:~# host wiki.vpn.domain.tld 2025-08-08T14:48:36+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki.vpn.domain.tld., handlers (3): - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100 - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100 - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100 2025-08-08T14:48:36+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki.vpn.domain.tld. -> pattern=. wildcard=false match_subdomain=true priority=-100 2025-08-08T14:48:36+10:00 TRAC [request_id: 5af76651] client/internal/dns/upstream.go:114: received upstream question: domain=wiki.vpn.domain.tld. type=1 class=1 2025-08-08T14:48:37+10:00 TRAC [request_id: 5af76651] client/internal/dns/upstream.go:151: took 301.380076ms to query the upstream 192.168.1.254:53 for question domain=wiki.vpn.domain.tld. Host wiki.vpn.domain.tld not found: 3(NXDOMAIN) root@mx:~# host wiki.vpn.domain.tld 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki.vpn.domain.tld., handlers (3): - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100 - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100 - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki.vpn.domain.tld. -> pattern=. wildcard=false match_subdomain=true priority=-100 2025-08-08T14:52:17+10:00 TRAC [request_id: 9d2b7861] client/internal/dns/upstream.go:114: received upstream question: domain=wiki.vpn.domain.tld. type=1 class=1 2025-08-08T14:52:17+10:00 TRAC [request_id: 9d2b7861] client/internal/dns/upstream.go:151: took 1.176969ms to query the upstream 192.168.1.254:53 for question domain=wiki.vpn.domain.tld. wiki.vpn.domain.tld is an alias for wiki01.netbird.selfhosted. 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki01.netbird.selfhosted., handlers (3): - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100 - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100 - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki01.netbird.selfhosted. -> pattern=netbird.selfhosted. wildcard=false match_subdomain=true priority=100 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/local/local.go:58: received local question: domain=wiki01.netbird.selfhosted. type=28 class=1 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:166: DNS request domain=wiki01.netbird.selfhosted., handlers (3): - pattern: domain=93.100.in-addr.arpa. original: domain=93.100.in-addr.arpa. wildcard=false match_subdomain=true priority=100 - pattern: domain=netbird.selfhosted. original: domain=netbird.selfhosted. wildcard=false match_subdomain=true priority=100 - pattern: domain=. original: domain=. wildcard=false match_subdomain=true priority=-100 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/handler_chain.go:174: handler matched: domain=wiki01.netbird.selfhosted. -> pattern=netbird.selfhosted. wildcard=false match_subdomain=true priority=100 2025-08-08T14:52:17+10:00 TRAC client/internal/dns/local/local.go:58: received local question: domain=wiki01.netbird.selfhosted. type=15 class=1 ``` *Observations* So based on QTYPES values from [Wikipedia](https://en.wikipedia.org/wiki/List_of_DNS_record_types), it requests an MX record first (type=15) followed by a AAAA (type=28)... but not the A record (type=1) for the CGNAT IPv4 address we are after. Unfortunately I can't tell why it's requesting for a AAAA record (ipv6) instead of an A record (ipv4) or even both. Actually, the MX record query could be the mistake, why does it request the MX (type=15) instead of A (type=1) Ref: - https://github.com/netbirdio/netbird/blob/main/client/internal/dns/local/local.go#L58 - https://github.com/netbirdio/netbird/blob/main/client/internal/dns/handler_chain.go#L174 - I originally posted my issue on Slack: https://netbirdio.slack.com/archives/C02KHAE8VLZ/p1754626563100329 **Have you tried these troubleshooting steps?** - [x] Reviewed [client troubleshooting](https://docs.netbird.io/how-to/troubleshooting-client) (if applicable) - [x] Checked for newer NetBird versions (at the time 0.53 was the latest) - [x] Searched for similar issues on GitHub (including closed ones) - [x] Restarted the NetBird client - [N/A] Disabled other VPN software - [x] Checked firewall settings
saavagebueno added the bugdns labels 2025-11-20 07:05:51 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label bug dns
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#2208
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?