[GH-ISSUE #5921] macOS client: domain-type Network Resources never register scutil resolver; local forwarder returns empty instead of forwarding to peer :22054 (v0.68.3) #12358

Open
opened 2026-08-05 02:05:44 -04:00 by saavagebueno · 0 comments
Owner

Originally created by @realsby on GitHub (Apr 18, 2026).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/5921

Summary

On macOS, the NetBird client does not register a system DNS resolver (scutil) for domain-type Network Resources, and its local forwarder at 100.X.255.254:53 returns NOERROR with an empty answer section when queried directly for a resource's target FQDN. netbird networks list shows Resolved IPs: - permanently.

Related: #5654 (closed) — same headline symptom but different root cause (that one was about a retry-after-handshake race for the upstream probe). In our case the routing peer's :22054 resolver is reachable and answers correctly — the gap is entirely on the client side: it never registers the target domain in the OS resolver nor forwards queries for it to the peer.

Environment

Server v0.68.3 (self-hosted)
Routing peer v0.68.3 (Kubernetes, kernel WireGuard, Linux arm64)
Client macOS (arm64), v0.68.3, interface type Userspace
Network Resources 6 resources, type domain, addresses like my-svc.my-ns.svc.cluster.local
Policies destinationResource per resource (a group → each resource)
Nameserver entries none configured

Expected behavior

  1. For each domain-type Network Resource reachable by the client's groups, the client registers a scutil resolver on macOS scoped to the resource's target domain, pointing at 100.X.255.254:53 (this is the PR #3614 / v0.40 fix for #3536).
  2. DNS queries for the resource FQDN hit the local forwarder, which forwards through the WG tunnel to the routing peer's :22054 resolver.
  3. netbird networks list shows a populated Resolved IPs: field after a successful resolve.

Actual behavior

  1. scutil --dns shows a resolver only for netbird.selfhosted and the corresponding reverse zone. No resolver is registered for the resource target domain (svc.cluster.local in our case).
  2. When the local forwarder at 100.X.255.254:53 is queried directly with the resource FQDN, it returns status: NOERROR, ANSWER: 0. It does not forward to the peer, and does not NXDOMAIN.
  3. Resolved IPs: - stays empty indefinitely through reconnects (netbird down && netbird up) and daemon restarts.

Evidence

1. scutil — no resolver for the resource target domain

resolver #5
  domain   : netbird.selfhosted
  nameserver[0] : 100.X.255.254
  port     : 53

scutil --dns | grep svc.cluster.local returns nothing.

2. Local forwarder returns empty for the resource FQDN

$ dig @100.X.255.254 my-svc.my-ns.svc.cluster.local
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: xxxxx
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

Same shape for every resource FQDN. The synthetic form <resource>.<network>.netbird.selfhosted returns NXDOMAIN (understood — domain-type resources don't generate that alias).

3. Routing peer :22054 resolver works correctly

From the same macOS client, same session, through the same WG tunnel:

$ dig @<routing-peer-wg-ip> -p 22054 my-svc.my-ns.svc.cluster.local +short
10.X.Y.Z

This rules out: server config, policy chain, routing peer DNS config, upstream DNS reachability from the peer, mesh connectivity between client and peer on :22054.

4. netbird networks list — resources present, never resolved

- ID: my-svc
  Domains: my-svc.my-ns.svc.cluster.local
  Status: Selected
  Resolved IPs: -

All six resources in identical state. Does not change after reconnects or multi-minute waits.

5. netbird status -d — client connected, no nameserver health issue

Management: Connected
Signal:     Connected
Interface type: Userspace
Nameservers:    (empty)
Networks: -     (note: empty here, but `netbird networks list` shows 6 resources)
Peers count:    4/5 Connected

Reproduction

  1. Self-hosted v0.68.3 server + routing peer on Linux.
  2. Create domain-type Network Resources with target FQDNs in a zone the routing peer can resolve (e.g. *.svc.cluster.local inside a Kubernetes cluster).
  3. Create destinationResource policies so a group has access to each resource.
  4. Connect a macOS client (v0.68.3) in that group.
  5. netbird networks list → resources show Resolved IPs: -.
  6. scutil --dns | grep <target-zone> → no resolver registered.
  7. dig @<peer-wg-ip> -p 22054 <resource-fqdn> → correct IP returned. So the server/peer side works; the client-side registration and forwarding is broken.

What's already ruled out

  • Not the retry-after-handshake race from #5654 — we're on v0.68.3 and have reconnected many times long after handshake completes.
  • Not a policy issue — the peer's :22054 resolver is reachable and answering.
  • Not a routing peer DNS issue — upstream DNS resolves correctly from inside the peer, and its own netbird status shows all 6 Networks: properly.
  • Not a network policy / firewall issue on the cluster side.

Workaround (partial)

Adding a nameserver entry with match domain = the target zone (e.g. svc.cluster.local) pointing to any routable IP can force the client to register a scutil resolver for that domain. [Will update with result.]

A more robust architectural workaround is to switch to a subnet-type resource (e.g. the K8s Service CIDR) plus a manual /etc/resolver/<zone> file pointing at the routing peer's :22054, which bypasses the broken auto-registration path entirely.

References

  • #3536 — original bug (pre-v0.40) where domain resources didn't register with the OS resolver
  • #3614 — the PR that shipped the fix for #3536
  • #5654 — same headline symptom on Linux / different root cause (retry race)
Originally created by @realsby on GitHub (Apr 18, 2026). Original GitHub issue: https://github.com/netbirdio/netbird/issues/5921 ## Summary On macOS, the NetBird client does **not** register a system DNS resolver (scutil) for domain-type Network Resources, and its local forwarder at `100.X.255.254:53` returns `NOERROR` with an empty answer section when queried directly for a resource's target FQDN. `netbird networks list` shows `Resolved IPs: -` permanently. Related: #5654 (closed) — same headline symptom but different root cause (that one was about a retry-after-handshake race for the upstream probe). In our case **the routing peer's `:22054` resolver is reachable and answers correctly** — the gap is entirely on the client side: it never registers the target domain in the OS resolver nor forwards queries for it to the peer. ## Environment | | | |---|---| | Server | v0.68.3 (self-hosted) | | Routing peer | v0.68.3 (Kubernetes, kernel WireGuard, Linux arm64) | | Client | macOS (arm64), v0.68.3, interface type `Userspace` | | Network Resources | 6 resources, type `domain`, addresses like `my-svc.my-ns.svc.cluster.local` | | Policies | `destinationResource` per resource (a group → each resource) | | Nameserver entries | none configured | ## Expected behavior 1. For each domain-type Network Resource reachable by the client's groups, the client registers a scutil resolver on macOS scoped to the resource's target domain, pointing at `100.X.255.254:53` (this is the PR #3614 / v0.40 fix for #3536). 2. DNS queries for the resource FQDN hit the local forwarder, which forwards through the WG tunnel to the routing peer's `:22054` resolver. 3. `netbird networks list` shows a populated `Resolved IPs:` field after a successful resolve. ## Actual behavior 1. `scutil --dns` shows a resolver only for `netbird.selfhosted` and the corresponding reverse zone. **No resolver is registered for the resource target domain** (`svc.cluster.local` in our case). 2. When the local forwarder at `100.X.255.254:53` is queried **directly** with the resource FQDN, it returns `status: NOERROR`, `ANSWER: 0`. It does not forward to the peer, and does not NXDOMAIN. 3. `Resolved IPs: -` stays empty indefinitely through reconnects (`netbird down && netbird up`) and daemon restarts. ## Evidence ### 1. scutil — no resolver for the resource target domain ``` resolver #5 domain : netbird.selfhosted nameserver[0] : 100.X.255.254 port : 53 ``` `scutil --dns | grep svc.cluster.local` returns nothing. ### 2. Local forwarder returns empty for the resource FQDN ``` $ dig @100.X.255.254 my-svc.my-ns.svc.cluster.local ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: xxxxx ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ``` Same shape for every resource FQDN. The synthetic form `<resource>.<network>.netbird.selfhosted` returns `NXDOMAIN` (understood — domain-type resources don't generate that alias). ### 3. Routing peer `:22054` resolver works correctly From the same macOS client, same session, through the same WG tunnel: ``` $ dig @<routing-peer-wg-ip> -p 22054 my-svc.my-ns.svc.cluster.local +short 10.X.Y.Z ``` This rules out: server config, policy chain, routing peer DNS config, upstream DNS reachability from the peer, mesh connectivity between client and peer on `:22054`. ### 4. `netbird networks list` — resources present, never resolved ``` - ID: my-svc Domains: my-svc.my-ns.svc.cluster.local Status: Selected Resolved IPs: - ``` All six resources in identical state. Does not change after reconnects or multi-minute waits. ### 5. `netbird status -d` — client connected, no nameserver health issue ``` Management: Connected Signal: Connected Interface type: Userspace Nameservers: (empty) Networks: - (note: empty here, but `netbird networks list` shows 6 resources) Peers count: 4/5 Connected ``` ## Reproduction 1. Self-hosted v0.68.3 server + routing peer on Linux. 2. Create domain-type Network Resources with target FQDNs in a zone the routing peer can resolve (e.g. `*.svc.cluster.local` inside a Kubernetes cluster). 3. Create `destinationResource` policies so a group has access to each resource. 4. Connect a macOS client (v0.68.3) in that group. 5. `netbird networks list` → resources show `Resolved IPs: -`. 6. `scutil --dns | grep <target-zone>` → no resolver registered. 7. `dig @<peer-wg-ip> -p 22054 <resource-fqdn>` → correct IP returned. So the server/peer side works; the client-side registration and forwarding is broken. ## What's already ruled out - Not the retry-after-handshake race from #5654 — we're on v0.68.3 and have reconnected many times long after handshake completes. - Not a policy issue — the peer's `:22054` resolver is reachable and answering. - Not a routing peer DNS issue — upstream DNS resolves correctly from inside the peer, and its own `netbird status` shows all 6 `Networks:` properly. - Not a network policy / firewall issue on the cluster side. ## Workaround (partial) Adding a nameserver entry with match domain = the target zone (e.g. `svc.cluster.local`) pointing to any routable IP can force the client to register a scutil resolver for that domain. [Will update with result.] A more robust architectural workaround is to switch to a subnet-type resource (e.g. the K8s Service CIDR) plus a manual `/etc/resolver/<zone>` file pointing at the routing peer's `:22054`, which bypasses the broken auto-registration path entirely. ## References - #3536 — original bug (pre-v0.40) where domain resources didn't register with the OS resolver - #3614 — the PR that shipped the fix for #3536 - #5654 — same headline symptom on Linux / different root cause (retry race)
Sign in to join this conversation.
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#12358