[GH-ISSUE #6017] Relay server not working #12677

Open
opened 2026-08-05 02:06:26 -04:00 by saavagebueno · 1 comment
Owner

Originally created by @christianhuth on GitHub (Apr 28, 2026).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/6017

Describe the problem

My peers are able to connect to the Management Platform, but aren't able to reach each other. I assume that this is due to a broken relay server. When I run netbird status --detail I can see, that only the STUN server is available:

Relays: 
  [stun:netbird.<my-domain>:3478] is Available
  [rels://netbird.<my-domain>:443] is Unavailable, reason: relay client not connected

To Reproduce

Run the following stack:

Use the following values.yaml to install Netbird:

---
dashboard:
  config:
    mgmtApiEndpoint: https://netbird.<my-domain>
    mgmtGrpcApiEndpoint: https://netbird.<my-domain>
    authAuthority: "https://netbird.<my-domain>/oauth2"
    authClientId: "netbird-dashboard"
    authAudience: "netbird-dashboard"
  httpRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - path: { type: PathPrefix, value: / }
  image:
    repository: <my-private-registry>/netbirdio/dashboard
    tag: v2.37.1
oidc:
  enabled: true
  idpManager:
    enabled: true
    managerType: embedded
server:
  config:
    auth:
      dashboardRedirectURIs:
        - https://netbird.<my-domain>/nb-auth
        - https://netbird.<my-domain>/nb-silent-auth
      issuer: https://netbird.<my-domain>oauth2
    exposedAddress: https://<my-domain>:443
  grpcRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - method: { service: signalexchange.SignalExchange }
      - matches:
          - method: { service: management.ManagementService }
  httpRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - path: { type: PathPrefix, value: /api }
          - path: { type: PathPrefix, value: /oauth2 }
  image:
    repository: <my-private-registry>/netbirdio/netbird-server
    tag: 0.70.0
  relayHttpRoute:
    enabled: true
    hostnames:
      - netbird.<my-domain>
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: shared-gateway
        namespace: gateway
        sectionName: netbird-https
    rules:
      - matches:
          - path: { type: PathPrefix, value: /relay }
          - path: { type: PathPrefix, value: /ws-proxy }
  secrets:
    authSecret:
      secretName: netbird-server
    storeEncryptionKey:
      secretName: netbird-server
  stunService:
    annotations:
      lbipam.cilium.io/ips: "<my-public-ip>"
      lbipam.cilium.io/sharing-cross-namespace: "*"
      lbipam.cilium.io/sharing-key: "<a-secret-sharing-key>"
    type: LoadBalancer

This gives you:

  • Dashboard exposed through HTTPRoute
  • Server exposed through HTTPRoute
  • Server-Relay exposed through HTTPRoute
  • Server-GRPC exposed through GRPCRoute
  • STUN exposed through LoadBalancer service (shared IP with the Ingress Gateway)
  • Everything uses the subdomain netbird. --> resolves to the IP of the STUN server as well
  • All Routes protected through TLS certificates

Expected behavior

  • dns resolution works
  • peers are able to connect to each other
  • ping router-75ffd8cd67-5drt9-227-180.netbird.selfhosted works

Are you using NetBird Cloud?

I am running on-premise version of Netbird. Everything installed in Kubernetes.

NetBird version

  • Local client: 0.68.0
  • Management Platform: 0.70.0
  • Router Pod Image: 0.70.0-rootless
  • Kubernetes Operator Chart: 0.3.1
  • Kubernetes Operator Image: 0.3.1

I tested other versions as well, with the same result.

Is any other VPN software installed?

No

Debug output

To help us resolve the problem, please attach the following anonymized status output

netbird-status.txt

Additional context

Add any other context about the problem here.

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • Disabled other VPN software
  • Checked firewall settings
Originally created by @christianhuth on GitHub (Apr 28, 2026). Original GitHub issue: https://github.com/netbirdio/netbird/issues/6017 **Describe the problem** My peers are able to connect to the Management Platform, but aren't able to reach each other. I assume that this is due to a broken relay server. When I run netbird status --detail I can see, that only the STUN server is available: ```bash Relays: [stun:netbird.<my-domain>:3478] is Available [rels://netbird.<my-domain>:443] is Unavailable, reason: relay client not connected ``` **To Reproduce** Run the following stack: - Netbird Management Platform self-hosted; installed with [this community Chart](https://github.com/KitStream/helms/tree/main/charts/netbird) v0.4.2 - [Netbird Kubernetes Operator](https://github.com/netbirdio/kubernetes-operator); installed with [the official Chart](https://github.com/netbirdio/helms/tree/main/charts/kubernetes-operator) v0.3.1 - Cilium as CNI - Cilium for Gateway API (shared gateway) - Cilium for LoadBalancer services Use the following `values.yaml` to install Netbird: ```yaml --- dashboard: config: mgmtApiEndpoint: https://netbird.<my-domain> mgmtGrpcApiEndpoint: https://netbird.<my-domain> authAuthority: "https://netbird.<my-domain>/oauth2" authClientId: "netbird-dashboard" authAudience: "netbird-dashboard" httpRoute: enabled: true hostnames: - netbird.<my-domain> parentRefs: - group: gateway.networking.k8s.io kind: Gateway name: shared-gateway namespace: gateway sectionName: netbird-https rules: - matches: - path: { type: PathPrefix, value: / } image: repository: <my-private-registry>/netbirdio/dashboard tag: v2.37.1 oidc: enabled: true idpManager: enabled: true managerType: embedded server: config: auth: dashboardRedirectURIs: - https://netbird.<my-domain>/nb-auth - https://netbird.<my-domain>/nb-silent-auth issuer: https://netbird.<my-domain>oauth2 exposedAddress: https://<my-domain>:443 grpcRoute: enabled: true hostnames: - netbird.<my-domain> parentRefs: - group: gateway.networking.k8s.io kind: Gateway name: shared-gateway namespace: gateway sectionName: netbird-https rules: - matches: - method: { service: signalexchange.SignalExchange } - matches: - method: { service: management.ManagementService } httpRoute: enabled: true hostnames: - netbird.<my-domain> parentRefs: - group: gateway.networking.k8s.io kind: Gateway name: shared-gateway namespace: gateway sectionName: netbird-https rules: - matches: - path: { type: PathPrefix, value: /api } - path: { type: PathPrefix, value: /oauth2 } image: repository: <my-private-registry>/netbirdio/netbird-server tag: 0.70.0 relayHttpRoute: enabled: true hostnames: - netbird.<my-domain> parentRefs: - group: gateway.networking.k8s.io kind: Gateway name: shared-gateway namespace: gateway sectionName: netbird-https rules: - matches: - path: { type: PathPrefix, value: /relay } - path: { type: PathPrefix, value: /ws-proxy } secrets: authSecret: secretName: netbird-server storeEncryptionKey: secretName: netbird-server stunService: annotations: lbipam.cilium.io/ips: "<my-public-ip>" lbipam.cilium.io/sharing-cross-namespace: "*" lbipam.cilium.io/sharing-key: "<a-secret-sharing-key>" type: LoadBalancer ``` This gives you: - Dashboard exposed through `HTTPRoute` - Server exposed through `HTTPRoute` - Server-Relay exposed through `HTTPRoute` - Server-GRPC exposed through `GRPCRoute` - STUN exposed through `LoadBalancer` service (shared IP with the Ingress Gateway) - Everything uses the subdomain netbird.<my-domain> --> resolves to the IP of the STUN server as well - All Routes protected through TLS certificates **Expected behavior** - dns resolution works - peers are able to connect to each other - `ping router-75ffd8cd67-5drt9-227-180.netbird.selfhosted` works **Are you using NetBird Cloud?** I am running on-premise version of Netbird. Everything installed in Kubernetes. **NetBird version** - Local client: 0.68.0 - Management Platform: 0.70.0 - Router Pod Image: 0.70.0-rootless - Kubernetes Operator Chart: 0.3.1 - Kubernetes Operator Image: 0.3.1 I tested other versions as well, with the same result. **Is any other VPN software installed?** No **Debug output** To help us resolve the problem, please attach the following anonymized status output [netbird-status.txt](https://github.com/user-attachments/files/27173286/netbird-status.txt) **Additional context** Add any other context about the problem here. **Have you tried these troubleshooting steps?** - [X] Reviewed [client troubleshooting](https://docs.netbird.io/how-to/troubleshooting-client) (if applicable) - [X] Checked for newer NetBird versions - [X] Searched for similar issues on GitHub (including closed ones) - [X] Restarted the NetBird client - [X] Disabled other VPN software - [X] Checked firewall settings
saavagebueno added the triage-needed label 2026-08-05 02:06:26 -04:00
Author
Owner

@christianhuth commented on GitHub (May 1, 2026):

I also followed the official docs to test and debug TURN connections: https://docs.netbird.io/selfhosted/troubleshooting#debugging-turn-connections

The results look like this:

IceGatheringState: complete
host   udp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:53002    N/A
host   tcp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:9        N/A
host   udp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:53003    N/A
host   tcp   81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:9        N/A
srflx  udp   10.0.0.13:62611                                     0.0.0.0:0
srflx  udp   10.0.0.13:49039                                     0.0.0.0:0
null   null  undefined:null                                      N/A

According to the docs there should be two relay entries that don't show up here.

<!-- gh-comment-id:4361788915 --> @christianhuth commented on GitHub (May 1, 2026): I also followed the official docs to test and debug TURN connections: https://docs.netbird.io/selfhosted/troubleshooting#debugging-turn-connections The results look like this: ```bash IceGatheringState: complete host udp 81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:53002 N/A host tcp 81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:9 N/A host udp 81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:53003 N/A host tcp 81026960-10c5-4dcd-bfbb-e9a6fa9fb359.local:9 N/A srflx udp 10.0.0.13:62611 0.0.0.0:0 srflx udp 10.0.0.13:49039 0.0.0.0:0 null null undefined:null N/A ``` According to the docs there should be two `relay` entries that don't show up here.
Sign in to join this conversation.
No Label triage-needed
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#12677