[PR #6034] infrastructure_files: expose relay port on UDP so QUIC reaches the listener #28945

Open
opened 2026-08-05 08:07:11 -04:00 by saavagebueno · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/netbirdio/netbird/pull/6034
Author: @dfarrel1
Created: 4/30/2026
Status: 🔄 Open

Base: mainHead: fix/relay-compose-udp-port-mapping


📝 Commits (1)

  • 96af623 infrastructure_files: expose relay port on UDP so QUIC reaches the listener

📊 Changes

1 file changed (+7 additions, -1 deletions)

View changed files

📝 infrastructure_files/docker-compose.yml.tmpl (+7 -1)

📄 Description

Problem

The compose template's relay service exposes the relay port using Docker's HOST:CONTAINER shorthand:

ports:
  - $NETBIRD_RELAY_PORT:$NETBIRD_RELAY_PORT

Docker treats this shorthand as tcp only (per the Compose spec and the Docker -p reference). The relay process internally listens on the same port for both TCP (WebSocket) and UDP (QUIC) — that part works fine inside the container — but docker-proxy on the host never forwards UDP. From the outside, only the WebSocket path is reachable.

Symptom (silent)

The relay log says it's listening for QUIC, so operators have no obvious indication anything is wrong:

INFO relay/server/listener/quic/listener.go:39: QUIC server listening on address: :33073
INFO relay/server/listener/ws/listener.go:44:  WS server listening address: :33073

But on the host:

$ ss -tnlp | grep :33073
LISTEN 0 4096 0.0.0.0:33073 ... docker-proxy   # TCP forwarded
$ ss -unlp | grep :33073
                                               # nothing — UDP not forwarded

$ docker inspect <relay> --format '{{json .HostConfig.PortBindings}}'
{"33073/tcp":[{"HostIp":"","HostPort":"33073"}]}

The daemon-side getDialers() (shared/relay/client/dialers_generic.go) races QUIC and WebSocket dialers in parallel and uses whichever connection completes first. Because UDP packets to the relay are dropped at docker-proxy, the QUIC handshake never completes; only WS is ever a survivable path; WS always wins by default. netbird status does not surface which protocol is in use, so operators have no obvious way to discover the issue.

Net effect: every self-hosted operator following this template is on WebSocket only, never QUIC, and the silent fallback is invisible in default logging on either side.

Fix

Specify both protocols explicitly:

ports:
  - $NETBIRD_RELAY_PORT:$NETBIRD_RELAY_PORT/tcp
  - $NETBIRD_RELAY_PORT:$NETBIRD_RELAY_PORT/udp

Operators with their own external firewall in front of Docker (AWS Security Groups, GCP firewall rules, etc.) also need to open the corresponding UDP port — not part of this PR but probably worth a note in the relay self-hosted docs.

How we found this

Measured on a self-hosted netbird deployment (38 active peers across 3 relays). The 100% WebSocket behavior across the production fleet didn't match the daemon's documented race-dialer behavior. After tracing through dialers_generic.go::getDialers() and the host's port-mapping configuration, identified the silent TCP-only default of -p N:N. Once the explicit /udp mapping was added, the same fleet flipped to majority-QUIC connections (13 QUIC / 1 WS across 14 distinct peer source IPs in a 2-minute observation window post-fix; the 1 WS is consistent with the existing MTU > default branch in getDialers()).

Throughput impact

On a controlled bench env (2 peers behind separate NATGWs, eu-central-1 + il-central-1, iperf3 ramp matching production peer/relay shape) we measured QUIC head-to-head against WS through the same relay binary:

Path Single-flow 4-parallel 16-parallel
WebSocket ~25 Mbps ~23 Mbps ~5 Mbps total (catastrophic collapse)
QUIC ~37 Mbps ~20 Mbps ~9 Mbps total

QUIC is meaningfully better at single-flow and at very-high parallelism (where TCP-over-TCP head-of-line blocking collapses the WS path). At low parallelism the two are comparable. Either way, what's silently lost today is the QUIC option entirely.

Risk

Minimal. Purely additive — TCP/WebSocket continues to work exactly as before. The only behavior change is that operators get the QUIC path that the daemon and relay are already designed to provide.

Test plan

  • Verified the upstream-main infrastructure_files/docker-compose.yml.tmpl still has the TCP-only shorthand (commit 3fc5a8d4a).
  • After the fix, docker inspect <relay> ... HostConfig.PortBindings shows both 33073/tcp and 33073/udp.
  • ss -tnlp | grep :33073 shows TCP LISTEN; ss -unlp | grep :33073 shows UDP UNCONN — both present.
  • tcpdump -i <iface> 'udp port 33073' from a peer shows QUIC traffic flowing.
  • Peer pair through the relay confirms Connection type: Relayed works end-to-end on QUIC.

Summary by CodeRabbit

Release Notes

  • Chores
    • Updated relay service Docker configuration to explicitly support both TCP and UDP protocol connections, improving network connectivity reliability.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/netbirdio/netbird/pull/6034 **Author:** [@dfarrel1](https://github.com/dfarrel1) **Created:** 4/30/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/relay-compose-udp-port-mapping` --- ### 📝 Commits (1) - [`96af623`](https://github.com/netbirdio/netbird/commit/96af6233549b25b9548cb61b3d1463e60b1c0dbf) infrastructure_files: expose relay port on UDP so QUIC reaches the listener ### 📊 Changes **1 file changed** (+7 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `infrastructure_files/docker-compose.yml.tmpl` (+7 -1) </details> ### 📄 Description ## Problem The compose template's `relay` service exposes the relay port using Docker's HOST:CONTAINER shorthand: ```yaml ports: - $NETBIRD_RELAY_PORT:$NETBIRD_RELAY_PORT ``` Docker treats this shorthand as `tcp` only (per the [Compose spec](https://docs.docker.com/reference/compose-file/services/#ports) and the [Docker `-p` reference](https://docs.docker.com/reference/cli/docker/container/run/#publish)). The relay process internally listens on the same port for both TCP (WebSocket) and UDP (QUIC) — that part works fine inside the container — but `docker-proxy` on the host never forwards UDP. From the outside, only the WebSocket path is reachable. ## Symptom (silent) The relay log says it's listening for QUIC, so operators have no obvious indication anything is wrong: ``` INFO relay/server/listener/quic/listener.go:39: QUIC server listening on address: :33073 INFO relay/server/listener/ws/listener.go:44: WS server listening address: :33073 ``` But on the host: ``` $ ss -tnlp | grep :33073 LISTEN 0 4096 0.0.0.0:33073 ... docker-proxy # TCP forwarded $ ss -unlp | grep :33073 # nothing — UDP not forwarded $ docker inspect <relay> --format '{{json .HostConfig.PortBindings}}' {"33073/tcp":[{"HostIp":"","HostPort":"33073"}]} ``` The daemon-side `getDialers()` ([`shared/relay/client/dialers_generic.go`](https://github.com/netbirdio/netbird/blob/main/shared/relay/client/dialers_generic.go)) races QUIC and WebSocket dialers in parallel and uses whichever connection completes first. Because UDP packets to the relay are dropped at docker-proxy, the QUIC handshake never completes; only WS is ever a survivable path; WS always wins by default. `netbird status` does not surface which protocol is in use, so operators have no obvious way to discover the issue. Net effect: every self-hosted operator following this template is on WebSocket only, never QUIC, and the silent fallback is invisible in default logging on either side. ## Fix Specify both protocols explicitly: ```yaml ports: - $NETBIRD_RELAY_PORT:$NETBIRD_RELAY_PORT/tcp - $NETBIRD_RELAY_PORT:$NETBIRD_RELAY_PORT/udp ``` Operators with their own external firewall in front of Docker (AWS Security Groups, GCP firewall rules, etc.) also need to open the corresponding UDP port — not part of this PR but probably worth a note in the relay self-hosted docs. ## How we found this Measured on a self-hosted netbird deployment (38 active peers across 3 relays). The 100% WebSocket behavior across the production fleet didn't match the daemon's documented race-dialer behavior. After tracing through `dialers_generic.go::getDialers()` and the host's port-mapping configuration, identified the silent TCP-only default of `-p N:N`. Once the explicit `/udp` mapping was added, the same fleet flipped to majority-QUIC connections (13 QUIC / 1 WS across 14 distinct peer source IPs in a 2-minute observation window post-fix; the 1 WS is consistent with the existing MTU > default branch in `getDialers()`). ## Throughput impact On a controlled bench env (2 peers behind separate NATGWs, eu-central-1 + il-central-1, iperf3 ramp matching production peer/relay shape) we measured QUIC head-to-head against WS through the same relay binary: | Path | Single-flow | 4-parallel | 16-parallel | |---|---|---|---| | WebSocket | ~25 Mbps | ~23 Mbps | ~5 Mbps total *(catastrophic collapse)* | | QUIC | ~37 Mbps | ~20 Mbps | ~9 Mbps total | QUIC is meaningfully better at single-flow and at very-high parallelism (where TCP-over-TCP head-of-line blocking collapses the WS path). At low parallelism the two are comparable. Either way, what's silently lost today is the QUIC option entirely. ## Risk Minimal. Purely additive — TCP/WebSocket continues to work exactly as before. The only behavior change is that operators get the QUIC path that the daemon and relay are already designed to provide. ## Test plan - [x] Verified the upstream-main `infrastructure_files/docker-compose.yml.tmpl` still has the TCP-only shorthand (commit `3fc5a8d4a`). - [x] After the fix, `docker inspect <relay> ... HostConfig.PortBindings` shows both `33073/tcp` and `33073/udp`. - [x] `ss -tnlp | grep :33073` shows TCP `LISTEN`; `ss -unlp | grep :33073` shows UDP `UNCONN` — both present. - [x] `tcpdump -i <iface> 'udp port 33073'` from a peer shows QUIC traffic flowing. - [x] Peer pair through the relay confirms `Connection type: Relayed` works end-to-end on QUIC. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Chores** * Updated relay service Docker configuration to explicitly support both TCP and UDP protocol connections, improving network connectivity reliability. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
saavagebueno added the pull-request label 2026-08-05 08:07:11 -04:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#28945