[PR #5653] fix: add netlink.RouteGet() fallback for ICE/P2P on devices with policy-based routing #23527

Open
opened 2026-08-05 06:07:20 -04:00 by saavagebueno · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/netbirdio/netbird/pull/5653
Author: @thvevirtue
Created: 3/23/2026
Status: 🔄 Open

Base: mainHead: fix/policy-routing-ice-fallback


📝 Commits (1)

  • e3101a8 fix: add netlink.RouteGet() fallback for policy routing support

📊 Changes

4 files changed (+106 additions, -3 deletions)

View changed files

client/internal/routemanager/systemops/route_fallback_linux.go (+58 -0)
client/internal/routemanager/systemops/route_fallback_other.go (+12 -0)
📝 client/internal/routemanager/systemops/systemops_generic.go (+10 -2)
📝 sharedsock/sock_linux.go (+26 -1)

📄 Description

Problem

On devices using policy-based routing (e.g., Ubiquiti UniFi gateways), the default route is not in the main routing table but in a separate policy table (e.g., 201.eth4). NetBird uses go-netroute for route lookups, which only reads the main table via syscall.NetlinkRIB(RTM_GETROUTE, AF_UNSPEC). This causes two failures:

  1. SharedSocket.WriteTo() (sharedsock/sock_linux.go) — calls s.router.Route() to determine source IP for raw UDP packets. Fails with Route referenced unknown interface.
  2. GetNextHop() (systemops_generic.go) — called via write hooks when ICE sends packets. Same failure.

Result: All STUN and ICE connectivity-check packets are dropped. Local candidates are gathered and remote candidates are received, but no packets can be sent. ICE always times out and falls back to relay. Normal TCP traffic (management/signal/relay) works fine because standard Go net.Dial lets the kernel handle routing.

Example routing setup (UXG-Max)

\\

ip route show table main

192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.1
(... only directly connected subnets, NO default route)

ip route show table 201.eth4

default via dev eth4 proto dhcp metric 200

ip rule show

0: from all lookup local
32000: from all lookup main
32766: from all lookup 201.eth4
32767: from all lookup default
\\

Key log lines

\\
DEBG systemops_generic.go:348: Failed to get route for : Route referenced unknown interface
ice WARNING: Failed get server reflexive address: failed to send STUN packet: got an error while checking route, err: Route referenced unknown interface
ice INFO: Failed to send packet: got an error while checking route, err: Route referenced unknown interface
(repeated hundreds of times for every peer)
\\

Fix

Adds a netlink.RouteGet() fallback when go-netroute's Route() fails. netlink.RouteGet() asks the kernel to perform the actual routing decision, respecting policy routing rules (ip rule) and all tables — exactly matching what happens for real packets.

Changes

  • sharedsock/sock_linux.goWriteTo(): when s.router.Route() fails, fall back to netlink.RouteGet() for source IP resolution
  • systemops_generic.goGetNextHop(): same fallback pattern
  • route_fallback_linux.go (new) — Linux-specific getNextHopViaNetlink() using netlink.RouteGet()
  • route_fallback_other.go (new) — no-op stub for non-Linux platforms

Safety

  • Fallback only activates when go-netroute fails — devices with standard routing are completely unaffected
  • vishvananda/netlink is already a dependency
  • netlink.RouteGet() is the standard kernel route query mechanism on Linux

Testing

Tested and confirmed working on Ubiquiti UXG-Max (kernel 5.4.213-ui-ipq5322, ARM64):

  • Before: all 5 peers Connection type: Relayed, ICE candidates -/-
  • After: peers switch to Connection type: P2P

Affected platforms

  • Ubiquiti UXG-Max, UXG-Enterprise, UDM Pro (firmware 4.0+)
  • Likely all UniFi OS gateway devices
  • OpenWRT devices with mwan3 or custom policy routing
  • Any Linux device where the default route is in a non-main routing table
  • #5551 — Separate: iptable_raw crash on kernels without the module
  • #2530 — Teltonika RutOS, possibly same underlying cause
  • #2512 — OpenWRT, connect: invalid argument
  • #2116 — UniFi UDM Pro firmware 4.0

Co-Authored-By: Oz oz-agent@warp.dev

Summary by CodeRabbit

Release Notes

  • Improvements
    • Enhanced routing system with improved fallback mechanisms for more reliable network connectivity on Linux
    • Improved source IP address determination when routing network packets
    • Better cross-platform compatibility with graceful error handling when standard route lookups fail
    • More robust network packet handling with enhanced source address selection

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/netbirdio/netbird/pull/5653 **Author:** [@thvevirtue](https://github.com/thvevirtue) **Created:** 3/23/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/policy-routing-ice-fallback` --- ### 📝 Commits (1) - [`e3101a8`](https://github.com/netbirdio/netbird/commit/e3101a88cea3ac33544cd4e8cc861ab1f0b3bc43) fix: add netlink.RouteGet() fallback for policy routing support ### 📊 Changes **4 files changed** (+106 additions, -3 deletions) <details> <summary>View changed files</summary> ➕ `client/internal/routemanager/systemops/route_fallback_linux.go` (+58 -0) ➕ `client/internal/routemanager/systemops/route_fallback_other.go` (+12 -0) 📝 `client/internal/routemanager/systemops/systemops_generic.go` (+10 -2) 📝 `sharedsock/sock_linux.go` (+26 -1) </details> ### 📄 Description ## Problem On devices using policy-based routing (e.g., Ubiquiti UniFi gateways), the default route is not in the `main` routing table but in a separate policy table (e.g., `201.eth4`). NetBird uses `go-netroute` for route lookups, which only reads the main table via `syscall.NetlinkRIB(RTM_GETROUTE, AF_UNSPEC)`. This causes two failures: 1. **`SharedSocket.WriteTo()`** (`sharedsock/sock_linux.go`) — calls `s.router.Route()` to determine source IP for raw UDP packets. Fails with `Route referenced unknown interface`. 2. **`GetNextHop()`** (`systemops_generic.go`) — called via write hooks when ICE sends packets. Same failure. **Result:** All STUN and ICE connectivity-check packets are dropped. Local candidates are gathered and remote candidates are received, but no packets can be sent. ICE always times out and falls back to relay. Normal TCP traffic (management/signal/relay) works fine because standard Go `net.Dial` lets the kernel handle routing. ### Example routing setup (UXG-Max) \\\ # ip route show table main 192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.1 (... only directly connected subnets, NO default route) # ip route show table 201.eth4 default via <gateway> dev eth4 proto dhcp metric 200 # ip rule show 0: from all lookup local 32000: from all lookup main 32766: from all lookup 201.eth4 32767: from all lookup default \\\ ### Key log lines \\\ DEBG systemops_generic.go:348: Failed to get route for <IP>: Route referenced unknown interface ice WARNING: Failed get server reflexive address: failed to send STUN packet: got an error while checking route, err: Route referenced unknown interface ice INFO: Failed to send packet: got an error while checking route, err: Route referenced unknown interface (repeated hundreds of times for every peer) \\\ ## Fix Adds a `netlink.RouteGet()` fallback when `go-netroute`'s `Route()` fails. `netlink.RouteGet()` asks the kernel to perform the actual routing decision, respecting policy routing rules (`ip rule`) and all tables — exactly matching what happens for real packets. ### Changes - **`sharedsock/sock_linux.go`** — `WriteTo()`: when `s.router.Route()` fails, fall back to `netlink.RouteGet()` for source IP resolution - **`systemops_generic.go`** — `GetNextHop()`: same fallback pattern - **`route_fallback_linux.go`** (new) — Linux-specific `getNextHopViaNetlink()` using `netlink.RouteGet()` - **`route_fallback_other.go`** (new) — no-op stub for non-Linux platforms ### Safety - Fallback only activates when `go-netroute` fails — devices with standard routing are completely unaffected - `vishvananda/netlink` is already a dependency - `netlink.RouteGet()` is the standard kernel route query mechanism on Linux ## Testing **Tested and confirmed working** on Ubiquiti UXG-Max (kernel `5.4.213-ui-ipq5322`, ARM64): - Before: all 5 peers `Connection type: Relayed`, ICE candidates `-/-` - After: peers switch to `Connection type: P2P` ## Affected platforms - Ubiquiti UXG-Max, UXG-Enterprise, UDM Pro (firmware 4.0+) - Likely all UniFi OS gateway devices - OpenWRT devices with `mwan3` or custom policy routing - Any Linux device where the default route is in a non-main routing table ## Related issues - #5551 — Separate: `iptable_raw` crash on kernels without the module - #2530 — Teltonika RutOS, possibly same underlying cause - #2512 — OpenWRT, `connect: invalid argument` - #2116 — UniFi UDM Pro firmware 4.0 Co-Authored-By: Oz <oz-agent@warp.dev> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Improvements** * Enhanced routing system with improved fallback mechanisms for more reliable network connectivity on Linux * Improved source IP address determination when routing network packets * Better cross-platform compatibility with graceful error handling when standard route lookups fail * More robust network packet handling with enhanced source address selection <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
saavagebueno added the pull-request label 2026-08-05 06:07:20 -04:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#23527