client can't use p2p connection #2003

Closed
opened 2025-11-20 06:11:01 -05:00 by saavagebueno · 2 comments
Owner

Originally created by @SamPeng87 on GitHub (Jun 23, 2025).

Describe the problem
First, I have a routeros as the main router. The main configuration features are as follows:

  • Hairpinning NAT, because the STUN domain needs to be accessed from the internal network
  • NAT1, routeros only supports UDP, and I’m not sure if this affects netbird

Secondly, I have 3 VLANs. Each of the first two VLANs has a netbird client, and the other VLAN has a netbird self-hosted deployment in Docker. Yes, I have deployed netbird’s STUN and client within the internal network because I don’t have a VPS to do this.

The current issue is:

The client on the internal network shows the public IP as the internal IP 192.168.xx.xx. I am not sure if this has any impact; the connection status between them is P2P. This is good.
However, any device connecting to the peer from the external network shows as Relayed. I have been trying for a whole day.

There is only one situation where it improves:
In the coturn server configuration, setting --external-ip="$(detect-external-ip)/priverip" is not practical because my external IP changes frequently. I have a dynamic external IP. The docker cmd is static.

I don't know where the problem is or how to troubleshoot it. Checking the debug logs.
PIONS_LOG_DEBUG=all NB_LOG_LEVEL=debug netbird up -F

can't connect by p2p

2025-06-23T17:20:56+08:00 INFO [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/handshaker.go:89: received connection confirmation, running version 0.47.1 and with remote WireGuard listen port 51820
2025-06-23T17:20:56+08:00 INFO [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/handshaker.go:77: wait for remote offer confirmation
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:82: OnNewOffer for ICE
2025-06-23T17:20:56+08:00 DEBG relay/client/manager.go:143: open peer connection via permanent server: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_relay.go:68: handled offer by reusing existing relay connection
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:98: recreate ICE agent
2025-06-23T17:20:56+08:00 INFO client/internal/peer/ice/env.go:48: setting ICE disconnected timeout to 30 seconds
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:109: gather candidates
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:119: turn agent dial
ice DEBUG: 17:20:56.712477 agent.go:399: Started agent: isControlling? false, remoteUfrag: "KRPbHwIEAJERcQwb", remotePwd: "abbhhqaeHSsEkzBjAVwkaKpkvgzsrOnk"
ice INFO: 2025/06/23 17:20:56 Setting new connection state: Checking
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:212: ICE ConnectionState has changed to Checking
ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet.
ice WARNING: 2025/06/23 17:20:56 UDPMuxDefault should not listen on unspecified address, use NewMultiUDPMuxFromPort instead
ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet.
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:300: discovered local candidate udp4 host 0.0.0.0:51823
ice DEBUG: 17:20:56.713625 agent.go:820: Ignore duplicate candidate: udp4 srflx <public ip>:9168 related 0.0.0.0:51823
ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet.
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:300: discovered local candidate udp4 srflx <public ip>:9168 related 0.0.0.0:51823
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 172.21.128.1:51820
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp6 host [2408:8207:8435:2160:483a:e036:d745:883c]:51820
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 172.21.128.1:51820 for bSycnEodzARkiKEw
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 172.21.128.1:51820 for bSycnEodzARkiKEwstun:home.isampeng.net:3478
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 192.168.10.254:51820
2025-06-23T17:20:56+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 host 192.168.10.254:51820], its address is part of routed network 192.168.10.0/24
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp6 host [2408:8207:8435:2160:80c8:370c:6ba7:baba]:51820
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 127.0.0.1:51820
2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 srflx 192.168.10.254:51820 related 0.0.0.0:51820
2025-06-23T17:20:56+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 srflx 192.168.10.254:51820 related 0.0.0.0:51820], its address is part of routed network 192.168.10.0/24
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 127.0.0.1:51820 for bSycnEodzARkiKEw
2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 127.0.0.1:51820 for bSycnEodzARkiKEwstun:home.isampeng.net:3478
2025-06-23T17:20:58+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:578: send offer to peer
2025-06-23T17:20:58+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:282: OnRemoteAnswer, priority: PriorityRelay, status ICE: Disconnected, status relay: Connected

can connection by p2p

025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [2408:8207:8435:2160:10cb:3eb8:db30:5470]:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.10.253:51821
2025-06-23T17:27:47+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 host 192.168.10.253:51821], its address is part of routed network 192.168.10.0/24
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 198.19.249.3:51821
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 198.19.249.3:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.97.0:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [2408:8207:8435:2160:25ce:fa51:ec1e:df84]:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [fd07:b51a:cc66:0:a617:db5e:ab7:e9f1]:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.107.0:51821
2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 srflx 124.64.234.240:51821 related 0.0.0.0:51821
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 198.19.249.3:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.97.0:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.97.0:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.107.0:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.107.0:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 124.64.234.240:51821 for bxLpglfeAPPoILTQ
2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 124.64.234.240:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478
2025-06-23T17:27:47+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:531: start to communicate with peer via relay
2025-06-23T17:27:47+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/lazyconn/manager/manager.go:496: peer connected, pausing inactivity monitor while connection is not disconnected
2025-06-23T17:27:48+08:00 INFO [peer: UqVSMbHF+9DZE7rrc6vcgjgS4SD75kgfSyy4SJ7sjiM=] client/internal/peer/guard/guard.go:74: start reconnect loop...
2025-06-23T17:27:48+08:00 INFO [peer: o3ZzcLJu2RU/WwxWzF6/LeOLCEJr5WZqF6OjSbSRBF8=] client/internal/peer/guard/guard.go:74: start reconnect loop...
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:212: ICE ConnectionState has changed to Connected
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:125: agent dial succeeded
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:330: selected candidate pair [local <-> remote] -> [udp4 srflx <public ip>:9168 related 0.0.0.0:51823 <-> udp4 srflx 124.64.234.240:51821 related 0.0.0.0:51821], peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:154: on ICE conn is ready to use
2025-06-23T17:27:48+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:359: set ICE to active connection
2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/wg_watcher.go:80: disable WireGuard watcher
2025-06-23T17:27:48+08:00 DEBG client/iface/iface.go:120: updating interface utun100 peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=, endpoint 124.64.234.240:51821, allowedIPs [100.116.237.209/32]
2025-06-23T17:27:48+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/wg_watcher.go:112: WireGuard watcher stopped

I suspect it is caused by a routing issue because last week I was using a non-routeros as the main router, which was a very ordinary unmanaged router, but when I connected through that router, P2P worked normally. After changing the router, it stopped working. The network topology hasn't changed much. I have tried routeros many times but haven't found any good solution.

To Reproduce

Steps to reproduce the behavior:

Expected behavior

all outside client can use p2p connection type

Are you using NetBird Cloud?

self-host NetBird's control plane.

NetBird version

0.47

Is any other VPN software installed?

no

Debug output

To help us resolve the problem, please attach the following anonymized status output

netbird status -dA

Peers detail:
 desktop-mvk81eo.anon-E3Ef7.domain:
  NetBird IP: 100.116.34.5
  Public key: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 8 seconds ago
  Last WireGuard handshake: 1 minute, 39 seconds ago
  Transfer status (received/sent) 408 B/616 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

 friendlywrt.anon-E3Ef7.domain:
  NetBird IP: 100.116.65.72
  Public key: o3ZzcLJu2RU/WwxWzF6/LeOLCEJr5WZqF6OjSbSRBF8=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 15 seconds ago
  Last WireGuard handshake: 15 seconds ago
  Transfer status (received/sent) 665.6 KiB/145.2 KiB
  Quantum resistance: false
  Networks: 192.168.10.0/24
  Latency: 0s

 iphone-admin.anon-E3Ef7.domain:
  NetBird IP: 100.116.146.43
  Public key: UqVSMbHF+9DZE7rrc6vcgjgS4SD75kgfSyy4SJ7sjiM=
  Status: Connected
  -- detail --
  Connection type: Relayed
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 15 seconds ago
  Last WireGuard handshake: 5 seconds ago
  Transfer status (received/sent) 308 B/860 B
  Quantum resistance: false
  Networks: -
  Latency: 0s

 penglingjundemac-mini.anon-E3Ef7.domain:
  NetBird IP: 100.116.237.209
  Public key: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=
  Status: Connected
  -- detail --
  Connection type: P2P
  ICE candidate (Local/Remote): srflx/srflx
  ICE candidate endpoints (Local/Remote): 198.51.100.0:9168/198.51.100.1:51821
  Relay server address: rels://home.anon-gHZ82.domain:1203
  Last connection update: 4 minutes, 8 seconds ago
  Last WireGuard handshake: 9 seconds ago
  Transfer status (received/sent) 41.3 KiB/33.2 KiB
  Quantum resistance: false
  Networks: -
  Latency: 11.088333ms

Events:
  [INFO] SYSTEM (963dbb96-e643-4004-ab2c-1f4cf7d546f0)
    Message: Network map updated
    Time: 4 minutes, 16 seconds ago
OS: darwin/arm64
Daemon version: 0.47.1
CLI version: 0.47.1
Management: Connected to https://home.anon-gHZ82.domain:1203
Signal: Connected to https://home.anon-gHZ82.domain:1203
Relays:
  [stun:home.anon-gHZ82.domain:3478] is Available
  [turn:home.anon-gHZ82.domain:3478?transport=udp] is Available
  [rels://home.anon-gHZ82.domain:1203] is Available
Nameservers:
FQDN: penglingjundemac-studio-7.anon-E3Ef7.domain
NetBird IP: 100.116.205.173/16
Interface type: Userspace
Quantum resistance: false
Lazy connection: true
Networks: -
Forwarding rules: 0
Peers count: 4/4 Connected

Create and upload a debug bundle, and share the returned file key:

netbird debug for 1m -AS -U

use external-ip ,has one p2p
8f231557c3b2a09fc122090c8738b19a618321b016d4b5bf41def293d760b7c0/7d8079fb-0c1c-4a3a-86d6-9a3bf1aa0bd4

remove external-ip,no any p2p
8f231557c3b2a09fc122090c8738b19a618321b016d4b5bf41def293d760b7c0/14de4869-6f0e-4e6b-8f40-b71694e92800

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • Disabled other VPN software
  • Checked firewall settings
Originally created by @SamPeng87 on GitHub (Jun 23, 2025). **Describe the problem** First, I have a routeros as the main router. The main configuration features are as follows: * Hairpinning NAT, because the STUN domain needs to be accessed from the internal network * NAT1, routeros only supports UDP, and I’m not sure if this affects netbird Secondly, I have 3 VLANs. Each of the first two VLANs has a netbird client, and the other VLAN has a netbird self-hosted deployment in Docker. Yes, I have deployed netbird’s STUN and client within the internal network because I don’t have a VPS to do this. The current issue is: The client on the internal network shows the public IP as the internal IP 192.168.xx.xx. I am not sure if this has any impact; the connection status between them is P2P. This is good. However, any device connecting to the peer from the external network shows as Relayed. I have been trying for a whole day. There is only one situation where it improves: In the coturn server configuration, setting --external-ip="$(detect-external-ip)/priverip" is not practical because my external IP changes frequently. I have a dynamic external IP. The docker cmd is static. I don't know where the problem is or how to troubleshoot it. Checking the debug logs. PIONS_LOG_DEBUG=all NB_LOG_LEVEL=debug netbird up -F can't connect by p2p ``` 2025-06-23T17:20:56+08:00 INFO [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/handshaker.go:89: received connection confirmation, running version 0.47.1 and with remote WireGuard listen port 51820 2025-06-23T17:20:56+08:00 INFO [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/handshaker.go:77: wait for remote offer confirmation 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:82: OnNewOffer for ICE 2025-06-23T17:20:56+08:00 DEBG relay/client/manager.go:143: open peer connection via permanent server: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_relay.go:68: handled offer by reusing existing relay connection 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:98: recreate ICE agent 2025-06-23T17:20:56+08:00 INFO client/internal/peer/ice/env.go:48: setting ICE disconnected timeout to 30 seconds 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:109: gather candidates 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:119: turn agent dial ice DEBUG: 17:20:56.712477 agent.go:399: Started agent: isControlling? false, remoteUfrag: "KRPbHwIEAJERcQwb", remotePwd: "abbhhqaeHSsEkzBjAVwkaKpkvgzsrOnk" ice INFO: 2025/06/23 17:20:56 Setting new connection state: Checking 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:212: ICE ConnectionState has changed to Checking ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet. ice WARNING: 2025/06/23 17:20:56 UDPMuxDefault should not listen on unspecified address, use NewMultiUDPMuxFromPort instead ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet. 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:300: discovered local candidate udp4 host 0.0.0.0:51823 ice DEBUG: 17:20:56.713625 agent.go:820: Ignore duplicate candidate: udp4 srflx <public ip>:9168 related 0.0.0.0:51823 ice WARNING: 2025/06/23 17:20:56 Failed to ping without candidate pairs. Connection is not possible yet. 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:300: discovered local candidate udp4 srflx <public ip>:9168 related 0.0.0.0:51823 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 172.21.128.1:51820 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp6 host [2408:8207:8435:2160:483a:e036:d745:883c]:51820 2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 172.21.128.1:51820 for bSycnEodzARkiKEw 2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 172.21.128.1:51820 for bSycnEodzARkiKEwstun:home.isampeng.net:3478 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 192.168.10.254:51820 2025-06-23T17:20:56+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 host 192.168.10.254:51820], its address is part of routed network 192.168.10.0/24 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp6 host [2408:8207:8435:2160:80c8:370c:6ba7:baba]:51820 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 host 127.0.0.1:51820 2025-06-23T17:20:56+08:00 DEBG [peer: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= -> udp4 srflx 192.168.10.254:51820 related 0.0.0.0:51820 2025-06-23T17:20:56+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 srflx 192.168.10.254:51820 related 0.0.0.0:51820], its address is part of routed network 192.168.10.0/24 2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 127.0.0.1:51820 for bSycnEodzARkiKEw 2025-06-23T17:20:56+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 127.0.0.1:51820 for bSycnEodzARkiKEwstun:home.isampeng.net:3478 2025-06-23T17:20:58+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:578: send offer to peer 2025-06-23T17:20:58+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:282: OnRemoteAnswer, priority: PriorityRelay, status ICE: Disconnected, status relay: Connected ``` can connection by p2p ``` 025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [2408:8207:8435:2160:10cb:3eb8:db30:5470]:51821 2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.10.253:51821 2025-06-23T17:27:47+08:00 DEBG client/internal/peer/worker_ice.go:383: Ignoring candidate [udp4 host 192.168.10.253:51821], its address is part of routed network 192.168.10.0/24 2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 198.19.249.3:51821 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 198.19.249.3:51821 for bxLpglfeAPPoILTQ 2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.97.0:51821 2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [2408:8207:8435:2160:25ce:fa51:ec1e:df84]:51821 2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp6 host [fd07:b51a:cc66:0:a617:db5e:ab7:e9f1]:51821 2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 host 192.168.107.0:51821 2025-06-23T17:27:47+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:162: OnRemoteCandidate from peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= -> udp4 srflx 124.64.234.240:51821 related 0.0.0.0:51821 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 198.19.249.3:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.97.0:51821 for bxLpglfeAPPoILTQ 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.97.0:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.107.0:51821 for bxLpglfeAPPoILTQ 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 192.168.107.0:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 124.64.234.240:51821 for bxLpglfeAPPoILTQ 2025-06-23T17:27:47+08:00 DEBG client/iface/bind/udp_mux.go:363: ICE: registered 124.64.234.240:51821 for bxLpglfeAPPoILTQstun:home.isampeng.net:3478 2025-06-23T17:27:47+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:531: start to communicate with peer via relay 2025-06-23T17:27:47+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/lazyconn/manager/manager.go:496: peer connected, pausing inactivity monitor while connection is not disconnected 2025-06-23T17:27:48+08:00 INFO [peer: UqVSMbHF+9DZE7rrc6vcgjgS4SD75kgfSyy4SJ7sjiM=] client/internal/peer/guard/guard.go:74: start reconnect loop... 2025-06-23T17:27:48+08:00 INFO [peer: o3ZzcLJu2RU/WwxWzF6/LeOLCEJr5WZqF6OjSbSRBF8=] client/internal/peer/guard/guard.go:74: start reconnect loop... 2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:212: ICE ConnectionState has changed to Connected 2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:125: agent dial succeeded 2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:330: selected candidate pair [local <-> remote] -> [udp4 srflx <public ip>:9168 related 0.0.0.0:51823 <-> udp4 srflx 124.64.234.240:51821 related 0.0.0.0:51821], peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= 2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/worker_ice.go:154: on ICE conn is ready to use 2025-06-23T17:27:48+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/conn.go:359: set ICE to active connection 2025-06-23T17:27:48+08:00 DEBG [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/wg_watcher.go:80: disable WireGuard watcher 2025-06-23T17:27:48+08:00 DEBG client/iface/iface.go:120: updating interface utun100 peer 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=, endpoint 124.64.234.240:51821, allowedIPs [100.116.237.209/32] 2025-06-23T17:27:48+08:00 INFO [peer: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY=] client/internal/peer/wg_watcher.go:112: WireGuard watcher stopped ``` I suspect it is caused by a routing issue because last week I was using a non-routeros as the main router, which was a very ordinary unmanaged router, but when I connected through that router, P2P worked normally. After changing the router, it stopped working. The network topology hasn't changed much. I have tried routeros many times but haven't found any good solution. **To Reproduce** Steps to reproduce the behavior: **Expected behavior** all outside client can use p2p connection type **Are you using NetBird Cloud?** self-host NetBird's control plane. **NetBird version** 0.47 **Is any other VPN software installed?** no **Debug output** To help us resolve the problem, please attach the following anonymized status output netbird status -dA ``` Peers detail: desktop-mvk81eo.anon-E3Ef7.domain: NetBird IP: 100.116.34.5 Public key: zcGutZ3Jd6XfL/n95U6w2/9C5Nv6Ks81ZosMwqZdDDM= Status: Connected -- detail -- Connection type: Relayed ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: rels://home.anon-gHZ82.domain:1203 Last connection update: 4 minutes, 8 seconds ago Last WireGuard handshake: 1 minute, 39 seconds ago Transfer status (received/sent) 408 B/616 B Quantum resistance: false Networks: - Latency: 0s friendlywrt.anon-E3Ef7.domain: NetBird IP: 100.116.65.72 Public key: o3ZzcLJu2RU/WwxWzF6/LeOLCEJr5WZqF6OjSbSRBF8= Status: Connected -- detail -- Connection type: Relayed ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: rels://home.anon-gHZ82.domain:1203 Last connection update: 4 minutes, 15 seconds ago Last WireGuard handshake: 15 seconds ago Transfer status (received/sent) 665.6 KiB/145.2 KiB Quantum resistance: false Networks: 192.168.10.0/24 Latency: 0s iphone-admin.anon-E3Ef7.domain: NetBird IP: 100.116.146.43 Public key: UqVSMbHF+9DZE7rrc6vcgjgS4SD75kgfSyy4SJ7sjiM= Status: Connected -- detail -- Connection type: Relayed ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Relay server address: rels://home.anon-gHZ82.domain:1203 Last connection update: 4 minutes, 15 seconds ago Last WireGuard handshake: 5 seconds ago Transfer status (received/sent) 308 B/860 B Quantum resistance: false Networks: - Latency: 0s penglingjundemac-mini.anon-E3Ef7.domain: NetBird IP: 100.116.237.209 Public key: 5Do+fBY1bBfJ2xJa4i06qmUUfAsBq3VmNj+ObM7cyRY= Status: Connected -- detail -- Connection type: P2P ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): 198.51.100.0:9168/198.51.100.1:51821 Relay server address: rels://home.anon-gHZ82.domain:1203 Last connection update: 4 minutes, 8 seconds ago Last WireGuard handshake: 9 seconds ago Transfer status (received/sent) 41.3 KiB/33.2 KiB Quantum resistance: false Networks: - Latency: 11.088333ms Events: [INFO] SYSTEM (963dbb96-e643-4004-ab2c-1f4cf7d546f0) Message: Network map updated Time: 4 minutes, 16 seconds ago OS: darwin/arm64 Daemon version: 0.47.1 CLI version: 0.47.1 Management: Connected to https://home.anon-gHZ82.domain:1203 Signal: Connected to https://home.anon-gHZ82.domain:1203 Relays: [stun:home.anon-gHZ82.domain:3478] is Available [turn:home.anon-gHZ82.domain:3478?transport=udp] is Available [rels://home.anon-gHZ82.domain:1203] is Available Nameservers: FQDN: penglingjundemac-studio-7.anon-E3Ef7.domain NetBird IP: 100.116.205.173/16 Interface type: Userspace Quantum resistance: false Lazy connection: true Networks: - Forwarding rules: 0 Peers count: 4/4 Connected ``` Create and upload a debug bundle, and share the returned file key: netbird debug for 1m -AS -U use external-ip ,has one p2p 8f231557c3b2a09fc122090c8738b19a618321b016d4b5bf41def293d760b7c0/7d8079fb-0c1c-4a3a-86d6-9a3bf1aa0bd4 remove external-ip,no any p2p 8f231557c3b2a09fc122090c8738b19a618321b016d4b5bf41def293d760b7c0/14de4869-6f0e-4e6b-8f40-b71694e92800 **Screenshots** If applicable, add screenshots to help explain your problem. **Additional context** Add any other context about the problem here. **Have you tried these troubleshooting steps?** - [x] Reviewed [client troubleshooting](https://docs.netbird.io/how-to/troubleshooting-client) (if applicable) - [x] Checked for newer NetBird versions - [x] Searched for similar issues on GitHub (including closed ones) - [x] Restarted the NetBird client - [x] Disabled other VPN software - [x] Checked firewall settings
saavagebueno added the triage-needed label 2025-11-20 06:11:01 -05:00
Author
Owner

@Silex commented on GitHub (Jun 23, 2025):

Have one of the peers (or both) allow inbound UDP 51820.

@Silex commented on GitHub (Jun 23, 2025): Have one of the peers (or both) allow inbound UDP 51820.
Author
Owner

@SamPeng87 commented on GitHub (Jun 24, 2025):

Have one of the peers (or both) allow inbound UDP 51820.

Thank your Quick reply,I slove that problem

Not exactly. The public IP gave me inspiration. I suddenly realized that ICE is handled by coturn, and essentially coturn is responsible for correctly distributing the ports for WG. I studied this all day and found that this is very important for self-hosting. If the self-host is on the public internet, this is not a problem. Any peer accessing STUN will get the public IP. But for a private network, it’s not necessarily the case, especially if the self-host machine is behind NAT and the NAT is not properly configured. This issue will occur. I searched all possible keywords but couldn’t find anything in the issues list.

Leaving this here to help others solve similar problems later.

From a Layer 3 perspective, whenever any IP reaches the STUN server, if the NAT is a Hairpinning NAT, the STUN server will see a 3478 request coming from an internal IP. So STUN can never know that the peer should be given the public IP. I spent a long time reviewing coturn’s manual and configuration files, and the purpose of the external-ip parameter is to associate this external IP with the private IP. So I thought, why not keep it simple? I set up a source NAT (src-nat) on the router for internal sources on port 3478. That is, I made the packets masquerade as if they came from the public IP, so the STUN server sees the peer as a pure public IP. I ran an experiment, and immediately the external peer could see the ICE offer confirming the public IP. Then all that was left was to properly forward the WG peer ports on the firewall (or maybe not? I’m not sure if netbird does hole punching? Because my test worked perfectly this way). After doing this, four peers on the internal network, a phone on 4G, and a laptop on WiFi all could correctly use P2P.

@SamPeng87 commented on GitHub (Jun 24, 2025): > Have one of the peers (or both) allow inbound UDP 51820. Thank your Quick reply,I slove that problem Not exactly. The public IP gave me inspiration. I suddenly realized that ICE is handled by coturn, and essentially coturn is responsible for correctly distributing the ports for WG. I studied this all day and found that this is very important for self-hosting. If the self-host is on the public internet, this is not a problem. Any peer accessing STUN will get the public IP. But for a private network, it’s not necessarily the case, especially if the self-host machine is behind NAT and the NAT is not properly configured. This issue will occur. I searched all possible keywords but couldn’t find anything in the issues list. Leaving this here to help others solve similar problems later. From a Layer 3 perspective, whenever any IP reaches the STUN server, if the NAT is a Hairpinning NAT, the STUN server will see a 3478 request coming from an internal IP. So STUN can never know that the peer should be given the public IP. I spent a long time reviewing coturn’s manual and configuration files, and the purpose of the external-ip parameter is to associate this external IP with the private IP. So I thought, why not keep it simple? I set up a source NAT (src-nat) on the router for internal sources on port 3478. That is, I made the packets masquerade as if they came from the public IP, so the STUN server sees the peer as a pure public IP. I ran an experiment, and immediately the external peer could see the ICE offer confirming the public IP. Then all that was left was to properly forward the WG peer ports on the firewall (or maybe not? I’m not sure if netbird does hole punching? Because my test worked perfectly this way). After doing this, four peers on the internal network, a phone on 4G, and a laptop on WiFi all could correctly use P2P.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#2003