After upgrading to Netbird v0.51.1 on Ubuntu started error: Required key not available #2086

Open
opened 2025-11-20 06:12:31 -05:00 by saavagebueno · 5 comments
Owner

Originally created by @gcervantes2005 on GitHub (Jul 21, 2025).

Describe the problem
After upgrading to Netbird v0.51.1 or installing it fresh on Ubuntu, attempts to ping hosts outside local LAN result in this error:

From 100.74.218.190 icmp_seq=3 Destination Host Unreachable
ping: sendmsg: Required key not available

A clear and concise description of what the problem is.

To Reproduce
Install version 0.51.1 on ubuntu on an existing network. Server is running on docker below is the yaml

`version: "3.4"
services:

Caddy reverse proxy

caddy:
image: caddy
restart: unless-stopped
networks: [ netbird ]
ports:
- '443:443'
- '443:443/udp'
- '80:80'
- '8080:8080'
volumes:
- netbird_caddy_data:/data
- ./Caddyfile:/etc/caddy/Caddyfile
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"

UI dashboard

dashboard:
image: netbirdio/dashboard:latest
restart: unless-stopped
networks: [netbird]
env_file:
- ./dashboard.env
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"

Signal

signal:
image: netbirdio/signal:latest
restart: unless-stopped
networks: [netbird]
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"

Relay

relay:
image: netbirdio/relay:latest
restart: unless-stopped
networks: [netbird]
env_file:
- ./relay.env
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"

Management

management:
image: netbirdio/management:latest
restart: unless-stopped
networks: [netbird]
volumes:
- netbird_management:/var/lib/netbird
- ./management.json:/etc/netbird/management.json
command: [
"--port", "80",
"--log-file", "console",
"--log-level", "info",
"--disable-anonymous-metrics=false",
"--single-account-mode-domain=netbird.selfhosted",
"--dns-domain=netbird.selfhosted",
"--idp-sign-key-refresh-enabled",
]
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"

Coturn, AKA relay server

coturn:
image: coturn/coturn
restart: unless-stopped
#domainname: netbird.relay.selfhosted
volumes:
- ./turnserver.conf:/etc/turnserver.conf:ro
network_mode: host
command:
- -c /etc/turnserver.conf
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"

Zitadel - identity provider

zitadel:
restart: 'always'
networks: [netbird]
image: 'ghcr.io/zitadel/zitadel:v2.64.1'
command: 'start-from-init --masterkeyFromEnv --tlsMode external'
env_file:
- ./zitadel.env
depends_on:
zdb:
condition: 'service_healthy'
volumes:
- ./machinekey:/machinekey
- netbird_zitadel_certs:/zdb-certs:ro
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"

Postgres for Zitadel

zdb:
restart: 'always'
networks: [netbird]
image: 'postgres:16-alpine'
env_file:
- ./zdb.env
volumes:
- netbird_zdb_data:/var/lib/postgresql/data:rw
healthcheck:
test: ["CMD-SHELL", "pg_isready", "-d", "db_prod"]
interval: 5s
timeout: 60s
retries: 10
start_period: 5s
logging:
driver: "json-file"
options:
max-size: "500m"
max-file: "2"
volumes:
netbird_zdb_data:
netbird_management:
netbird_caddy_data:
netbird_zitadel_certs:

networks:
netbird:`

I fixed it by downgrading to 0.50.3
After downgrading to version 0.50.3 everything started to work fine again with the following commands

Download the specific version

wget https://github.com/netbirdio/netbird/releases/download/v0.50.3/netbird_0.50.3_linux_amd64.deb

Install the package

sudo dpkg -i netbird_0.50.3_linux_amd64.deb

If there are dependency issues, fix them with:

sudo apt --fix-broken install

Originally created by @gcervantes2005 on GitHub (Jul 21, 2025). **Describe the problem** After upgrading to Netbird v0.51.1 or installing it fresh on Ubuntu, attempts to ping hosts outside local LAN result in this error: From 100.74.218.190 icmp_seq=3 Destination Host Unreachable ping: sendmsg: Required key not available A clear and concise description of what the problem is. **To Reproduce** Install version 0.51.1 on ubuntu on an existing network. Server is running on docker below is the yaml `version: "3.4" services: # Caddy reverse proxy caddy: image: caddy restart: unless-stopped networks: [ netbird ] ports: - '443:443' - '443:443/udp' - '80:80' - '8080:8080' volumes: - netbird_caddy_data:/data - ./Caddyfile:/etc/caddy/Caddyfile logging: driver: "json-file" options: max-size: "500m" max-file: "2" # UI dashboard dashboard: image: netbirdio/dashboard:latest restart: unless-stopped networks: [netbird] env_file: - ./dashboard.env logging: driver: "json-file" options: max-size: "500m" max-file: "2" # Signal signal: image: netbirdio/signal:latest restart: unless-stopped networks: [netbird] logging: driver: "json-file" options: max-size: "500m" max-file: "2" # Relay relay: image: netbirdio/relay:latest restart: unless-stopped networks: [netbird] env_file: - ./relay.env logging: driver: "json-file" options: max-size: "500m" max-file: "2" # Management management: image: netbirdio/management:latest restart: unless-stopped networks: [netbird] volumes: - netbird_management:/var/lib/netbird - ./management.json:/etc/netbird/management.json command: [ "--port", "80", "--log-file", "console", "--log-level", "info", "--disable-anonymous-metrics=false", "--single-account-mode-domain=netbird.selfhosted", "--dns-domain=netbird.selfhosted", "--idp-sign-key-refresh-enabled", ] logging: driver: "json-file" options: max-size: "500m" max-file: "2" # Coturn, AKA relay server coturn: image: coturn/coturn restart: unless-stopped #domainname: netbird.relay.selfhosted volumes: - ./turnserver.conf:/etc/turnserver.conf:ro network_mode: host command: - -c /etc/turnserver.conf logging: driver: "json-file" options: max-size: "500m" max-file: "2" # Zitadel - identity provider zitadel: restart: 'always' networks: [netbird] image: 'ghcr.io/zitadel/zitadel:v2.64.1' command: 'start-from-init --masterkeyFromEnv --tlsMode external' env_file: - ./zitadel.env depends_on: zdb: condition: 'service_healthy' volumes: - ./machinekey:/machinekey - netbird_zitadel_certs:/zdb-certs:ro logging: driver: "json-file" options: max-size: "500m" max-file: "2" # Postgres for Zitadel zdb: restart: 'always' networks: [netbird] image: 'postgres:16-alpine' env_file: - ./zdb.env volumes: - netbird_zdb_data:/var/lib/postgresql/data:rw healthcheck: test: ["CMD-SHELL", "pg_isready", "-d", "db_prod"] interval: 5s timeout: 60s retries: 10 start_period: 5s logging: driver: "json-file" options: max-size: "500m" max-file: "2" volumes: netbird_zdb_data: netbird_management: netbird_caddy_data: netbird_zitadel_certs: networks: netbird:` **I fixed it by downgrading to 0.50.3** After downgrading to version 0.50.3 everything started to work fine again with the following commands # Download the specific version wget https://github.com/netbirdio/netbird/releases/download/v0.50.3/netbird_0.50.3_linux_amd64.deb # Install the package sudo dpkg -i netbird_0.50.3_linux_amd64.deb # If there are dependency issues, fix them with: sudo apt --fix-broken install
saavagebueno added the bugtriage-neededself-hosting labels 2025-11-20 06:12:31 -05:00
Author
Owner

@Gdenich commented on GitHub (Jul 21, 2025):

I can confirm the problem. I have spent half the day with the connection problem, checking everything possible, networking, firewalls between Proxmox nodes (remote and within cluster).
I have installed the new version of clients on two Ubuntu LCXs, and I have the following error when pinging the remote peer:

❯ ping 100.105.134.2 PING 100.105.134.2 (100.105.134.2) 56(84) bytes of data. From 100.105.113.205 icmp_seq=1 Destination Host Unreachable ping: sendmsg: Required key not available From 100.105.113.205 icmp_seq=2 Destination Host Unreachable ping: sendmsg: Required key not available From 100.105.113.205 icmp_seq=3 Destination Host Unreachable ping: sendmsg: Required key not available

❯ netbird status
OS: linux/amd64
Daemon version: 0.51.2
CLI version: 0.51.2
Management: Connected
Signal: Connected
Relays: 2/3 Available
Nameservers: 0/0 Available
FQDN: kopia-backup-agent.netbird.selfhosted
NetBird IP: 100.107.123.135/16
Interface type: Kernel
Quantum resistance: false
Lazy connection: false
Networks: -
Forwarding rules: 0
Peers count: 11/50 Connected

Next to the LXCs with the latest version (that have connection problem), I have an LXC, that has version 0.49.0 and pings the remote LXC perfectly.
What is important, that problematic LXCs with version 0.51.2 can ping the other Netbird clients on the same Proxmox node, on same network interface, but can not ping remote machines.

@Gdenich commented on GitHub (Jul 21, 2025): I can confirm the problem. I have spent half the day with the connection problem, checking everything possible, networking, firewalls between Proxmox nodes (remote and within cluster). I have installed the new version of clients on two Ubuntu LCXs, and I have the following error when pinging the remote peer: `❯ ping 100.105.134.2 PING 100.105.134.2 (100.105.134.2) 56(84) bytes of data. From 100.105.113.205 icmp_seq=1 Destination Host Unreachable ping: sendmsg: Required key not available From 100.105.113.205 icmp_seq=2 Destination Host Unreachable ping: sendmsg: Required key not available From 100.105.113.205 icmp_seq=3 Destination Host Unreachable ping: sendmsg: Required key not available` ❯ netbird status OS: linux/amd64 Daemon version: 0.51.2 CLI version: 0.51.2 Management: Connected Signal: Connected Relays: 2/3 Available Nameservers: 0/0 Available FQDN: kopia-backup-agent.netbird.selfhosted NetBird IP: 100.107.123.135/16 Interface type: Kernel Quantum resistance: false Lazy connection: false Networks: - Forwarding rules: 0 Peers count: 11/50 Connected Next to the LXCs with the latest version (that have connection problem), I have an LXC, that has version 0.49.0 and pings the remote LXC perfectly. What is important, that problematic LXCs with version 0.51.2 can ping the other Netbird clients on the same Proxmox node, on same network interface, but can not ping remote machines.
Author
Owner

@nazarewk commented on GitHub (Jul 22, 2025):

Could you send us a debug bundle while trying to reproduce issues? netbird debug for 2m -SU should be sufficient

@nazarewk commented on GitHub (Jul 22, 2025): Could you send us a debug bundle while trying to reproduce issues? `netbird debug for 2m -SU` should be sufficient
Author
Owner

@nazarewk commented on GitHub (Jul 22, 2025):

You might be hit by https://github.com/netbirdio/netbird/releases/tag/v0.51.0 breaking change requiring update of the Relay server before clients function properly. Can you confirm you're running the latest version also for the Relay?

@nazarewk commented on GitHub (Jul 22, 2025): You might be hit by https://github.com/netbirdio/netbird/releases/tag/v0.51.0 breaking change requiring update of the Relay server before clients function properly. Can you confirm you're running the latest version also for the Relay?
Author
Owner

@Gdenich commented on GitHub (Jul 24, 2025):

You might be hit by https://github.com/netbirdio/netbird/releases/tag/v0.51.0 breaking change requiring update of the Relay server before clients function properly. Can you confirm you're running the latest version also for the Relay?

Hi @nazarewk, I have updated the docker images to the latest version, and now everything is working on 0.51.2 client!
Thank you!

@Gdenich commented on GitHub (Jul 24, 2025): > You might be hit by https://github.com/netbirdio/netbird/releases/tag/v0.51.0 breaking change requiring update of the Relay server before clients function properly. Can you confirm you're running the latest version also for the Relay? Hi @nazarewk, I have updated the docker images to the latest version, and now everything is working on 0.51.2 client! Thank you!
Author
Owner

@legion-gaming commented on GitHub (Aug 16, 2025):

I've updated to the latest version of the docker images as well as the latest version of the client. I used a setup key to provision a linux client (an access point that I'm building), and when I do this:

ping -I wt0 1.1.1.1

I get back: ping: sendmsg: Required key not available

@legion-gaming commented on GitHub (Aug 16, 2025): I've updated to the latest version of the docker images as well as the latest version of the client. I used a setup key to provision a linux client (an access point that I'm building), and when I do this: `ping -I wt0 1.1.1.1` I get back: ping: sendmsg: Required key not available
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#2086