[GH-ISSUE #2290] netbird try (and can't) set /proc/sys/net/ipv4/conf/all/src_valid_mark #4815

Open
opened 2026-08-05 00:59:47 -04:00 by saavagebueno · 23 comments
Owner

Originally created by @lfarkas on GitHub (Jul 19, 2024).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/2290

with the latest netbird and on home assistant with netbird addon i've got this error:

2024-07-19T14:57:37+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:100: Error setting up sysctl: 1 errors occurred:
	* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
2024-07-19T14:57:37+02:00 INFO client/internal/routemanager/manager.go:135: Routing setup complete

after i google it there are same problems with wireguard:
https://forums.docker.com/t/sysctl-error-setting-key-net-ipv4-conf-all-src-valid-mark-read-only-file-system/92567/8
but even though this container has NET_ADMIN and NET_RAW privileges it's still got this error, although the above
many solution eg this one:
https://community.home-assistant.io/t/wireguard-stopped-working/352348
suggest:
Turns out I can create this error at will with this in the config for a peer:
allowed_ips:
- 0.0.0.0/0
but this does not fail:
allowed_ips: []

Originally created by @lfarkas on GitHub (Jul 19, 2024). Original GitHub issue: https://github.com/netbirdio/netbird/issues/2290 with the latest netbird and on home assistant with netbird addon i've got this error: ``` 2024-07-19T14:57:37+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:100: Error setting up sysctl: 1 errors occurred: * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system 2024-07-19T14:57:37+02:00 INFO client/internal/routemanager/manager.go:135: Routing setup complete ``` after i google it there are same problems with wireguard: https://forums.docker.com/t/sysctl-error-setting-key-net-ipv4-conf-all-src-valid-mark-read-only-file-system/92567/8 but even though this container has NET_ADMIN and NET_RAW privileges it's still got this error, although the above many solution eg this one: https://community.home-assistant.io/t/wireguard-stopped-working/352348 suggest: Turns out I can create this error at will with this in the config for a peer: allowed_ips: - 0.0.0.0/0 but this does not fail: allowed_ips: []
saavagebueno added the linux label 2026-08-05 00:59:47 -04:00
Author
Owner

@lfarkas commented on GitHub (Oct 11, 2024):

any news on this?

<!-- gh-comment-id:2407857220 --> @lfarkas commented on GitHub (Oct 11, 2024): any news on this?
Author
Owner

@lixmal commented on GitHub (Oct 25, 2024):

Have you tried setting the option, e.g. with docker

docker run --sysctl net.ipv4.conf.all.src_valid_mark=1 [...]

?

<!-- gh-comment-id:2437173549 --> @lixmal commented on GitHub (Oct 25, 2024): Have you tried setting the option, e.g. with docker ``` docker run --sysctl net.ipv4.conf.all.src_valid_mark=1 [...] ``` ?
Author
Owner

@lfarkas commented on GitHub (Oct 25, 2024):

it's a home assistant addon where the container run by the supervisor...
but has a lot's of privileges:

ost_network: true
host_dbus: true
privileged:
  - SYS_ADMIN
  - SYS_RESOURCE
  - NET_ADMIN
  - NET_RAW
  - BPF
<!-- gh-comment-id:2437536083 --> @lfarkas commented on GitHub (Oct 25, 2024): it's a home assistant addon where the container run by the supervisor... but has a lot's of privileges: ``` ost_network: true host_dbus: true privileged: - SYS_ADMIN - SYS_RESOURCE - NET_ADMIN - NET_RAW - BPF ```
Author
Owner

@corrreia commented on GitHub (Oct 25, 2024):

The container itself doesn't seem to have the perms required, right?

<!-- gh-comment-id:2437549534 --> @corrreia commented on GitHub (Oct 25, 2024): The container itself doesn't seem to have the perms required, right?
Author
Owner

@nazarewk commented on GitHub (Apr 28, 2025):

Hello @lfarkas,

We're currently reviewing our open issues and would like to verify if this problem still exists in the latest NetBird version.

Could you please confirm if the issue is still there?

We may close this issue temporarily if we don't hear back from you within 2 weeks, but feel free to reopen it with updated information.

Thanks for your contribution to improving the project!

<!-- gh-comment-id:2835665274 --> @nazarewk commented on GitHub (Apr 28, 2025): Hello @lfarkas, We're currently reviewing our open issues and would like to verify if this problem still exists in the [latest NetBird version](https://github.com/netbirdio/netbird/releases). Could you please confirm if the issue is still there? We may close this issue temporarily if we don't hear back from you within **2 weeks**, but feel free to reopen it with updated information. Thanks for your contribution to improving the project!
Author
Owner

@lfarkas commented on GitHub (Apr 28, 2025):

still exists v0.43.0:

2025-04-28T20:09:05+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred:
	* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
2025-04-28T20:09:05+02:00 WARN client/firewall/nftables/router_linux.go:956: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-04-28T20:09:05+02:00 WARN client/firewall/nftables/router_linux.go:863: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-04-28T20:09:05+02:00 ERRO client/firewall/nftables/router_linux.go:108: failed to set up data plane mark: flush: conn.Receive: netlink receive: operation not supported
netlink receive: operation not supported
<!-- gh-comment-id:2836081541 --> @lfarkas commented on GitHub (Apr 28, 2025): still exists v0.43.0: ``` 2025-04-28T20:09:05+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred: * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system 2025-04-28T20:09:05+02:00 WARN client/firewall/nftables/router_linux.go:956: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH 2025-04-28T20:09:05+02:00 WARN client/firewall/nftables/router_linux.go:863: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH 2025-04-28T20:09:05+02:00 ERRO client/firewall/nftables/router_linux.go:108: failed to set up data plane mark: flush: conn.Receive: netlink receive: operation not supported netlink receive: operation not supported ```
Author
Owner

@HaleTom commented on GitHub (Jun 30, 2025):

2025-06-30T07:08:02Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred:
        * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
        * write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system
        * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system

And later:

2025-06-30T07:08:04Z WARN client/internal/routemanager/systemops/systemops_linux.go:161: Default route is configured but sysctl operations failed, VPN traffic may not be routed correctly, consider using NB_USE_LEGACY_ROUTING=true or setting net.ipv4.conf.*.rp_filter to 2 (loose) or 0 (off)

Perhaps the docker setup doco needs to be updated here?

Workaround:

peer_name=HOSTNAME
setup_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=SYS_ADMIN --cap-add=SYS_RESOURCE --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --sysctl net.ipv4.conf.eth0.rp_filter=2 --detach netbirdio/netbird:latest

Could the "triage needed" be removed based on this and the previous?

<!-- gh-comment-id:3018108212 --> @HaleTom commented on GitHub (Jun 30, 2025): ``` 2025-06-30T07:08:02Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred: * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system * write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system ``` And later: ``` 2025-06-30T07:08:04Z WARN client/internal/routemanager/systemops/systemops_linux.go:161: Default route is configured but sysctl operations failed, VPN traffic may not be routed correctly, consider using NB_USE_LEGACY_ROUTING=true or setting net.ipv4.conf.*.rp_filter to 2 (loose) or 0 (off) ``` Perhaps the docker setup doco needs to be updated [here](https://docs.netbird.io/how-to/examples#net-bird-client-in-docker)? **Workaround:** ``` peer_name=HOSTNAME setup_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=SYS_ADMIN --cap-add=SYS_RESOURCE --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --sysctl net.ipv4.conf.eth0.rp_filter=2 --detach netbirdio/netbird:latest ``` Could the "triage needed" be removed based on this and the previous?
Author
Owner

@nazarewk commented on GitHub (Jun 30, 2025):

@HaleTom does it work fully with those additional container arguments?

I have never dug much into details, but the general recommendation for those errors was always to configure sysctls on the host system before running the container.

<!-- gh-comment-id:3019140944 --> @nazarewk commented on GitHub (Jun 30, 2025): @HaleTom does it work fully with those additional container arguments? I have never dug much into details, but the general recommendation for those errors was always to configure sysctls on the host system before running the container.
Author
Owner

@HaleTom commented on GitHub (Jul 5, 2025):

Yes - all good with those args.

For the doco:

You may need to add an analogous eth1 if you have two ethernet network interfaces, or a different interface as listed in the ERR messages.

<!-- gh-comment-id:3038420476 --> @HaleTom commented on GitHub (Jul 5, 2025): Yes - all good with those args. For the doco: You may need to add an analogous `eth1` if you have two ethernet network interfaces, or a different interface as listed in the ERR messages.
Author
Owner

@nazarewk commented on GitHub (Jul 10, 2025):

@HaleTom related to https://github.com/netbirdio/netbird/issues/2345 & https://github.com/netbirdio/docs/pull/390

Could I ask you to try --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF (alternatively switch CAP_BPF with CAP_SYS_ADMIN if you're running kernel older than 5.8) in your setup? I found those 3 working just fine on my system with slightly less privileges than suggested.

<!-- gh-comment-id:3059103132 --> @nazarewk commented on GitHub (Jul 10, 2025): @HaleTom related to https://github.com/netbirdio/netbird/issues/2345 & https://github.com/netbirdio/docs/pull/390 Could I ask you to try `--cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF` (alternatively switch `CAP_BPF` with `CAP_SYS_ADMIN` if you're running kernel older than 5.8) in your setup? I found those 3 working just fine on my system with slightly less privileges than suggested.
Author
Owner

@lfarkas commented on GitHub (Jul 11, 2025):

these used and still not working:-(

privileged:

  • SYS_ADMIN
  • SYS_RESOURCE
  • NET_ADMIN
  • NET_RAW
  • BPF
<!-- gh-comment-id:3060819013 --> @lfarkas commented on GitHub (Jul 11, 2025): these used and still not working:-( privileged: - SYS_ADMIN - SYS_RESOURCE - NET_ADMIN - NET_RAW - BPF
Author
Owner

@nazarewk commented on GitHub (Jul 11, 2025):

@lfarkas could you share what exactly isn't working with those capabilities? Could you share the logs/debug bundle?

<!-- gh-comment-id:3061115832 --> @nazarewk commented on GitHub (Jul 11, 2025): @lfarkas could you share what exactly isn't working with those capabilities? Could you share the logs/debug bundle?
Author
Owner

@lfarkas commented on GitHub (Jul 11, 2025):

Add-on version: v0.50.2
 You are running the latest version of this add-on.
 System: Home Assistant OS 16.0  (amd64 / generic-x86-64)
 Home Assistant Core: 2025.7.1
 Home Assistant Supervisor: 2025.07.1
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
Log level is set to WARNING
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service netbird: starting
s6-rc: info: service netbird successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
2025-07-10T22:30:15+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred:
	* write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
2025-07-10T22:30:15+02:00 WARN client/firewall/nftables/router_linux.go:952: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-07-10T22:30:15+02:00 WARN client/firewall/nftables/router_linux.go:859: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH
2025-07-10T22:30:15+02:00 ERRO client/firewall/nftables/router_linux.go:108: failed to set up data plane mark: flush: conn.Receive: netlink receive: operation not supported
netlink receive: operation not supported
2025-07-10T22:30:15+02:00 WARN client/internal/dns/server.go:440: the DNS manager of this peer doesn't support custom port. Disabling primary DNS setup. Learn more at: https://docs.netbird.io/how-to/manage-dns-in-your-network#local-resolver
2025-07-10T22:30:15+02:00 ERRO client/internal/dns/server.go:495: failed to apply DNS host manager update: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured
2025-07-10T22:51:15+02:00 WARN [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/wg_watcher.go:130: WireGuard handshake timed out, closing relay connection: 2025-07-10 22:47:46.409118108 +0200 CEST
2025-07-11T01:58:56+02:00 WARN management/client/grpc.go:171: disconnected from the Management service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T05:11:11+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T05:31:56+02:00 ERRO [peer: gsrpCbJwc8lkmNV783rxIHpyj+zZIhy/rFj5HsfVuBY=] client/internal/peer/conn.go:574: failed to send offer: signal is not ready
2025-07-11T06:07:45+02:00 ERRO [peer: RtObgAe/KslyFa/t0a/iGwy7HohRzO8xhNNUPIR1ri8=] client/internal/peer/wg_watcher.go:53: WireGuard watcher already enabled
2025-07-11T06:18:15+02:00 ERRO [peer: f+tmDAAoOYRUT/WAoJl0PsqalR4zJvt7ljkxZboO9iE=] client/internal/peer/wg_watcher.go:53: WireGuard watcher already enabled
2025-07-11T08:11:24+02:00 WARN [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/wg_watcher.go:130: WireGuard handshake timed out, closing relay connection: 2025-07-11 08:09:42.131427796 +0200 CEST
2025-07-11T11:48:48+02:00 WARN management/client/grpc.go:171: disconnected from the Management service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:27:56+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:28:50+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
2025-07-11T12:28:59+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:32:35+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR
2025-07-11T12:32:45+02:00 WARN signal/client/grpc.go:143: disconnected from the Signal Exchange due to an error: didn't receive a registration header from the Signal server whille connecting to the streams
2025-07-11T12:33:20+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
2025-07-11T12:34:22+02:00 ERRO [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
2025-07-11T12:34:24+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message
 Live
<!-- gh-comment-id:3061982939 --> @lfarkas commented on GitHub (Jul 11, 2025): ``` Add-on version: v0.50.2 You are running the latest version of this add-on. System: Home Assistant OS 16.0 (amd64 / generic-x86-64) Home Assistant Core: 2025.7.1 Home Assistant Supervisor: 2025.07.1 ----------------------------------------------------------- Please, share the above information when looking for help or support in, e.g., GitHub, forums or the Discord chat. ----------------------------------------------------------- s6-rc: info: service base-addon-banner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service base-addon-log-level: starting s6-rc: info: service fix-attrs successfully started Log level is set to WARNING s6-rc: info: service base-addon-log-level successfully started s6-rc: info: service legacy-cont-init: starting s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service netbird: starting s6-rc: info: service netbird successfully started s6-rc: info: service legacy-services: starting s6-rc: info: service legacy-services successfully started 2025-07-10T22:30:15+02:00 ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred: * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system 2025-07-10T22:30:15+02:00 WARN client/firewall/nftables/router_linux.go:952: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH 2025-07-10T22:30:15+02:00 WARN client/firewall/nftables/router_linux.go:859: Will use nftables to manipulate the filter table because iptables is not available: exec: "iptables": executable file not found in $PATH 2025-07-10T22:30:15+02:00 ERRO client/firewall/nftables/router_linux.go:108: failed to set up data plane mark: flush: conn.Receive: netlink receive: operation not supported netlink receive: operation not supported 2025-07-10T22:30:15+02:00 WARN client/internal/dns/server.go:440: the DNS manager of this peer doesn't support custom port. Disabling primary DNS setup. Learn more at: https://docs.netbird.io/how-to/manage-dns-in-your-network#local-resolver 2025-07-10T22:30:15+02:00 ERRO client/internal/dns/server.go:495: failed to apply DNS host manager update: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured 2025-07-10T22:51:15+02:00 WARN [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/wg_watcher.go:130: WireGuard handshake timed out, closing relay connection: 2025-07-10 22:47:46.409118108 +0200 CEST 2025-07-11T01:58:56+02:00 WARN management/client/grpc.go:171: disconnected from the Management service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR 2025-07-11T05:11:11+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR 2025-07-11T05:31:56+02:00 ERRO [peer: gsrpCbJwc8lkmNV783rxIHpyj+zZIhy/rFj5HsfVuBY=] client/internal/peer/conn.go:574: failed to send offer: signal is not ready 2025-07-11T06:07:45+02:00 ERRO [peer: RtObgAe/KslyFa/t0a/iGwy7HohRzO8xhNNUPIR1ri8=] client/internal/peer/wg_watcher.go:53: WireGuard watcher already enabled 2025-07-11T06:18:15+02:00 ERRO [peer: f+tmDAAoOYRUT/WAoJl0PsqalR4zJvt7ljkxZboO9iE=] client/internal/peer/wg_watcher.go:53: WireGuard watcher already enabled 2025-07-11T08:11:24+02:00 WARN [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/wg_watcher.go:130: WireGuard handshake timed out, closing relay connection: 2025-07-11 08:09:42.131427796 +0200 CEST 2025-07-11T11:48:48+02:00 WARN management/client/grpc.go:171: disconnected from the Management service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR 2025-07-11T12:27:56+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR 2025-07-11T12:28:50+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message 2025-07-11T12:28:59+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR 2025-07-11T12:32:35+02:00 WARN signal/client/grpc.go:162: disconnected from the Signal service but will retry silently. Reason: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: INTERNAL_ERROR 2025-07-11T12:32:45+02:00 WARN signal/client/grpc.go:143: disconnected from the Signal Exchange due to an error: didn't receive a registration header from the Signal server whille connecting to the streams 2025-07-11T12:33:20+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message 2025-07-11T12:34:22+02:00 ERRO [peer: vHwiLU2vP4BqhiKwrZ04UIskMKPxIomXz39OqxnSb1Q=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message 2025-07-11T12:34:24+02:00 ERRO [peer: Yg/JDeFsAfMnue9KOTNm77L0AlG1g3Y6pYIm3KhUxyw=] client/internal/peer/conn.go:574: failed to send offer: rpc error: code = Unknown desc = error sending message Live ```
Author
Owner

@nazarewk commented on GitHub (Jul 11, 2025):

@lfarkas I'm not sure how the service is set up, but it looks like you're missing the following arguments mentioned above in the thread in case it is a container

  • --sysctl="net.ipv4.conf.all.src_valid_mark=1"
  • --sysctl="net.ipv4.conf.all.rp_filter=2"
  • --sysctl="net.ipv4.conf.eth0.rp_filter=2" - you might need to use other interface tha eth0 here

I would recommend setting the above on the container host, at least on the interface-specific rule.

This seems more like some kind of Home Assistant OS issue?

PS: I just noticed we have a repo for HA integration and looks like you're the main contributor?

<!-- gh-comment-id:3062153769 --> @nazarewk commented on GitHub (Jul 11, 2025): @lfarkas I'm not sure how the service is set up, but it looks like you're missing the following arguments mentioned above in the thread in case it is a container - `--sysctl="net.ipv4.conf.all.src_valid_mark=1"` - `--sysctl="net.ipv4.conf.all.rp_filter=2"` - `--sysctl="net.ipv4.conf.eth0.rp_filter=2"` - you might need to use other interface tha `eth0` here I would recommend setting the above on the container host, at least on the interface-specific rule. This seems more like some kind of Home Assistant OS issue? PS: I just noticed we have a repo for HA integration and looks like you're the main contributor?
Author
Owner

@lfarkas commented on GitHub (Jul 11, 2025):

yes unfortunately on HAOS it's not possible to set these sysctl i can only set capabilities only these:
https://developers.home-assistant.io/docs/add-ons/configuration/
Privilege for access to hardware/system. Available access: BPF, CHECKPOINT_RESTORE, DAC_READ_SEARCH, IPC_LOCK, NET_ADMIN, NET_RAW, PERFMON, SYS_ADMIN, SYS_MODULE, SYS_NICE, SYS_PTRACE, SYS_RAWIO, SYS_RESOURCE or SYS_TIME.
or host_network or full_access but these 2 wouldn't like to add...

<!-- gh-comment-id:3062366246 --> @lfarkas commented on GitHub (Jul 11, 2025): yes unfortunately on HAOS it's not possible to set these sysctl i can only set capabilities only these: https://developers.home-assistant.io/docs/add-ons/configuration/ Privilege for access to hardware/system. Available access: BPF, CHECKPOINT_RESTORE, DAC_READ_SEARCH, IPC_LOCK, NET_ADMIN, NET_RAW, PERFMON, SYS_ADMIN, SYS_MODULE, SYS_NICE, SYS_PTRACE, SYS_RAWIO, SYS_RESOURCE or SYS_TIME. or host_network or full_access but these 2 wouldn't like to add...
Author
Owner

@nazarewk commented on GitHub (Jul 11, 2025):

yes unfortunately on HAOS it's not possible to set these sysctl

Does HAOS allow setting those on the host-level? If it's already set by the time NetBird container starts up it won't try to modify those settings.

<!-- gh-comment-id:3062470792 --> @nazarewk commented on GitHub (Jul 11, 2025): > yes unfortunately on HAOS it's not possible to set these sysctl Does HAOS allow setting those on the host-level? If it's already set by the time NetBird container starts up it won't try to modify those settings.
Author
Owner

@HaleTom commented on GitHub (Jul 16, 2025):

@HaleTom related to #2345 & netbirdio/docs#390

Could I ask you to try --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF (alternatively switch CAP_BPF with CAP_SYS_ADMIN if you're running kernel older than 5.8) in your setup? I found those 3 working just fine on my system with slightly less privileges than suggested.

2025-07-16T13:40:21Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred:
        * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
        * write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system
        * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system

docker run --restart unless-stopped --name netbird-no-sysctl --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_SYS_ADMIN --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --detach netbirdio/netbird:latest gives:

2025-07-16T13:45:07Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred:
        * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system
        * write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system
        * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system
<!-- gh-comment-id:3078714206 --> @HaleTom commented on GitHub (Jul 16, 2025): > [@HaleTom](https://github.com/HaleTom) related to [#2345](https://github.com/netbirdio/netbird/issues/2345) & [netbirdio/docs#390](https://github.com/netbirdio/docs/pull/390) > > Could I ask you to try `--cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF` (alternatively switch `CAP_BPF` with `CAP_SYS_ADMIN` if you're running kernel older than 5.8) in your setup? I found those 3 working just fine on my system with slightly less privileges than suggested. ``` 2025-07-16T13:40:21Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred: * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system * write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system ``` `docker run --restart unless-stopped --name netbird-no-sysctl --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_SYS_ADMIN --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --detach netbirdio/netbird:latest` gives: ``` 2025-07-16T13:45:07Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 3 errors occurred: * write sysctl net.ipv4.conf.all.src_valid_mark: open /proc/sys/net/ipv4/conf/all/src_valid_mark: read-only file system * write sysctl net.ipv4.conf.all.rp_filter: open /proc/sys/net/ipv4/conf/all/rp_filter: read-only file system * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system ```
Author
Owner

@nazarewk commented on GitHub (Jul 16, 2025):

@HaleTom thanks, please mix it with the addition of --sysctl arguments mentioned few comments up

<!-- gh-comment-id:3078726272 --> @nazarewk commented on GitHub (Jul 16, 2025): @HaleTom thanks, please mix it with the addition of [`--sysctl` arguments mentioned few comments up](https://github.com/netbirdio/netbird/issues/2290#issuecomment-3062153769)
Author
Owner

@HaleTom commented on GitHub (Jul 16, 2025):

@nazarewk I'm revisiting this and I'm afraid I gave you a bum steer with my command before. The command with eth0 doesn't work (see log entry below). Sorry - this means that the current documentation produces a docker error (also listed below).

TL;DR:

  • Remove all references to eth* from docker command line (see errors further below if this is there)
  • Create a docker network and apply the sysctls to the interface attached to that network
# Create network to apply com.docker.network.endpoint.sysctls:
docker network create netbird-net

docker run \
  --restart unless-stopped \
  --name netbird \
  --hostname "$peer_name" \
  --env NB_SETUP_KEY="$setup_key" \
  --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF \
  --volume netbird-client:/etc/netbird \
  --sysctl net.ipv4.conf.all.src_valid_mark=1 \
  --sysctl net.ipv4.conf.all.rp_filter=2 \
  --network=name=netbird-net,"driver-opt=com.docker.network.endpoint.sysctls=net.ipv4.conf.IFNAME.rp_filter=2"\
  --detach netbirdio/netbird:latest

Above, docker will replace IFNAME with the interface that it creates.


As I (incorrectly) said to do it before:

docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --sysctl net.ipv4.conf.eth0.rp_filter=2 --detach netbirdio/netbird:latest

This produces:

docker: Error response from daemon: interface specific sysctl setting "net.ipv4.conf.eth0.rp_filter" must be supplied using driver option 'com.docker.network.endpoint.sysctls'

Dropping eth0 from the command line prevents docker complaining immediately:

docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --detach netbirdio/netbird:latest

But I still get:

2025-07-16T13:55:03Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred:
        * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system

AFAICT from this issue, docker always mounts /proc/sys RO, so even with --cap-add=ALL, it's not possible to have the container change these values unless docker is run --privileged (which is generally considered bad form).

<!-- gh-comment-id:3078911271 --> @HaleTom commented on GitHub (Jul 16, 2025): @nazarewk I'm revisiting this and I'm afraid I gave you a bum steer with my command before. The command with `eth0` doesn't work (see log entry below). Sorry - this means that the current documentation produces a docker error (also listed below). TL;DR: - Remove all references to `eth*` from docker command line (see errors further below if this is there) - Create a docker network and apply the sysctls to the interface attached to that network ``` # Create network to apply com.docker.network.endpoint.sysctls: docker network create netbird-net docker run \ --restart unless-stopped \ --name netbird \ --hostname "$peer_name" \ --env NB_SETUP_KEY="$setup_key" \ --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF \ --volume netbird-client:/etc/netbird \ --sysctl net.ipv4.conf.all.src_valid_mark=1 \ --sysctl net.ipv4.conf.all.rp_filter=2 \ --network=name=netbird-net,"driver-opt=com.docker.network.endpoint.sysctls=net.ipv4.conf.IFNAME.rp_filter=2"\ --detach netbirdio/netbird:latest ``` Above, docker will replace `IFNAME` with the interface that it creates. ---- As I (incorrectly) said to do it before: ``` docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --sysctl net.ipv4.conf.eth0.rp_filter=2 --detach netbirdio/netbird:latest ``` This produces: ``` docker: Error response from daemon: interface specific sysctl setting "net.ipv4.conf.eth0.rp_filter" must be supplied using driver option 'com.docker.network.endpoint.sysctls' ``` Dropping `eth0` from the command line prevents docker complaining immediately: ``` docker run --restart unless-stopped --name netbird --hostname "$peer_name" --cap-add=NET_ADMIN --cap-add=NET_RAW --cap-add=CAP_BPF --env NB_SETUP_KEY="$setup_key" --volume netbird-client:/etc/netbird --sysctl net.ipv4.conf.all.src_valid_mark=1 --sysctl net.ipv4.conf.all.rp_filter=2 --detach netbirdio/netbird:latest ``` But I still get: ``` 2025-07-16T13:55:03Z ERRO client/internal/routemanager/systemops/systemops_linux.go:102: Error setting up sysctl: 1 error occurred: * write sysctl net.ipv4.conf.eth0.rp_filter: open /proc/sys/net/ipv4/conf/eth0/rp_filter: read-only file system ``` [AFAICT from this issue](https://github.com/docker/for-linux/issues/197), docker always mounts /proc/sys RO, so even with `--cap-add=ALL`, it's not possible to have the container change these values unless docker is run `--privileged` (which is generally considered bad form).
Author
Owner

@nazarewk commented on GitHub (Jul 16, 2025):

@HaleTom, thanks for the more extensive information. I'll try to take a look at it later on.

<!-- gh-comment-id:3078989864 --> @nazarewk commented on GitHub (Jul 16, 2025): @HaleTom, thanks for the more extensive information. I'll try to take a look at it later on.
Author
Owner

@Blacks-Army commented on GitHub (Oct 23, 2025):

I have the same error running with docker in userns-remap mode

<!-- gh-comment-id:3438782566 --> @Blacks-Army commented on GitHub (Oct 23, 2025): I have the same error running with docker in userns-remap mode
Author
Owner

@uGiFarukh commented on GitHub (Mar 5, 2026):

This issue still exists on latest netbird 0.66.2 docker container on arm64

<!-- gh-comment-id:4006364106 --> @uGiFarukh commented on GitHub (Mar 5, 2026): This issue still exists on latest netbird 0.66.2 docker container on arm64
Author
Owner

@stefan-fast commented on GitHub (Jul 18, 2026):

Still relevant in the newest version v0.74.7 unfortunately.

<!-- gh-comment-id:5011095932 --> @stefan-fast commented on GitHub (Jul 18, 2026): Still relevant in the newest version `v0.74.7` unfortunately.
Sign in to join this conversation.
No Label linux
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#4815