SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-06-10 01:12:51 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

NetBird systemd service fails after package update due to missing log directory #1913

New Issue
Open
opened 2025-11-20 06:09:05 -05:00 by saavagebueno · 7 comments
saavagebueno commented 2025-11-20 06:09:05 -05:00
Owner
Copy Link

Originally created by @kimasplund on GitHub (May 23, 2025).

NetBird systemd service fails after package update due to missing log directory

Description

After updating NetBird through the package manager, the systemd service fails to start with exit code 209/STDOUT. The root cause is that the uninstaller removes the /var/log/netbird/ directory during package updates, but the installer/updater does not recreate it.

Environment

  • OS: Ubuntu 25.04 (Linux 6.14.6-061406-generic)
  • NetBird version: 0.44.0
  • Installation method: Package manager (systemd service)

Steps to Reproduce

  1. Have NetBird installed and running
  2. Update NetBird package (e.g., via apt upgrade)
  3. Check service status: sudo systemctl status netbird
  4. Service fails with exit code 209/STDOUT

Expected Behavior

The NetBird service should start successfully after package updates without manual intervention.

Actual Behavior

The service fails to start because the log directory /var/log/netbird/ is missing.

Error Logs

May 23 09:16:50 Kim-UB25-TP systemd[1]: Started netbird.service - A WireGuard-based mesh network that connects your devices into a single private network..
May 23 09:16:50 Kim-UB25-TP (netbird)[28159]: netbird.service: Failed to set up standard output: No such file or directory
May 23 09:16:50 Kim-UB25-TP (netbird)[28159]: netbird.service: Failed at step STDOUT spawning /usr/bin/netbird: No such file or directory
May 23 09:16:50 Kim-UB25-TP systemd[1]: netbird.service: Main process exited, code=exited, status=209/STDOUT
May 23 09:16:50 Kim-UB25-TP systemd[1]: netbird.service: Failed with result 'exit-code'.

Workaround

Manually create the log directory after updates:

sudo mkdir -p /var/log/netbird
sudo chown root:root /var/log/netbird
sudo chmod 755 /var/log/netbird
sudo systemctl restart netbird

Proposed Solution

The package post-install script should ensure the log directory exists:

  1. Check if /var/log/netbird/ exists
  2. If not, create it with appropriate permissions
  3. This should be done in both the installer and updater scripts

Additional Context

The systemd service configuration expects to write logs to /var/log/netbird/client.log as specified in the ExecStart command:

ExecStart=/usr/bin/netbird service run --config /etc/netbird/config.json --log-level info --daemon-addr unix:///var/run/netbird.sock --log-file /var/log/netbird/client.log

This appears to be a packaging issue where the uninstall process (during updates) removes the log directory but the install process doesn't recreate it.

Originally created by @kimasplund on GitHub (May 23, 2025). # NetBird systemd service fails after package update due to missing log directory ## Description After updating NetBird through the package manager, the systemd service fails to start with exit code 209/STDOUT. The root cause is that the uninstaller removes the `/var/log/netbird/` directory during package updates, but the installer/updater does not recreate it. ## Environment - **OS**: Ubuntu 25.04 (Linux 6.14.6-061406-generic) - **NetBird version**: 0.44.0 - **Installation method**: Package manager (systemd service) ## Steps to Reproduce 2. Have NetBird installed and running 2. Update NetBird package (e.g., via apt upgrade) 3. Check service status: `sudo systemctl status netbird` 4. Service fails with exit code 209/STDOUT ## Expected Behavior The NetBird service should start successfully after package updates without manual intervention. ## Actual Behavior The service fails to start because the log directory `/var/log/netbird/` is missing. ## Error Logs ``` May 23 09:16:50 Kim-UB25-TP systemd[1]: Started netbird.service - A WireGuard-based mesh network that connects your devices into a single private network.. May 23 09:16:50 Kim-UB25-TP (netbird)[28159]: netbird.service: Failed to set up standard output: No such file or directory May 23 09:16:50 Kim-UB25-TP (netbird)[28159]: netbird.service: Failed at step STDOUT spawning /usr/bin/netbird: No such file or directory May 23 09:16:50 Kim-UB25-TP systemd[1]: netbird.service: Main process exited, code=exited, status=209/STDOUT May 23 09:16:50 Kim-UB25-TP systemd[1]: netbird.service: Failed with result 'exit-code'. ``` ## Workaround Manually create the log directory after updates: ```bash sudo mkdir -p /var/log/netbird sudo chown root:root /var/log/netbird sudo chmod 755 /var/log/netbird sudo systemctl restart netbird ``` ## Proposed Solution The package post-install script should ensure the log directory exists: 1. Check if `/var/log/netbird/` exists 2. If not, create it with appropriate permissions 3. This should be done in both the installer and updater scripts ## Additional Context The systemd service configuration expects to write logs to `/var/log/netbird/client.log` as specified in the ExecStart command: ``` ExecStart=/usr/bin/netbird service run --config /etc/netbird/config.json --log-level info --daemon-addr unix:///var/run/netbird.sock --log-file /var/log/netbird/client.log ``` This appears to be a packaging issue where the uninstall process (during updates) removes the log directory but the install process doesn't recreate it.
saavagebueno added the triage-needed label 2025-11-20 06:09:05 -05:00
saavagebueno commented 2025-11-20 06:09:06 -05:00
Author
Owner
Copy Link

@kimasplund commented on GitHub (Jun 10, 2025):

how is this still a thing guys?! is it just me his thing affects? i need to recreate the directory on every update...

@kimasplund commented on GitHub (Jun 10, 2025): how is this still a thing guys?! is it just me his thing affects? i need to recreate the directory on every update...
saavagebueno commented 2025-11-20 06:09:07 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jun 10, 2025):

@kimasplund, it is not a common issue. There might be something specific about how the system is set.

Can you share more details on your OS? Is there a mounted log directory? Is app Armor or SELinux enabled? any custom systemd setup?

@mlsmaycon commented on GitHub (Jun 10, 2025): @kimasplund, it is not a common issue. There might be something specific about how the system is set. Can you share more details on your OS? Is there a mounted log directory? Is app Armor or SELinux enabled? any custom systemd setup?
saavagebueno commented 2025-11-20 06:09:07 -05:00
Author
Owner
Copy Link

@kimasplund commented on GitHub (Jun 10, 2025):

nope. nothing special. just default Ubuntu 25.04 on my laptop.

System Details Report

Report details

  • Date generated: 2025-06-10 16:35:08

Hardware Information:

  • Hardware Model: Lenovo ThinkPad P16 Gen 2
  • Memory: 64.0 GiB
  • Processor: 13th Gen Intel® Core™ i9-13980HX × 32
  • Graphics: Intel® Graphics (RPL-S)
  • Graphics 1: NVIDIA RTX 3500 Ada Generation Laptop GPU
  • Disk Capacity: 4.0 TB

Software Information:

  • Firmware Version: N3TET57W (1.57 )
  • OS Name: Ubuntu 25.04
  • OS Build: (null)
  • OS Type: 64-bit
  • GNOME Version: 48
  • Windowing System: Wayland
  • Kernel Version: Linux 6.14.8-061408-generic

tmpfs on /var/log type tmpfs (rw,noatime,size=65536k,mode=755,inode64)

@kimasplund commented on GitHub (Jun 10, 2025): nope. nothing special. just default Ubuntu 25.04 on my laptop. # System Details Report --- ## Report details - **Date generated:** 2025-06-10 16:35:08 ## Hardware Information: - **Hardware Model:** Lenovo ThinkPad P16 Gen 2 - **Memory:** 64.0 GiB - **Processor:** 13th Gen Intel® Core™ i9-13980HX × 32 - **Graphics:** Intel® Graphics (RPL-S) - **Graphics 1:** NVIDIA RTX 3500 Ada Generation Laptop GPU - **Disk Capacity:** 4.0 TB ## Software Information: - **Firmware Version:** N3TET57W (1.57 ) - **OS Name:** Ubuntu 25.04 - **OS Build:** (null) - **OS Type:** 64-bit - **GNOME Version:** 48 - **Windowing System:** Wayland - **Kernel Version:** Linux 6.14.8-061408-generic tmpfs on /var/log type tmpfs (rw,noatime,size=65536k,mode=755,inode64)
saavagebueno commented 2025-11-20 06:09:07 -05:00
Author
Owner
Copy Link

@kimasplund commented on GitHub (Jun 11, 2025):

but sure is annoying as... to need to constantly recreate the log folder. come on guys. just add the function to check it on upgrade. even provided you with the patch for it ;D

@kimasplund commented on GitHub (Jun 11, 2025): but sure is annoying as... to need to constantly recreate the log folder. come on guys. just add the function to check it on upgrade. even provided you with the patch for it ;D
saavagebueno commented 2025-11-20 06:09:07 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jun 11, 2025):

we are running a few tests on your PR and will approve it when they are done

@mlsmaycon commented on GitHub (Jun 11, 2025): we are running a few tests on your PR and will approve it when they are done
saavagebueno commented 2025-11-20 06:09:07 -05:00
Author
Owner
Copy Link

@webash commented on GitHub (Oct 26, 2025):

I've also just experienced this. I think the difference on my system is that I'm running on OpenMediaVault on a Raspberry Pi that has a SD card as its OS storage, so OMV by default uses a flashmemory-aware plugin that keeps noisy writes from going straight to the SD card in a memory FS temporary, flushing it on reboot. I wonder if that's part of the problem in this particular case?

@webash commented on GitHub (Oct 26, 2025): I've also just experienced this. I think the difference on my system is that I'm running on OpenMediaVault on a Raspberry Pi that has a SD card as its OS storage, so OMV by default uses a flashmemory-aware plugin that keeps noisy writes from going straight to the SD card in a memory FS temporary, flushing it on reboot. I wonder if that's part of the problem in this particular case?
saavagebueno commented 2025-11-20 06:09:07 -05:00
Author
Owner
Copy Link

@reyemxela commented on GitHub (Nov 7, 2025):

Just wanted to add my experience to this issue. I'm trying to add Netbird to my custom atomic image based on Bazzite. The issue with installing it on an image-based OS is that during install/build time, the eventual /var of the future running system doesn't exist. Only attempting to create the log folder during install is causing the problems in this case.

The main issue boils down to the use of StandardOutput and StandardError in the systemd service file. Those lines require the target folder to exist, and the whole service dies if it doesn't. If those are taken out, the log folder gets created correctly when the ExecStart command runs.

To be honest, I'm not totally sure the reasoning behind using those directives at all. Systemd/Journald handle stdout/stderr quite nicely already, so it seems odd to me to force the outputs to those log files unless absolutely necessary.

@reyemxela commented on GitHub (Nov 7, 2025): Just wanted to add my experience to this issue. I'm trying to add Netbird to my custom atomic image based on [Bazzite](https://github.com/ublue-os/bazzite). The issue with installing it on an image-based OS is that during install/build time, the eventual `/var` of the future running system doesn't exist. Only attempting to create the log folder during install is causing the problems in this case. The main issue boils down to the use of `StandardOutput` and `StandardError` in the systemd service file. Those lines require the target folder to exist, and the whole service dies if it doesn't. If those are taken out, the log folder gets created correctly when the ExecStart command runs. To be honest, I'm not totally sure the reasoning behind using those directives at all. Systemd/Journald handle stdout/stderr quite nicely already, so it seems odd to me to force the outputs to those log files unless absolutely necessary.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label triage-needed
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#1913
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?