[GH-ISSUE #1505] Support for a FreeBSD as a client OS #2561

New Issue

Originally created by @yaroslav-gwit on GitHub (Jan 29, 2024).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/1505

Is your feature request related to a problem? Please describe.
First of all - thank you for this beautiful project and all your hard work! I wanted to introduce NetBird as a way to allow remote users access our FreeBSD virtualisation nodes.

FreeBSD bhyve VMs and FreeBSD Jails are using the private bridges (bridges that are not connected to any external interface) as internal networks and then the host node uses PF to orchestrate all the network traffic (NAT, VM-to-VM access, etc).

It's an SDN-like solution, which leverages site-to-site WG links for host-to-host communications (and all the internal route management), but it's very tedious to use the "naked" WG for every remote user (or client device) that wants to access our cluster(s).

We are using ZeroTier as of now to onboard the remote users/client devices, but I am not happy with their business licencing terms (I am a paid customer, using their Pro plan), as they are trying to convert all the Pro users to the Custom Enterprise Plan - Link to the Reddit discussion for those interested. I am also not happy with how ZT works under FreeBSD, but that's a whole another story on it's own.

Describe the solution you'd like
The NetBird installation script didn't work for the FreeBSD (nor that I expected it to, because the OS is not on the "supported" list, but it doesn't hurt to try), and I could not find any build instructions to compile the client manually (could be just me skimming thought the repo and missing the build/make script).

Ideally, I'd like to see the NetBird deployment script include an automatic deployment options for FreeBSD, but it will likely take a long time to implement. For now I tried to compile the client like so (but the build has failed):

git clone ${NetBird_Repo}
cd netbird/client
go121 build

Here is output:

# github.com/netbirdio/netbird/client/ssh
ssh/server.go:193:5: undefined: setWinSize
# github.com/netbirdio/netbird/client/internal/dns
internal/dns/server.go:149:25: s.initialize undefined (type *DefaultServer has no field or method initialize, but does have Initialize)

Describe alternatives you've considered
As an alternative, I tried using a Linux VM running under the FreeBSD to deploy the service and distribute the routes. It works well, but far from ideal - it's harder to automate PF around this process, and we lose the autostart if the VM is running on top of the encrypted ZFS dataset (it becomes a manual process to unlock the dataset first, and then start the VM).

Additional context
I am happy to provide any additional context on request.

Originally created by @yaroslav-gwit on GitHub (Jan 29, 2024). Original GitHub issue: https://github.com/netbirdio/netbird/issues/1505 **Is your feature request related to a problem? Please describe.** First of all - thank you for this beautiful project and all your hard work! I wanted to introduce `NetBird` as a way to allow remote users access our FreeBSD virtualisation nodes. FreeBSD `bhyve` VMs and FreeBSD `Jails` are using the private bridges (bridges that are not connected to any external interface) as internal networks and then the host node uses PF to orchestrate all the network traffic (NAT, VM-to-VM access, etc). It's an SDN-like solution, which leverages site-to-site WG links for host-to-host communications (and all the internal route management), but it's very tedious to use the "naked" WG for every remote user (or client device) that wants to access our cluster(s). We are using ZeroTier as of now to onboard the remote users/client devices, but I am not happy with their business licencing terms (I am a paid customer, using their `Pro` plan), as they are trying to convert all the `Pro` users to the `Custom Enterprise Plan` - [Link to the Reddit discussion for those interested](https://www.reddit.com/r/zerotier/comments/1abipi0/zerotier_and_pricing_changes/). I am also not happy with how ZT works under FreeBSD, but that's a whole another story on it's own. **Describe the solution you'd like** The `NetBird` installation script didn't work for the FreeBSD (nor that I expected it to, because the OS is not on the "supported" list, but it doesn't hurt to try), and I could not find any build instructions to compile the client manually (could be just me skimming thought the repo and missing the build/make script). Ideally, I'd like to see the `NetBird` deployment script include an automatic deployment options for FreeBSD, but it will likely take a long time to implement. For now I tried to compile the client like so (but the build has failed): ```shell git clone ${NetBird_Repo} cd netbird/client go121 build ``` Here is output: ```shell # github.com/netbirdio/netbird/client/ssh ssh/server.go:193:5: undefined: setWinSize # github.com/netbirdio/netbird/client/internal/dns internal/dns/server.go:149:25: s.initialize undefined (type *DefaultServer has no field or method initialize, but does have Initialize) ``` **Describe alternatives you've considered** As an alternative, I tried using a Linux VM running under the FreeBSD to deploy the service and distribute the routes. It works well, but far from ideal - it's harder to automate PF around this process, and we lose the autostart if the VM is running on top of the encrypted ZFS dataset (it becomes a manual process to unlock the dataset first, and then start the VM). **Additional context** I am happy to provide any additional context on request.

saavagebueno added the feature-request label 2026-08-05 00:47:49 -04:00

saavagebueno commented

@skillcoder commented on GitHub (Feb 24, 2024):

Seems setWinSize can be ignored on headless freebsd with a new file ssh/window_freebsd.go with following content

//go:build freebsd

package ssh

import (
    "os"
)

func setWinSize(file *os.File, width, height int) {
}

But regarding network implementation need to research.
I will try to reuse already implemented approach but with freebsd build tag
After re-implement dns management need to implement these for freebsd

internal/config.go:33:48: undefined: iface.WgInterfaceDefault
internal/config.go:188:25: undefined: iface.WgInterfaceDefault
internal/engine.go:1164:15: undefined: iface.NewWGIFace
internal/engine.go:1170:23: e.wgInterface.CreateOnAndroid undefined (type *iface.WGIface has no field or method CreateOnAndroid)

and much more in netbird/iface
and

cmd/ssh.go:55:12: undefined: util.IsAdmin

@skillcoder commented on GitHub (Feb 24, 2024): Seems setWinSize can be ignored on headless freebsd with a new file `ssh/window_freebsd.go` with following content ```go //go:build freebsd package ssh import ( "os" ) func setWinSize(file *os.File, width, height int) { } ``` But regarding network implementation need to research. I will try to reuse already implemented approach but with `freebsd` build tag After re-implement dns management need to implement these for `freebsd` ``` internal/config.go:33:48: undefined: iface.WgInterfaceDefault internal/config.go:188:25: undefined: iface.WgInterfaceDefault internal/engine.go:1164:15: undefined: iface.NewWGIFace internal/engine.go:1170:23: e.wgInterface.CreateOnAndroid undefined (type *iface.WGIface has no field or method CreateOnAndroid) ``` and much more in `netbird/iface` and ``` cmd/ssh.go:55:12: undefined: util.IsAdmin ```

saavagebueno commented

@skillcoder commented on GitHub (Feb 24, 2024):

I'm able to compile client, but I'm afraid current implementation know nothing about proper rc.d script and etc location in freebsd.

# ./netbird login
Error: failed to connect to daemon error: context deadline exceeded
If the daemon is not running please run: 
netbird service install 
netbird service start

@yaroslav-gwit Could you please share your rc script and how you setup settings for initial run, for example NB_SETUP_KEY
I will try to proper implement netbird up command
As I understand need to create rc script in /usr/local/etc/rc.d/ and generate config.json in /usr/local/etc/netbird/
Probably also we will have problems with tun interface and wg kernel module, let's assume we install kernel module manually via sudo pkg install wireguard

@skillcoder commented on GitHub (Feb 24, 2024): I'm able to compile client, but I'm afraid current implementation know nothing about proper rc.d script and etc location in freebsd. ``` # ./netbird login Error: failed to connect to daemon error: context deadline exceeded If the daemon is not running please run: netbird service install netbird service start ``` @yaroslav-gwit Could you please share your rc script and how you setup settings for initial run, for example `NB_SETUP_KEY` I will try to proper implement `netbird up` command As I understand need to create rc script in `/usr/local/etc/rc.d/` and generate `config.json` in `/usr/local/etc/netbird/` Probably also we will have problems with `tun` interface and `wg` kernel module, let's assume we install kernel module manually via `sudo pkg install wireguard`

saavagebueno commented

@skillcoder commented on GitHub (Feb 24, 2024):

I'm able to successfully registered in management server with following command

./netbird -c ./config.json up -F

2024-02-24T19:16:58+03:00 INFO client/internal/login.go:130: peer has been successfully registered on Management Service

but can't connect

2024-02-24T19:16:58+03:00 INFO client/internal/connect.go:96: starting NetBird client version development
2024-02-24T19:16:59+03:00 ERRO client/internal/engine.go:267: failed creating tunnel interface wt0: [not implemented]
2024-02-24T19:16:59+03:00 ERRO client/internal/connect.go:235: error while starting Netbird Connection Engine: not implemented
2024-02-24T19:17:01+03:00 ERRO client/internal/engine.go:267: failed creating tunnel interface wt0: [not implemented]
2024-02-24T19:17:01+03:00 ERRO client/internal/connect.go:235: error while starting Netbird Connection Engine: not implemented

@skillcoder commented on GitHub (Feb 24, 2024): I'm able to successfully registered in management server with following command ```shell ./netbird -c ./config.json up -F ``` ``` 2024-02-24T19:16:58+03:00 INFO client/internal/login.go:130: peer has been successfully registered on Management Service ``` but can't connect ``` 2024-02-24T19:16:58+03:00 INFO client/internal/connect.go:96: starting NetBird client version development 2024-02-24T19:16:59+03:00 ERRO client/internal/engine.go:267: failed creating tunnel interface wt0: [not implemented] 2024-02-24T19:16:59+03:00 ERRO client/internal/connect.go:235: error while starting Netbird Connection Engine: not implemented 2024-02-24T19:17:01+03:00 ERRO client/internal/engine.go:267: failed creating tunnel interface wt0: [not implemented] 2024-02-24T19:17:01+03:00 ERRO client/internal/connect.go:235: error while starting Netbird Connection Engine: not implemented ```

saavagebueno commented

@yaroslav-gwit commented on GitHub (Feb 25, 2024):

Great news, thanks. I'll give it a go on my side, too.

Which WG implementation are you guys using under the hood? Did you integrate wireguard-go, or do you rely on the kernel code to provide WG support?

Both options work fine under FreeBSD, and WireGuard is even integrated into a default kernel (since 13.1, I think). To create a new WG interface, it's enough to execute:

ifconfig wg create

Example output:

wg0

Then you could simply do this to turn it into wt interface:

ifconfig wg0 name wt0

@yaroslav-gwit commented on GitHub (Feb 25, 2024): Great news, thanks. I'll give it a go on my side, too. Which WG implementation are you guys using under the hood? Did you integrate `wireguard-go`, or do you rely on the kernel code to provide WG support? Both options work fine under FreeBSD, and WireGuard is even integrated into a default kernel (since 13.1, I think). To create a new WG interface, it's enough to execute: ```shell ifconfig wg create ``` Example output: ```shell wg0 ``` Then you could simply do this to turn it into `wt` interface: ```shell ifconfig wg0 name wt0 ```

saavagebueno commented

@skillcoder commented on GitHub (Mar 8, 2024):

Thank you, @yaroslav-gwit. I will try to implement this approach. If you have more ideas on how it should work, please share them. Additionally, if you have a good rc script for Netbird, it would also be valuable to create a FreeBSD port to simplify Netbird installation on FreeBSD.

Which WG implementation are you guys using under the hood? Did you integrate wireguard-go, or do you rely on the kernel code to provide WG support?

I'm currently know nothing about netbird, just start using it.
@braginini Can anyone from the core team answer these questions?
My current approach is to use wireguard package capabilities, just to make it work and later implement it in a most freebsd idiomatic way, I suppose ifconfig wg is best way. @yaroslav-gwit Please correct me if I'm wrong.

@skillcoder commented on GitHub (Mar 8, 2024): Thank you, @yaroslav-gwit. I will try to implement this approach. If you have more ideas on how it should work, please share them. Additionally, if you have a good `rc` script for Netbird, it would also be valuable to create a FreeBSD port to simplify Netbird installation on FreeBSD. > Which WG implementation are you guys using under the hood? Did you integrate wireguard-go, or do you rely on the kernel code to provide WG support? I'm currently know nothing about netbird, just start using it. @braginini Can anyone from the core team answer these questions? My current approach is to use `wireguard` package capabilities, just to make it work and later implement it in a most freebsd idiomatic way, I suppose `ifconfig wg` is best way. @yaroslav-gwit Please correct me if I'm wrong.

saavagebueno commented

@skillcoder commented on GitHub (Mar 8, 2024):

Meanwhile, I'm able to connect to the service on private node through netbird by using netstack mode

# on freebsd 13.0
export NB_USE_NETSTACK_MODE=true
export NB_SOCKS5_LISTENER_PORT=30000
./client -c ~/.netbird-config.json up -F
curl --socks5-hostname localhost:30000 100.12X.XXX.XXX:33000

@skillcoder commented on GitHub (Mar 8, 2024): Meanwhile, I'm able to connect to the service on private node through netbird by using [netstack mode](https://docs.netbird.io/how-to/netbird-on-faas#how-to-enable-netstack-mode) ```shell # on freebsd 13.0 export NB_USE_NETSTACK_MODE=true export NB_SOCKS5_LISTENER_PORT=30000 ./client -c ~/.netbird-config.json up -F curl --socks5-hostname localhost:30000 100.12X.XXX.XXX:33000 ```

saavagebueno commented

@skillcoder commented on GitHub (Mar 8, 2024):

The current problem is github.com/vishvananda/netlink not supported FreeBSD.
The [not implemented] from this package.
So need to make a PR to the netlink to add support for freebsd kernel networks interfaces, or for now create own wrapper for cli commands like ifconfig wg create

@skillcoder commented on GitHub (Mar 8, 2024): The current problem is `github.com/vishvananda/netlink` not supported FreeBSD. The `[not implemented]` from this package. So need to make a PR to the `netlink` to add support for freebsd kernel networks interfaces, or for now create own wrapper for cli commands like `ifconfig wg create`

saavagebueno commented

@skillcoder commented on GitHub (Mar 8, 2024):

netlink protocol supported in FreeBSD since 13.2
But github.com/vishvananda/netlink looks very dead.
Fortunately seems currently golang is implementing netlink support for freebsd in https://pkg.go.dev/golang.org/x/sys/unix
https://groups.google.com/g/golang-codereviews/c/0YRr1-YEatI?pli=1

@skillcoder commented on GitHub (Mar 8, 2024): `netlink` protocol supported in FreeBSD since [13.2](https://www.freebsd.org/releases/13.2R/relnotes/) But `github.com/vishvananda/netlink` looks very dead. Fortunately seems currently golang is implementing netlink support for freebsd in https://pkg.go.dev/golang.org/x/sys/unix https://groups.google.com/g/golang-codereviews/c/0YRr1-YEatI?pli=1

saavagebueno commented

@yaroslav-gwit commented on GitHub (Mar 8, 2024):

Re rc script:
We could "borrow" some ideas from Nebula and Tailscale services. They both work really well under FreeBSD. I can look them up tomorrow at some point and post the results here.

Re netlink:
Shell wrappers could be a good starting point before we have a proper kernel interface implementation. FreeBSD is not Linux, and ifconfig isn't going anywhere any time soon.

@yaroslav-gwit commented on GitHub (Mar 8, 2024): Re `rc` script: We could "borrow" some ideas from Nebula and Tailscale services. They both work really well under FreeBSD. I can look them up tomorrow at some point and post the results here. Re `netlink`: Shell wrappers could be a good starting point before we have a proper kernel interface implementation. FreeBSD is not Linux, and `ifconfig` isn't going anywhere any time soon.

saavagebueno commented

@yaroslav-gwit commented on GitHub (Mar 28, 2024):

Sorry for the delay, here are the rc files (I will split it into 2 messages, so it's easier to read):

TailScale:

#!/bin/sh

# PROVIDE: tailscaled
# REQUIRE: NETWORKING
# KEYWORD: shutdown
#
# Add the following lines to /etc/rc.conf.local or /etc/rc.conf
# to enable this service:
#
# tailscaled_enable (bool):     Set it to YES to enable tailscaled.
#                               Default is "NO".
# tailscaled_state_dir (str):   Set the path to use for the state directory.
#                               Default is "/var/db/tailscale"
# tailscaled_port (number):     Set the port to listen on for incoming VPN packets.
#                               Default is "41641".
# tailscaled_syslog_output_enable (bool):       Set to enable syslog output.
#                                               Default is "NO". See daemon(8).
# tailscaled_syslog_output_priority (str):      Set syslog priority if syslog enabled.
#                                               Default is "info". See daemon(8).
# tailscaled_syslog_output_facility (str):      Set syslog facility if syslog enabled.
#                                               Default is "daemon". See daemon(8).
# tailscaled_exitnode_enable (bool):    Set it to YES to announce tailscaled as
#                                       an exit node. Default is "NO".
# tailscaled_up_args (str):             Additional arguments to pass to tailscale up
#                                       Default is "" (empty string).
# tailscaled_tun_dev (str):     Set the name of the tun interface tailscaled creates.
#                               Default is "tailscale0"

. /etc/rc.subr

name=tailscaled
rcvar=tailscaled_enable

load_rc_config $name

: ${tailscaled_enable:="NO"}
: ${tailscaled_state_dir:="/var/db/tailscale"}
: ${tailscaled_port:="41641"}
: ${tailscaled_exitnode_enable:="NO"}
: ${tailscaled_up_args:=""}
: ${tailscaled_tun_dev:="tailscale0"}

DAEMON=$(/usr/sbin/daemon 2>&1 | grep -q syslog ; echo $?)
if [ ${DAEMON} -eq 0 ]; then
        : ${tailscaled_syslog_output_enable:="NO"}
        : ${tailscaled_syslog_output_priority:="info"}
        : ${tailscaled_syslog_output_facility:="daemon"}
        if checkyesno tailscaled_syslog_output_enable; then
                tailscaled_syslog_output_flags="-t ${name} -T ${name}"

                if [ -n "${tailscaled_syslog_output_priority}" ]; then
                        tailscaled_syslog_output_flags="${tailscaled_syslog_output_flags} -s ${tailscaled_syslog_output_priority}"
                fi

                if [ -n "${tailscaled_syslog_output_facility}" ]; then
                        tailscaled_syslog_output_flags="${tailscaled_syslog_output_flags} -l ${tailscaled_syslog_output_facility}"
                fi
        fi
else
        tailscaled_syslog_output_enable="NO"
        tailscaled_syslog_output_flags=""
fi

pidfile=/var/run/${name}.pid
procname="/usr/local/bin/${name}"
ctlname="/usr/local/bin/tailscale"

start_cmd="${name}_start"
start_postcmd="${name}_poststart"
stop_postcmd="${name}_poststop"

tailscaled_start()
{
        # Check for orphaned tailscale network interface
        # And if it exists, then destroy it
        /sbin/ifconfig ${tailscaled_tun_dev} >/dev/null 2>&1 && (
                /sbin/ifconfig ${tailscaled_tun_dev} | fgrep -qw PID ||
                /sbin/ifconfig ${tailscaled_tun_dev} destroy
        )

        /usr/sbin/daemon -f ${tailscaled_syslog_output_flags} -p ${pidfile} ${procname} -port ${tailscaled_port} -tun ${tailscaled_tun_dev} -statedir ${tailscaled_state_dir}
}

tailscaled_poststart()
{
        if checkyesno tailscaled_exitnode_enable; then
                logger -s -t tailscale "Enabling Exit node mode"
                tailscaled_up_args=" --advertise-exit-node ${tailscaled_up_args}"
        fi
        if [ -n "${tailscaled_up_args}" ]; then
                ${ctlname} up ${tailscaled_up_args}
        fi
}

tailscaled_poststop()
{
        /sbin/ifconfig ${tailscaled_tun_dev} >/dev/null 2>&1 && (
                logger -s -t tailscaled "Destroying ${tailscaled_tun_dev} adapter"
                /sbin/ifconfig ${tailscaled_tun_dev} destroy || logger -s -t tailscaled "Failed to destroy ${tailscaled_tun_dev} adapter"
        )
}

run_rc_command "$1"

@yaroslav-gwit commented on GitHub (Mar 28, 2024): Sorry for the delay, here are the `rc` files (I will split it into 2 messages, so it's easier to read): TailScale: ```rc #!/bin/sh # PROVIDE: tailscaled # REQUIRE: NETWORKING # KEYWORD: shutdown # # Add the following lines to /etc/rc.conf.local or /etc/rc.conf # to enable this service: # # tailscaled_enable (bool): Set it to YES to enable tailscaled. # Default is "NO". # tailscaled_state_dir (str): Set the path to use for the state directory. # Default is "/var/db/tailscale" # tailscaled_port (number): Set the port to listen on for incoming VPN packets. # Default is "41641". # tailscaled_syslog_output_enable (bool): Set to enable syslog output. # Default is "NO". See daemon(8). # tailscaled_syslog_output_priority (str): Set syslog priority if syslog enabled. # Default is "info". See daemon(8). # tailscaled_syslog_output_facility (str): Set syslog facility if syslog enabled. # Default is "daemon". See daemon(8). # tailscaled_exitnode_enable (bool): Set it to YES to announce tailscaled as # an exit node. Default is "NO". # tailscaled_up_args (str): Additional arguments to pass to tailscale up # Default is "" (empty string). # tailscaled_tun_dev (str): Set the name of the tun interface tailscaled creates. # Default is "tailscale0" . /etc/rc.subr name=tailscaled rcvar=tailscaled_enable load_rc_config $name : ${tailscaled_enable:="NO"} : ${tailscaled_state_dir:="/var/db/tailscale"} : ${tailscaled_port:="41641"} : ${tailscaled_exitnode_enable:="NO"} : ${tailscaled_up_args:=""} : ${tailscaled_tun_dev:="tailscale0"} DAEMON=$(/usr/sbin/daemon 2>&1 | grep -q syslog ; echo $?) if [ ${DAEMON} -eq 0 ]; then : ${tailscaled_syslog_output_enable:="NO"} : ${tailscaled_syslog_output_priority:="info"} : ${tailscaled_syslog_output_facility:="daemon"} if checkyesno tailscaled_syslog_output_enable; then tailscaled_syslog_output_flags="-t ${name} -T ${name}" if [ -n "${tailscaled_syslog_output_priority}" ]; then tailscaled_syslog_output_flags="${tailscaled_syslog_output_flags} -s ${tailscaled_syslog_output_priority}" fi if [ -n "${tailscaled_syslog_output_facility}" ]; then tailscaled_syslog_output_flags="${tailscaled_syslog_output_flags} -l ${tailscaled_syslog_output_facility}" fi fi else tailscaled_syslog_output_enable="NO" tailscaled_syslog_output_flags="" fi pidfile=/var/run/${name}.pid procname="/usr/local/bin/${name}" ctlname="/usr/local/bin/tailscale" start_cmd="${name}_start" start_postcmd="${name}_poststart" stop_postcmd="${name}_poststop" tailscaled_start() { # Check for orphaned tailscale network interface # And if it exists, then destroy it /sbin/ifconfig ${tailscaled_tun_dev} >/dev/null 2>&1 && ( /sbin/ifconfig ${tailscaled_tun_dev} | fgrep -qw PID || /sbin/ifconfig ${tailscaled_tun_dev} destroy ) /usr/sbin/daemon -f ${tailscaled_syslog_output_flags} -p ${pidfile} ${procname} -port ${tailscaled_port} -tun ${tailscaled_tun_dev} -statedir ${tailscaled_state_dir} } tailscaled_poststart() { if checkyesno tailscaled_exitnode_enable; then logger -s -t tailscale "Enabling Exit node mode" tailscaled_up_args=" --advertise-exit-node ${tailscaled_up_args}" fi if [ -n "${tailscaled_up_args}" ]; then ${ctlname} up ${tailscaled_up_args} fi } tailscaled_poststop() { /sbin/ifconfig ${tailscaled_tun_dev} >/dev/null 2>&1 && ( logger -s -t tailscaled "Destroying ${tailscaled_tun_dev} adapter" /sbin/ifconfig ${tailscaled_tun_dev} destroy || logger -s -t tailscaled "Failed to destroy ${tailscaled_tun_dev} adapter" ) } run_rc_command "$1" ```

saavagebueno commented

@yaroslav-gwit commented on GitHub (Mar 28, 2024):

Nebula rc file:

#!/bin/sh 

# PROVIDE: nebula
# REQUIRE: LOGIN DAEMON NETWORKING
# KEYWORD: shutdown

# To enable nebula, add 'nebula_enable="YES"' to /etc/rc.conf or
# /etc/rc.conf.local

# Optional settings:
# nebula_config (string):   Full path to nebula configuration file
#                           (/usr/local/etc/nebula/config.yml)
# nebula_logfile (string):   Full path to nebula log file
#                           (/var/log/nebula.log)

. /etc/rc.subr 

name="nebula"
rcvar="nebula_enable"
desc="Scalable overlay networking tool with a focus on performance, simplicity and security"
pidfile="/var/run/nebula.pid"
procname="/usr/local/bin/nebula"

load_rc_config $name 

# Defaults
: ${nebula_enable:=NO}
: ${nebula_config:=/usr/local/etc/nebula/config.yml}
: ${nebula_logfile:=/var/log/nebula.log}

command="/usr/sbin/daemon"
command_args="-c -p ${pidfile} -t nebula -o ${nebula_logfile} ${procname} -config ${nebula_config}"

required_files="${nebula_config} ${command}"

extra_commands="configtest"
configtest_cmd="${procname} -test -config ${nebula_config}"

run_rc_command "$1"

@yaroslav-gwit commented on GitHub (Mar 28, 2024): Nebula `rc` file: ```rc #!/bin/sh # PROVIDE: nebula # REQUIRE: LOGIN DAEMON NETWORKING # KEYWORD: shutdown # To enable nebula, add 'nebula_enable="YES"' to /etc/rc.conf or # /etc/rc.conf.local # Optional settings: # nebula_config (string): Full path to nebula configuration file # (/usr/local/etc/nebula/config.yml) # nebula_logfile (string): Full path to nebula log file # (/var/log/nebula.log) . /etc/rc.subr name="nebula" rcvar="nebula_enable" desc="Scalable overlay networking tool with a focus on performance, simplicity and security" pidfile="/var/run/nebula.pid" procname="/usr/local/bin/nebula" load_rc_config $name # Defaults : ${nebula_enable:=NO} : ${nebula_config:=/usr/local/etc/nebula/config.yml} : ${nebula_logfile:=/var/log/nebula.log} command="/usr/sbin/daemon" command_args="-c -p ${pidfile} -t nebula -o ${nebula_logfile} ${procname} -config ${nebula_config}" required_files="${nebula_config} ${command}" extra_commands="configtest" configtest_cmd="${procname} -test -config ${nebula_config}" run_rc_command "$1" ```

saavagebueno commented

@skillcoder commented on GitHub (Mar 30, 2024):

Unfortunately root is required

ifconfig wg create
ifconfig: SIOCIFCREATE2: Operation not permitted

sudo ifconfig wg create
wg1

@skillcoder commented on GitHub (Mar 30, 2024): Unfortunately root is required ```shell ifconfig wg create ifconfig: SIOCIFCREATE2: Operation not permitted ``` ```shell sudo ifconfig wg create wg1 ```

saavagebueno commented

@yaroslav-gwit commented on GitHub (Mar 30, 2024):

Yes, you do need root level privileges to execute most ifconfig commands that change network settings. And after all, it looks like both Nebula and Tailscale are running under root account in FreeBSD anyway.

Quick snapshot, of the process details for Tailscale on FreeBSD:

  PID USER       PRI  NI  VIRT   RES S  CPU%  MEM%   TIME+  Command
59733 root        52   0 1261M 32812 S   0.0  0.4  2:07.89  /usr/local/bin/tailscaled -port 41641 -tun tailscale0 -statedir /var/db/tailscale

@yaroslav-gwit commented on GitHub (Mar 30, 2024): Yes, you do need root level privileges to execute most `ifconfig` commands that change network settings. And after all, it looks like both `Nebula` and `Tailscale` are running under `root` account in FreeBSD anyway. Quick snapshot, of the process details for `Tailscale` on FreeBSD: ```text PID USER PRI NI VIRT RES S CPU% MEM% TIME+ Command 59733 root 52 0 1261M 32812 S 0.0 0.4 2:07.89 /usr/local/bin/tailscaled -port 41641 -tun tailscale0 -statedir /var/db/tailscale ```

saavagebueno commented

@yaroslav-gwit commented on GitHub (Mar 30, 2024):

A draft Go implementation could be something similar to this:

// This function takes in a new WT interface name, creates a new WG interface,
// and renames it to be `wt0` instead of the regular `wg0`.
//
// Returns an error, if there were any problems along the way.
func generateInterface(wtIfaceName string) error {
	// Get the effective user ID
	euid := os.Geteuid()
	// Check if the effective user ID is 0 (root)
	if euid != 0 {
		return fmt.Errorf("netbird must run as root on FreeBSD")
	}
	// Create a new WireGuard network interface
	out, err := exec.Command("ifconfig", "wg", "create").CombinedOutput()
	if err != nil {
		return fmt.Errorf("%s; %s", strings.TrimSpace(string(out)), err.Error())
	}
	// Rename the interface from WG to WT
	currentWgIface := strings.TrimSpace(string(out))
	out, err = exec.Command("ifconfig", currentWgIface, "name", wtIfaceName).CombinedOutput()
	if err != nil {
		return fmt.Errorf("%s; %s", strings.TrimSpace(string(out)), err.Error())
	}

	return nil
}

@yaroslav-gwit commented on GitHub (Mar 30, 2024): A draft Go implementation could be something similar to this: ```go // This function takes in a new WT interface name, creates a new WG interface, // and renames it to be `wt0` instead of the regular `wg0`. // // Returns an error, if there were any problems along the way. func generateInterface(wtIfaceName string) error { // Get the effective user ID euid := os.Geteuid() // Check if the effective user ID is 0 (root) if euid != 0 { return fmt.Errorf("netbird must run as root on FreeBSD") } // Create a new WireGuard network interface out, err := exec.Command("ifconfig", "wg", "create").CombinedOutput() if err != nil { return fmt.Errorf("%s; %s", strings.TrimSpace(string(out)), err.Error()) } // Rename the interface from WG to WT currentWgIface := strings.TrimSpace(string(out)) out, err = exec.Command("ifconfig", currentWgIface, "name", wtIfaceName).CombinedOutput() if err != nil { return fmt.Errorf("%s; %s", strings.TrimSpace(string(out)), err.Error()) } return nil } ```

saavagebueno commented

@skillcoder commented on GitHub (Mar 30, 2024):

Finally wg interface created: wt0 with freebsd ifconfig wrapper

wt0: flags=8080<NOARP,MULTICAST> metric 0 mtu 1420
	options=80000<LINKSTATE>
	inet 100.XX.XX.XX netmask 0xffff0000
	groups: wg
	nd6 options=109<PERFORMNUD,IFDISABLED,NO_DAD>

But ... new portion of linux specific things need to re-implemented for freebsd

2024-03-30T20:13:34Z ERRO client/internal/engine.go:274: failed creating firewall manager: not implemented for this OS: freebsd
2024-03-30T20:13:34Z ERRO client/internal/engine.go:287: failed to pull up wgInterface [wt0]: Not supported OS freebsd. SharedSocket is only supported on Linux
2024-03-30T20:13:34Z ERRO client/internal/connect.go:235: error while starting Netbird Connection Engine: Not supported OS freebsd. SharedSocket is only supported on Linux

@skillcoder commented on GitHub (Mar 30, 2024): Finally `wg interface created: wt0` with freebsd `ifconfig` wrapper ```shell wt0: flags=8080<NOARP,MULTICAST> metric 0 mtu 1420 options=80000<LINKSTATE> inet 100.XX.XX.XX netmask 0xffff0000 groups: wg nd6 options=109<PERFORMNUD,IFDISABLED,NO_DAD> ``` But ... new portion of linux specific things need to re-implemented for freebsd ``` 2024-03-30T20:13:34Z ERRO client/internal/engine.go:274: failed creating firewall manager: not implemented for this OS: freebsd 2024-03-30T20:13:34Z ERRO client/internal/engine.go:287: failed to pull up wgInterface [wt0]: Not supported OS freebsd. SharedSocket is only supported on Linux 2024-03-30T20:13:34Z ERRO client/internal/connect.go:235: error while starting Netbird Connection Engine: Not supported OS freebsd. SharedSocket is only supported on Linux ```

saavagebueno commented

@skillcoder commented on GitHub (Mar 31, 2024):

It looks like implementing kernel space support is too challenging since freebsd do not supported by external libs.
For example https://github.com/mdlayher/socket (SetBPF method not implemented for FreeBSD)
Also highly likely we will have a problem with DNS
So for now I will try to implement netbird for freebsd in userspace.

@skillcoder commented on GitHub (Mar 31, 2024): It looks like implementing kernel space support is too challenging since freebsd do not supported by external libs. For example https://github.com/mdlayher/socket (SetBPF method not implemented for FreeBSD) Also highly likely we will have a problem with DNS So for now I will try to implement netbird for freebsd in userspace.

saavagebueno commented

@skillcoder commented on GitHub (Mar 31, 2024):

Done, netbird on FreeBSD in userspace mode works.
Now I will refactor current code and prepare PR for the merge.
This unblock us to create port and package of netbird on freebsd.

@skillcoder commented on GitHub (Mar 31, 2024): Done, netbird on FreeBSD in userspace mode works. Now I will refactor current code and prepare PR for the merge. This unblock us to create port and package of netbird on freebsd.

saavagebueno commented

@yaroslav-gwit commented on GitHub (Mar 31, 2024):

Did you use WireGuard-Go as an underlying implementation?

@yaroslav-gwit commented on GitHub (Mar 31, 2024): Did you use `WireGuard-Go` as an underlying implementation?

saavagebueno commented

@skillcoder commented on GitHub (Mar 31, 2024):

Yes, it is the same as on linux in userspace mode.
9c2dc05df1/go.mod (L175C39-L175C73)

@skillcoder commented on GitHub (Mar 31, 2024): Yes, it is the same as on linux in userspace mode. https://github.com/netbirdio/netbird/blob/9c2dc05df1a735adc4ab8ec19fc6371c528817c2/go.mod#L175C39-L175C73

saavagebueno commented

@drewhemm commented on GitHub (Jun 3, 2024):

Really hoping for netbird integration with OPNsense, that would be a compelling combination.

@drewhemm commented on GitHub (Jun 3, 2024): Really hoping for netbird integration with OPNsense, that would be a compelling combination.

saavagebueno commented

@yaroslav-gwit commented on GitHub (Jun 4, 2024):

@skillcoder Could you please publish a walk-through on how to run NetBird on FreeBSD (a build process, commands to start the agent, etc)? I bet people will on-board it, and will start using it in various places (pfSense, OPNSense, Jails, etc).

@yaroslav-gwit commented on GitHub (Jun 4, 2024): @skillcoder Could you please publish a walk-through on how to run NetBird on FreeBSD (a build process, commands to start the agent, etc)? I bet people will on-board it, and will start using it in various places (pfSense, OPNSense, Jails, etc).

saavagebueno commented

@skillcoder commented on GitHub (Jul 27, 2024):

@yaroslav-gwit This is how I did it:

Install FreeBSD 13.2 (I have to use this outdated version for now)
install dependencies and build netbird client

pkg install sudo git vim-tiny go zsh
git clone https://github.com/netbirdio/netbird.git
cd netbird/client
go121 build

since netbird client can be run only as root, add yourself to wheel and uncomment %wheel in

vim  /usr/local/etc/sudoers

to be able to use export run zsh

zsh

Connect to network in NETSTACK_MODE (for now I use only this, since I already use to it)
(But user-land wireguard also supported and you could configure it by yourself, I just never did it for now on my production servers)

# still in ~/netbird/client
export NB_USE_NETSTACK_MODE=true
export NB_SOCKS5_LISTENER_PORT=30000
sudo ./client -c ~/.netbird-config.json up -F

Follow https://login.netbird.io/authorize?... link to add this machine to your netbird network (required only once)
in separate terminal on the same server
Copy link from browser and call with curl (required only once)

curl "http://localhost:53000/?code=..."

Check connection to some address inside our netbird network

curl --socks5-hostname localhost:30000 100.12X.XXX.XXX:33000

It works, I just checked it with fresh FreeBSD 13.2 installation.

@skillcoder commented on GitHub (Jul 27, 2024): @yaroslav-gwit This is how I did it: - Install FreeBSD 13.2 (I have to use this outdated version for now) - install dependencies and build netbird client ```shell pkg install sudo git vim-tiny go zsh git clone https://github.com/netbirdio/netbird.git cd netbird/client go121 build ``` - since netbird client can be run only as root, add yourself to wheel and uncomment %wheel in ```shell vim /usr/local/etc/sudoers ``` - to be able to use export run zsh ```shell zsh ``` - Connect to network in NETSTACK_MODE (for now I use only this, since I already use to it) (But user-land wireguard also supported and you could configure it by yourself, I just never did it for now on my production servers) ```shell # still in ~/netbird/client export NB_USE_NETSTACK_MODE=true export NB_SOCKS5_LISTENER_PORT=30000 sudo ./client -c ~/.netbird-config.json up -F ``` - Follow https://login.netbird.io/authorize?... link to add this machine to your netbird network (required only once) in separate terminal on the same server - Copy link from browser and call with curl (required only once) ```shell curl "http://localhost:53000/?code=..." ``` - Check connection to some address inside our netbird network ```shell curl --socks5-hostname localhost:30000 100.12X.XXX.XXX:33000 ``` It works, I just checked it with fresh `FreeBSD 13.2` installation.

saavagebueno commented

@skillcoder commented on GitHub (Jul 27, 2024):

Do we still need FreeBSD port and binary package?
Or someone already did it?

@skillcoder commented on GitHub (Jul 27, 2024): Do we still need FreeBSD port and binary package? Or someone already did it?

saavagebueno commented

@skillcoder commented on GitHub (Aug 2, 2024):

I will take FreeBSD port creation.

@skillcoder commented on GitHub (Aug 2, 2024): I will take FreeBSD port creation.

saavagebueno commented

@skillcoder commented on GitHub (Aug 3, 2024):

Done.
PoC port created and it works.

Now in my test environment setup process looks like this

cd /usr/ports/net/netbird
make install

# First start the netbird service:
 
sysrc netbird_enable=YES
service netbird start

# To connect to your netbird network:

netbird login

# NOTE: Use `curl "http://localhost:53000/?code=..."` on host which need to login if you setup netbird on remote host.

Before submitting this new port to the FreeBSD port committers, we need to resolve a few issues that the core team insists be addressed to ensure all functionality works as expected on FreeBSD and does not break anything. Specifically:

Fix the skipped DNS management tests and add more tests.
Fix the skipped Routing management tests and add more tests.

Additionally, I noticed that in my test environment, the Netbird client loses all connections every 30 seconds, which also needs to be fixed.

We also need to decide where to place the Netbird client config.json. I believe /etc/netbird is a very inappropriate location for FreeBSD, and /usr/local/etc/netbird/config.json is also not the best choice since this is an automatically generated config that we should never need to modify by hand. Therefore, I propose placing it in /var/db/netbird/, which is the standard location for system service state on FreeBSD.

Logs will be written to the standard location /var/log/netbird/client.log.

I want to enable --anonymize flag by default to simplify log sharing.

And will work on rc flags to be able easily configure RC script with standard FreeBSD way via rc.conf to pass custom:

--setup-key
--preshared-key
--management-url
--admin-url
--config
--log-level
--daemon-addr
--hostname
disabling default --anonymize option

@skillcoder commented on GitHub (Aug 3, 2024): Done. PoC port created and it works. Now in my test environment setup process looks like this ``` cd /usr/ports/net/netbird make install # First start the netbird service: sysrc netbird_enable=YES service netbird start # To connect to your netbird network: netbird login # NOTE: Use `curl "http://localhost:53000/?code=..."` on host which need to login if you setup netbird on remote host. ``` Before submitting this new port to the FreeBSD port committers, we need to resolve a few issues that the core team insists be addressed to ensure all functionality works as expected on FreeBSD and does not break anything. Specifically: - Fix the skipped DNS management tests and add more tests. - Fix the skipped Routing management tests and add more tests. Additionally, I noticed that in my test environment, the Netbird client loses all connections every 30 seconds, which also needs to be fixed. We also need to decide where to place the Netbird client `config.json`. I believe `/etc/netbird` is a very inappropriate location for FreeBSD, and `/usr/local/etc/netbird/config.json` is also not the best choice since this is an automatically generated config that we should never need to modify by hand. Therefore, I propose placing it in `/var/db/netbird/`, which is the standard location for system service state on FreeBSD. Logs will be written to the standard location `/var/log/netbird/client.log`. I want to enable `--anonymize` flag by default to simplify log sharing. And will work on rc flags to be able easily configure RC script with standard FreeBSD way via `rc.conf` to pass custom: - --setup-key - --preshared-key - --management-url - --admin-url - --config - --log-level - --daemon-addr - --hostname - disabling default --anonymize option

saavagebueno commented

@skillcoder commented on GitHub (Aug 4, 2024):

For config location I have created a separate issue https://github.com/netbirdio/netbird/issues/2383

@skillcoder commented on GitHub (Aug 4, 2024): For config location I have created a separate issue https://github.com/netbirdio/netbird/issues/2383

saavagebueno commented

@Oriann commented on GitHub (Sep 16, 2024):

I am looking forward to this ! For many people that are using OPNSense for example this will be a dealbreaker because we wont need another container for peer. Thank you guys for your work on this.

@Oriann commented on GitHub (Sep 16, 2024): I am looking forward to this ! For many people that are using OPNSense for example this will be a dealbreaker because we wont need another container for peer. Thank you guys for your work on this.

saavagebueno commented

https://github.com/opnsense/plugins/pull/4386

@sheridans commented on GitHub (Dec 16, 2024):

If netbird works under FreeBSD, and you need assistance with an OPNsense plugin, I'm open t look at that. I've a little experience with OPNsense development, literally just been working on a Tailscale plugin for it.

@sheridans commented on GitHub (Dec 16, 2024): If netbird works under FreeBSD, and you need assistance with an OPNsense plugin, I'm open t look at that. I've a little experience with OPNsense development, literally just been working on a Tailscale plugin for it. https://github.com/opnsense/plugins/pull/4386

saavagebueno commented

@julianbrennum commented on GitHub (Mar 10, 2025):

Is there a roadmap for this?

@julianbrennum commented on GitHub (Mar 10, 2025): Is there a roadmap for this?

saavagebueno commented

@Oriann commented on GitHub (Mar 18, 2025):

Hello guys is there some work on this ? I volunteer for testing on opnsense if you give me package or install script.

@Oriann commented on GitHub (Mar 18, 2025): Hello guys is there some work on this ? I volunteer for testing on opnsense if you give me package or install script.

saavagebueno commented

@hakansa commented on GitHub (Mar 18, 2025):

Hey Folks,

We've submitted our FreeBSD port for NetBird but haven’t received any response from the FreeBSD team yet. We’d really appreciate it if you could leave a comment on the issue. Your support can help bring more attention to the submission and speed up the review process.

Here’s the link to the issue: Bugzilla

/cc @julianbrennum @Oriann

@hakansa commented on GitHub (Mar 18, 2025): Hey Folks, We've submitted our FreeBSD port for NetBird but haven’t received any response from the FreeBSD team yet. We’d really appreciate it if you could leave a comment on the issue. Your support can help bring more attention to the submission and speed up the review process. Here’s the link to the issue: [Bugzilla](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=284877) /cc @julianbrennum @Oriann

saavagebueno commented

@fichtner commented on GitHub (Mar 19, 2025):

I made a note there but to be frank that looks like any other FreeBSD submission. It's hard to find a committer initially.

Cheers,
Franco

@fichtner commented on GitHub (Mar 19, 2025): I made a note there but to be frank that looks like any other FreeBSD submission. It's hard to find a committer initially. Cheers, Franco

saavagebueno commented