SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-05-16 21:22:12 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

Seeing sysctl errors when adding a lot of routes #686

New Issue
Closed
opened 2025-11-20 05:15:58 -05:00 by saavagebueno · 10 comments
saavagebueno commented 2025-11-20 05:15:58 -05:00
Owner
Copy Link

Originally created by @hurricanehrndz on GitHub (Mar 8, 2024).

Describe the problem

A clear and concise description of what the problem is.

2024-03-08T14:53:05-07:00 ERRO client/internal/routemanager/client.go:276: route {subnet} couldn't be added for peer 100.122.204.13, err: sysctl: cannot allocate memory

To Reproduce
Add many routes to a routing

Steps to reproduce the behavior:

  1. Advertise and extensive amount of routes
  2. Connect
  3. Grep for sysctl lines
  4. See error

Expected behavior

I am assuming errors with allocating memory is not good

A clear and concise description of what you expected to happen.

Are you using NetBird Cloud?
No, self hosted

NetBird version
0.26.2
netbird version

NetBird status -d output:

Originally created by @hurricanehrndz on GitHub (Mar 8, 2024). **Describe the problem** A clear and concise description of what the problem is. ```sh 2024-03-08T14:53:05-07:00 ERRO client/internal/routemanager/client.go:276: route {subnet} couldn't be added for peer 100.122.204.13, err: sysctl: cannot allocate memory ``` **To Reproduce** Add many routes to a routing Steps to reproduce the behavior: 1. Advertise and extensive amount of routes 2. Connect 3. Grep for sysctl lines 4. See error **Expected behavior** I am assuming errors with allocating memory is not good A clear and concise description of what you expected to happen. **Are you using NetBird Cloud?** No, self hosted **NetBird version** 0.26.2 `netbird version` **NetBird status -d output:**
saavagebueno added the bugclientroutesmacos labels 2025-11-20 05:15:58 -05:00
saavagebueno commented 2025-11-20 05:15:59 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 8, 2024):

@hurricanehrndz can you share how many routes, the OS and how much memory you have in the system?

@mlsmaycon commented on GitHub (Mar 8, 2024): @hurricanehrndz can you share how many routes, the OS and how much memory you have in the system?
saavagebueno commented 2025-11-20 05:15:59 -05:00
Author
Owner
Copy Link

@hurricanehrndz commented on GitHub (Mar 9, 2024):

@mlsmaycon this has nothing to do with memory, but how BSD systems operate. As I understand it the routing tables get expanded as routes are added. At times when said routing tables are being expanded the commands will fail as describe above.

I am on a macos device, I have implemented a less than elegant solution. It does work though.

From d660cad7d066078441a58dfb65d0f28b68c0995f Mon Sep 17 00:00:00 2001
From: Carlos Hernandez <carlos@hrndz.ca>
Date: Fri, 8 Mar 2024 15:46:00 -0700
Subject: [PATCH] Hail-mary for route issues

---
 client/internal/routemanager/client.go | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/client/internal/routemanager/client.go b/client/internal/routemanager/client.go
index ee98d50..dd812f8 100644
--- a/client/internal/routemanager/client.go
+++ b/client/internal/routemanager/client.go
@@ -271,7 +271,14 @@ func (c *clientNetwork) peersStateAndUpdateWatcher() {
 			}
 			return
 		case <-c.peerStateUpdate:
-			err := c.recalculateRouteAndUpdatePeerAndSystem()
+			var err error
+			for i := 1; i <=5; i++ {
+				err = c.recalculateRouteAndUpdatePeerAndSystem()
+				if err == nil {
+					break
+				}
+				log.Error("Re-Attempting to recalculateRouteAndUpdatePeerAndSystem")
+			}
 			if err != nil {
 				log.Error(err)
 			}
@hurricanehrndz commented on GitHub (Mar 9, 2024): @mlsmaycon this has nothing to do with memory, but how BSD systems operate. As I understand it the routing tables get expanded as routes are added. At times when said routing tables are being expanded the commands will fail as describe above. I am on a macos device, I have implemented a less than elegant solution. It does work though. ``` From d660cad7d066078441a58dfb65d0f28b68c0995f Mon Sep 17 00:00:00 2001 From: Carlos Hernandez <carlos@hrndz.ca> Date: Fri, 8 Mar 2024 15:46:00 -0700 Subject: [PATCH] Hail-mary for route issues --- client/internal/routemanager/client.go | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/client/internal/routemanager/client.go b/client/internal/routemanager/client.go index ee98d50..dd812f8 100644 --- a/client/internal/routemanager/client.go +++ b/client/internal/routemanager/client.go @@ -271,7 +271,14 @@ func (c *clientNetwork) peersStateAndUpdateWatcher() { } return case <-c.peerStateUpdate: - err := c.recalculateRouteAndUpdatePeerAndSystem() + var err error + for i := 1; i <=5; i++ { + err = c.recalculateRouteAndUpdatePeerAndSystem() + if err == nil { + break + } + log.Error("Re-Attempting to recalculateRouteAndUpdatePeerAndSystem") + } if err != nil { log.Error(err) } ```
saavagebueno commented 2025-11-20 05:15:59 -05:00
Author
Owner
Copy Link

@hurricanehrndz commented on GitHub (Mar 9, 2024):

You can find more info on the issue here:
https://lists.freebsd.org/pipermail/freebsd-current/2014-February/048416.html

@hurricanehrndz commented on GitHub (Mar 9, 2024): You can find more info on the issue here: https://lists.freebsd.org/pipermail/freebsd-current/2014-February/048416.html
saavagebueno commented 2025-11-20 05:16:00 -05:00
Author
Owner
Copy Link

@hurricanehrndz commented on GitHub (Mar 12, 2024):

@mlsmaycon did you want a PR?

@hurricanehrndz commented on GitHub (Mar 12, 2024): @mlsmaycon did you want a PR?
saavagebueno commented 2025-11-20 05:16:00 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 12, 2024):

@hurricanehrndz I would like to validate it a bit more on our end.

Can you share the number of routes it was added to when you faced the issue?

@mlsmaycon commented on GitHub (Mar 12, 2024): @hurricanehrndz I would like to validate it a bit more on our end. Can you share the number of routes it was added to when you faced the issue?
saavagebueno commented 2025-11-20 05:16:00 -05:00
Author
Owner
Copy Link

@hurricanehrndz commented on GitHub (Mar 12, 2024):

Let me see if I can get you a santized version of it

@hurricanehrndz commented on GitHub (Mar 12, 2024): Let me see if I can get you a santized version of it
saavagebueno commented 2025-11-20 05:16:00 -05:00
Author
Owner
Copy Link

@hurricanehrndz commented on GitHub (Mar 12, 2024):

112 routes

@hurricanehrndz commented on GitHub (Mar 12, 2024): 112 routes
saavagebueno commented 2025-11-20 05:16:00 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Mar 12, 2024):

Got it, thanks

@mlsmaycon commented on GitHub (Mar 12, 2024): Got it, thanks
saavagebueno commented 2025-11-20 05:16:00 -05:00
Author
Owner
Copy Link

@hurricanehrndz commented on GitHub (Mar 27, 2024):

Any further thoughts whether I should submit a PR for this?

@hurricanehrndz commented on GitHub (Mar 27, 2024): Any further thoughts whether I should submit a PR for this?
saavagebueno commented 2025-11-20 05:16:00 -05:00
Author
Owner
Copy Link

@lixmal commented on GitHub (Mar 27, 2024):

Hi @hurricanehrndz, I haven't been able to reproduce the issue so far, but from my research a single retry should suffice if this appears. Thanks for your patch, however I would like to issue the retry closer to the macos system command.

@lixmal commented on GitHub (Mar 27, 2024): Hi @hurricanehrndz, I haven't been able to reproduce the issue so far, but from my research a single retry should suffice if this appears. Thanks for your patch, however I would like to issue the retry closer to the macos system command.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label bug client macos routes
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#686
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?