SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-05-14 20:21:59 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

[PR #4489] feat: optimize GetAccount for large number of policies and rules #4274

New Issue
Open
opened 2025-11-20 08:08:04 -05:00 by saavagebueno · 0 comments
saavagebueno commented 2025-11-20 08:08:04 -05:00
Owner
Copy Link

Original Pull Request: https://github.com/netbirdio/netbird/pull/4489

State: open
Merged: No

Describe your changes

The policy rules are loaded one by one, for each policy. In a set-up with 480 policies, it generates 480 round trip to the database and a latency of about 500 ms. The fist change I propose here is therefore to load all the rules in one query, using a IN clause with the list of policy id's. This is very low risk and was tested "live" on my set-up.

The second change is low impact on the performance, it is just to avoid two times the same query which is silly. The ORM generates a select for all policies rules with the IN clause (exactly the same as my select). However the result of the select is not used, for some reason the ORM cannot attach it to the slice of rules pointers (hence the reason why it is done manually, as commented in the original code).

Because the code uses a generic "Preload(clause.Associations)" it is not possible to exclude one of the association to avoid this double select. Furthermore it seems a good practice to explicitly list the associations that should be preloaded, to have a more reliable behavior and better documented.

The refactored code produce the same account data on my set-up (large setup), I serialized the account in json and compared.

If requested I can a test in place for the GetAccount, the current seems very limitied (does not mirror the current schema).

I realize this change is important for large number of policies only, which might be an hedge case, but it is very low risk.

The saving is significant on my setup: from 600 -700 ms to 60 - 80ms. Of course I would prefer to have the change upstream than only in my setup.

Issue

4488

Stack

Checklist

  • Is it a bug fix
  • Is a typo/documentation fix
  • Is a feature enhancement
  • It is a refactor
  • Created tests that fail without the change (if possible)

By submitting this pull request, you confirm that you have read and agree to the terms of the Contributor License Agreement.

Documentation

Select exactly one:

  • I added/updated documentation for this change
  • [ X] Documentation is not needed for this change (explain why)

it is an internal optimization

Docs PR URL (required if "docs added" is checked)

Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

**Original Pull Request:** https://github.com/netbirdio/netbird/pull/4489 **State:** open **Merged:** No --- ## Describe your changes The policy rules are loaded one by one, for each policy. In a set-up with 480 policies, it generates 480 round trip to the database and a latency of about 500 ms. The fist change I propose here is therefore to load all the rules in one query, using a IN clause with the list of policy id's. This is very low risk and was tested "live" on my set-up. The second change is low impact on the performance, it is just to avoid two times the same query which is silly. The ORM generates a select for all policies rules with the IN clause (exactly the same as my select). However the result of the select is not used, for some reason the ORM cannot attach it to the slice of rules pointers (hence the reason why it is done manually, as commented in the original code). Because the code uses a generic "Preload(clause.Associations)" it is not possible to exclude one of the association to avoid this double select. Furthermore it seems a good practice to explicitly list the associations that should be preloaded, to have a more reliable behavior and better documented. The refactored code produce the same account data on my set-up (large setup), I serialized the account in json and compared. If requested I can a test in place for the GetAccount, the current seems very limitied (does not mirror the current schema). I realize this change is important for large number of policies only, which might be an hedge case, but it is very low risk. The saving is significant on my setup: from 600 -700 ms to 60 - 80ms. Of course I would prefer to have the change upstream than only in my setup. ## Issue [4488](https://github.com/netbirdio/netbird/issues/4488) ## Stack <!-- branch-stack --> ### Checklist - [ ] Is it a bug fix - [ ] Is a typo/documentation fix - [ ] Is a feature enhancement - [X] It is a refactor - [ ] Created tests that fail without the change (if possible) > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md). ## Documentation Select exactly one: - [ ] I added/updated documentation for this change - [ X] Documentation is **not needed** for this change (explain why) it is an internal optimization ### Docs PR URL (required if "docs added" is checked) Paste the PR link from https://github.com/netbirdio/docs here: https://github.com/netbirdio/docs/pull/__
saavagebueno added the pull-request label 2025-11-20 08:08:04 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#4274
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?