SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-07-28 08:52:37 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

Slow login for new users #582

New Issue
Closed
opened 2025-11-20 05:14:08 -05:00 by saavagebueno · 8 comments
saavagebueno commented 2025-11-20 05:14:08 -05:00
Owner
Copy Link

Originally created by @TSJasonH on GitHub (Jan 15, 2024).

Describe the problem
Netbird interface is slow to load for new users logging in for the first time. (4-5 minutes)

To Reproduce
Launch Netbird from Authentik.

Expected behavior
The netbird gui will load promptly for the user

Additional context
Based on the logs from the management docker I believe this is a problem with the sqlite database.

These entries flood the management server logs:

management-1  | 2024-01-15T21:23:00Z ERRO management/server/event.go:53: received an error while storing an activity event, error: database is locked
management-1  | 2024-01-15T21:23:00Z ERRO management/server/event.go:53: received an error while storing an activity event, error: database is locked
management-1  | 2024-01-15T21:23:00Z ERRO management/server/event.go:53: received an error while storing an activity event, error: database is locked

If I had to guess there may be too many groups in the token for the sqlite db to deal with.
I am sending the user's groups with the claim (include claims in id_token).

Originally created by @TSJasonH on GitHub (Jan 15, 2024). **Describe the problem** Netbird interface is slow to load for new users logging in for the first time. (4-5 minutes) **To Reproduce** Launch Netbird from Authentik. **Expected behavior** The netbird gui will load promptly for the user **Additional context** Based on the logs from the management docker I believe this is a problem with the sqlite database. These entries flood the management server logs: ``` management-1 | 2024-01-15T21:23:00Z ERRO management/server/event.go:53: received an error while storing an activity event, error: database is locked management-1 | 2024-01-15T21:23:00Z ERRO management/server/event.go:53: received an error while storing an activity event, error: database is locked management-1 | 2024-01-15T21:23:00Z ERRO management/server/event.go:53: received an error while storing an activity event, error: database is locked ``` If I had to guess there may be too many groups in the token for the sqlite db to deal with. I am sending the user's groups with the claim (include claims in id_token).
saavagebueno added the bugmanagement-serviceself-hostingstore labels 2025-11-20 05:14:08 -05:00
saavagebueno commented 2025-11-20 05:14:09 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jan 17, 2024):

Hello, @TSJasonH; after seeing your feature request on #1473, where you mentioned the store size, I can say that there is an issue with your setup.

Can you share more details about the following questions and requests:

  1. Do you have JWT group sync enabled?
  2. are your events in the activity tab repeated?
  3. Can you enable debug logs and share the management service logs? To do that you need to update the docker-compose.yml and add the flag "--log-level", "debug" to the service’s command section. Then you can run docker compose logs management > management-service.log 2>&1
@mlsmaycon commented on GitHub (Jan 17, 2024): Hello, @TSJasonH; after seeing your feature request on #1473, where you mentioned the store size, I can say that there is an issue with your setup. Can you share more details about the following questions and requests: 1. Do you have JWT group sync enabled? 2. are your events in the activity tab repeated? 3. Can you enable debug logs and share the management service logs? To do that you need to update the docker-compose.yml and add the flag `"--log-level", "debug"` to the service’s command section. Then you can run `docker compose logs management > management-service.log 2>&1`
saavagebueno commented 2025-11-20 05:14:09 -05:00
Author
Owner
Copy Link

@TSJasonH commented on GitHub (Jan 17, 2024):

Hi @mlsmaycon,

Sure thing.

  1. Do you have JWT group sync enabled?

Yes

  1. are your events in the activity tab repeated?

Yes. For example, when I login I see all the
"Group xyz added to user"
There are several groups naturally, but it does repeat groups of the same name.

  1. Can you enable debug logs and share the management service logs? To do that you need to update the docker-compose.yml and add the flag "--log-level", "debug" to the service’s command section. Then you can run docker compose logs management > management-service.log 2>&1

That log has been attached (I sanitized the hostnames and IP addresses only)

management-service.log

@TSJasonH commented on GitHub (Jan 17, 2024): Hi @mlsmaycon, Sure thing. > 1. Do you have JWT group sync enabled? Yes > 2. are your events in the activity tab repeated? Yes. For example, when I login I see all the "Group xyz added to user" There are several groups naturally, but it does repeat groups of the same name. > 3. Can you enable debug logs and share the management service logs? To do that you need to update the docker-compose.yml and add the flag `"--log-level", "debug"` to the service’s command section. Then you can run `docker compose logs management > management-service.log 2>&1` That log has been attached (I sanitized the hostnames and IP addresses only) [management-service.log](https://github.com/netbirdio/netbird/files/13967895/management-service.log)
saavagebueno commented 2025-11-20 05:14:09 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jan 17, 2024):

Thanks @TSJasonH, after reviewing the log, I can only see an issue with your event.db used for activity events.

Could you share the number of groups in your JWT, a screenshot of your account settings > Groups configuration page and run the following checks for us?

  1. Confirm that your jwt groups are associated with your user and the peers that belong to it.
  2. Then disable JWT group sync( that won't remove any group).
  3. Create a single group and add it to your user.
  4. Validate that it is stored by login out and login in again
  5. Check for duplicates events in your activity events
@mlsmaycon commented on GitHub (Jan 17, 2024): Thanks @TSJasonH, after reviewing the log, I can only see an issue with your event.db used for activity events. Could you share the number of groups in your JWT, a screenshot of your account settings > Groups configuration page and run the following checks for us? 1. Confirm that your jwt groups are associated with your user and the peers that belong to it. 2. Then disable JWT group sync( that won't remove any group). 3. Create a single group and add it to your user. 4. Validate that it is stored by login out and login in again 5. Check for duplicates events in your activity events
saavagebueno commented 2025-11-20 05:14:09 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jan 18, 2024):

Hello @TSJasonH we were able to reproduce the issue and will work on a fix

@mlsmaycon commented on GitHub (Jan 18, 2024): Hello @TSJasonH we were able to reproduce the issue and will work on a fix
saavagebueno commented 2025-11-20 05:14:10 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jan 20, 2024):

Hello @TSJasonH the PR #1486 will fix the issue with duplicated events.

For the database size, you have a few options, but unfortunately, they all involve removing data.

  1. You can delete the events.db file, and lose activity events until now, here are the steps for that:
# find the local path for the management volume
docker volume inspect netbird-mgmt | jq '.[].Mountpoint'
# stop the service
docker compose stop managmeent
# Move the events file to a backup location
mv <VOLUME PATH>/events.db ./events.db.backup
# start the service
docker compose stop managmeent
  1. clean the group add events with sqlite3 command, which might be the best option as keep most of the relevant events:
# install sqlite3
# find the local path for the management volume
docker volume inspect netbird-mgmt | jq '.[].Mountpoint'
# stop the service
docker compose stop managmeent
# do a backup of events file 
cp <VOLUME PATH>/events.db ./events.db.backup
# run sqlite3
sqlite3 <VOLUME PATH>/events.db
# run the delete statement
delete from events where activity = 20;
# exit sqlite3
# start the service
docker compose start managmeent
@mlsmaycon commented on GitHub (Jan 20, 2024): Hello @TSJasonH the PR #1486 will fix the issue with duplicated events. For the database size, you have a few options, but unfortunately, they all involve removing data. 1. You can delete the events.db file, and lose activity events until now, here are the steps for that: ```shell # find the local path for the management volume docker volume inspect netbird-mgmt | jq '.[].Mountpoint' # stop the service docker compose stop managmeent # Move the events file to a backup location mv <VOLUME PATH>/events.db ./events.db.backup # start the service docker compose stop managmeent ``` 2. clean the group add events with `sqlite3` command, which might be the best option as keep most of the relevant events: ```shell # install sqlite3 # find the local path for the management volume docker volume inspect netbird-mgmt | jq '.[].Mountpoint' # stop the service docker compose stop managmeent # do a backup of events file cp <VOLUME PATH>/events.db ./events.db.backup # run sqlite3 sqlite3 <VOLUME PATH>/events.db # run the delete statement delete from events where activity = 20; # exit sqlite3 # start the service docker compose start managmeent ```
saavagebueno commented 2025-11-20 05:14:10 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jan 26, 2024):

Hello @TSJasonH, the fix is included in the release 0.25.5. Please upgrade and perform the database cleanup as suggested.

@mlsmaycon commented on GitHub (Jan 26, 2024): Hello @TSJasonH, the fix is included in the release 0.25.5. Please upgrade and perform the database cleanup as suggested.
saavagebueno commented 2025-11-20 05:14:10 -05:00
Author
Owner
Copy Link

@TSJasonH commented on GitHub (Jan 26, 2024):

The "delete from events where activity = 20;" command completed without issues but didn't seem to actually shrink the size of the db file at all. I ended up just renaming the old one and allowing a new one to be created (no big deal on the data loss for me since this isn't in production yet).

Additionally the last docker command has "stop" where I think you meant to write "start" for the management container.

I can confirm that groups were applied properly now when logging into the interface, and it didn't take an extended period of time to complete.

Thanks!

@TSJasonH commented on GitHub (Jan 26, 2024): The "delete from events where activity = 20;" command completed without issues but didn't seem to actually shrink the size of the db file at all. I ended up just renaming the old one and allowing a new one to be created (no big deal on the data loss for me since this isn't in production yet). Additionally the last docker command has "stop" where I think you meant to write "start" for the management container. I can confirm that groups were applied properly now when logging into the interface, and it didn't take an extended period of time to complete. Thanks!
saavagebueno commented 2025-11-20 05:14:10 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Jan 26, 2024):

I see; I believe we missed a vacuum command to shrink the database file after deleting the rows.

@mlsmaycon commented on GitHub (Jan 26, 2024): I see; I believe we missed a vacuum command to shrink the database file after deleting the rows.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label bug management-service self-hosting store
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#582
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?