SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-05-16 21:22:12 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

Unable to access dashboard after upgrading management to 0.34.1 (Authentik) #1491

New Issue
Open
opened 2025-11-20 05:31:38 -05:00 by saavagebueno · 10 comments
saavagebueno commented 2025-11-20 05:31:38 -05:00
Owner
Copy Link

Originally created by @reginaldosoares on GitHub (Dec 11, 2024).

Issue:
The dashboard becomes inaccessible after upgrading the Management and Signal components to version 0.34.1.

Details:

Environment: Self-hosted instance maintained for over a year without issues.
Identity Provider (IdP): Authentik.

Troubleshooting Steps Taken:

  1. Rollback Attempt: Reverted to previous versions of Management and Signal; however, the dashboard remains inaccessible.
  2. Configuration Review: Examined and cross-checked IdP settings and Management configurations without identifying any issues.

Possible Cause:
I believe this issue may be related to the recent change introduced in the following pull request:
Update account peers on login on meta change #2991

Additionally, this problem might be correlated with:
Stuck on loading screen on "/peers" (Authentik) #3007

dashboard error message:

Request failed with status code 500

Error: Internal server error

management relevant logs:

2024-12-11T09:50:49Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:122: keys refreshed, new UTC expiration time: 2024-12-11 09:50:49.739044538 +0000 UTC

2024-12-11T09:50:49Z DEBG management/server/account.go:1515: account cj6m6d83pjfs73fpq20g not found in cache, reloading

2024-12-11T09:50:49Z ERRO [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2, accountID: , userID: 6] management/server/http/util/util.go:81: got a handler error: 403 Forbidden

2024-12-11T09:50:49Z ERRO [userID: 6, context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2, accountID: ] management/server/http/util/util.go:110: got unhandled error code, error: 403 Forbidden

2024-12-11T09:50:49Z ERRO [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2] management/server/telemetry/http_api_metrics.go:168: HTTP response f457f469-16c8-44cb-b265-a9d7c8e568d2: GET /api/users status 500

2024-12-11T09:50:49Z DEBG [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 546 ms and finished with status 500
image
Originally created by @reginaldosoares on GitHub (Dec 11, 2024). **Issue:** The dashboard becomes inaccessible after upgrading the Management and Signal components to version 0.34.1. **Details:** Environment: Self-hosted instance maintained for over a year without issues. Identity Provider (IdP): Authentik. **Troubleshooting Steps Taken:** 1. **Rollback Attempt:** Reverted to previous versions of Management and Signal; however, the dashboard remains inaccessible. 2. **Configuration Review:** Examined and cross-checked IdP settings and Management configurations without identifying any issues. **Possible Cause:** I believe this issue may be related to the recent change introduced in the following pull request: [Update account peers on login on meta change #2991](https://github.com/netbirdio/netbird/pull/2991) Additionally, this problem might be correlated with: [Stuck on loading screen on "/peers" (Authentik) #3007](https://github.com/netbirdio/netbird/issues/3007) **dashboard error message:** ``` Request failed with status code 500 Error: Internal server error ``` **management relevant logs:** ``` 2024-12-11T09:50:49Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:122: keys refreshed, new UTC expiration time: 2024-12-11 09:50:49.739044538 +0000 UTC 2024-12-11T09:50:49Z DEBG management/server/account.go:1515: account cj6m6d83pjfs73fpq20g not found in cache, reloading 2024-12-11T09:50:49Z ERRO [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2, accountID: , userID: 6] management/server/http/util/util.go:81: got a handler error: 403 Forbidden 2024-12-11T09:50:49Z ERRO [userID: 6, context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2, accountID: ] management/server/http/util/util.go:110: got unhandled error code, error: 403 Forbidden 2024-12-11T09:50:49Z ERRO [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2] management/server/telemetry/http_api_metrics.go:168: HTTP response f457f469-16c8-44cb-b265-a9d7c8e568d2: GET /api/users status 500 2024-12-11T09:50:49Z DEBG [context: HTTP, requestID: f457f469-16c8-44cb-b265-a9d7c8e568d2] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 546 ms and finished with status 500 ``` <img width="1260" alt="image" src="https://github.com/user-attachments/assets/ead0911e-9766-45f5-9ad6-26a81ff02938">
saavagebueno added the dashboardwaiting-feedbackidpself-hosting labels 2025-11-20 05:31:38 -05:00
saavagebueno commented 2025-11-20 05:31:38 -05:00
Author
Owner
Copy Link

@mgarces commented on GitHub (Dec 11, 2024):

hello, have you tried to open your dashboard in incognito mode?

@mgarces commented on GitHub (Dec 11, 2024): hello, have you tried to open your dashboard in incognito mode?
saavagebueno commented 2025-11-20 05:31:39 -05:00
Author
Owner
Copy Link

@reginaldosoares commented on GitHub (Dec 11, 2024):

Hello @mgarces, yes. Attempted different browsers, other account.

@reginaldosoares commented on GitHub (Dec 11, 2024): Hello @mgarces, yes. Attempted different browsers, other account.
saavagebueno commented 2025-11-20 05:31:39 -05:00
Author
Owner
Copy Link

@mgarces commented on GitHub (Dec 11, 2024):

can you look into developer tools on the browser, and see if there is any endpoint not working?

@mgarces commented on GitHub (Dec 11, 2024): can you look into developer tools on the browser, and see if there is any endpoint not working?
saavagebueno commented 2025-11-20 05:31:39 -05:00
Author
Owner
Copy Link

@mgarces commented on GitHub (Dec 11, 2024):

can you run the latest dashboard version too?

@mgarces commented on GitHub (Dec 11, 2024): can you run the latest dashboard version too?
saavagebueno commented 2025-11-20 05:31:39 -05:00
Author
Owner
Copy Link

@reginaldosoares commented on GitHub (Dec 11, 2024):

deployed on dashboard:v2.7.1

dev tools:
image

image

tracing logs of the management:
relevant log:

getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:155: HTTP request 9ac3aa45-80a5-49ae-95cf-6cc82d21582b: GET /api/users

2024-12-11T10:45:26Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:122: keys refreshed, new UTC expiration time: 2024-12-11 10:45:26.737845505 +0000 UTC

2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: , userID: 6] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z TRAC [userID: 6, context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: ] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: ""

2024-12-11T10:45:26Z DEBG management/server/account.go:1515: account cj6m6d83pjfs73fpq20g not found in cache, reloading

2024-12-11T10:45:26Z ERRO [userID: 6, context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: ] management/server/http/util/util.go:81: got a handler error: 403 Forbidden

2024-12-11T10:45:26Z ERRO [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: , userID: 6] management/server/http/util/util.go:110: got unhandled error code, error: 403 Forbidden

2024-12-11T10:45:26Z ERRO [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:168: HTTP response 9ac3aa45-80a5-49ae-95cf-6cc82d21582b: GET /api/users status 500

2024-12-11T10:45:26Z DEBG [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 673 ms and finished with status 500
@reginaldosoares commented on GitHub (Dec 11, 2024): deployed on dashboard:v2.7.1 dev tools: <img width="576" alt="image" src="https://github.com/user-attachments/assets/7677a7db-d20e-497f-9d88-3403fc976390"> <img width="1275" alt="image" src="https://github.com/user-attachments/assets/affc8f93-fe0a-4671-bcf6-cc85ce970978"> **tracing logs of the management:** relevant log: > getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: "" ``` 2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:155: HTTP request 9ac3aa45-80a5-49ae-95cf-6cc82d21582b: GET /api/users 2024-12-11T10:45:26Z DEBG [context: SYSTEM] management/server/jwtclaims/jwtValidator.go:122: keys refreshed, new UTC expiration time: 2024-12-11 10:45:26.737845505 +0000 UTC 2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: "" 2024-12-11T10:45:26Z TRAC [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: , userID: 6] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: "" 2024-12-11T10:45:26Z TRAC [userID: 6, context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: ] management/server/account.go:2206: getting account with authorization claims. User ID: "6", Account ID: "", Domain: "", Domain Category: "" 2024-12-11T10:45:26Z DEBG management/server/account.go:1515: account cj6m6d83pjfs73fpq20g not found in cache, reloading 2024-12-11T10:45:26Z ERRO [userID: 6, context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: ] management/server/http/util/util.go:81: got a handler error: 403 Forbidden 2024-12-11T10:45:26Z ERRO [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b, accountID: , userID: 6] management/server/http/util/util.go:110: got unhandled error code, error: 403 Forbidden 2024-12-11T10:45:26Z ERRO [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:168: HTTP response 9ac3aa45-80a5-49ae-95cf-6cc82d21582b: GET /api/users status 500 2024-12-11T10:45:26Z DEBG [context: HTTP, requestID: 9ac3aa45-80a5-49ae-95cf-6cc82d21582b] management/server/telemetry/http_api_metrics.go:181: request GET /api/users took 673 ms and finished with status 500 ```
saavagebueno commented 2025-11-20 05:31:39 -05:00
Author
Owner
Copy Link

@deffcolony commented on GitHub (Dec 18, 2024):

I got a similair issue but mine says Error: Unauthenticated this happend after upgrading to latest version of netbird
image

can we please validate if we have the similair settings? did something change in the scope? how can we resolve this issue?
image

@deffcolony commented on GitHub (Dec 18, 2024): I got a similair issue but mine says `Error: Unauthenticated` this happend after upgrading to latest version of netbird ![image](https://github.com/user-attachments/assets/0e7e7a55-edfd-43a9-9473-8fb405717c88) can we please validate if we have the similair settings? did something change in the scope? how can we resolve this issue? ![image](https://github.com/user-attachments/assets/de70e890-c382-4847-bf73-cff728c7781f)
saavagebueno commented 2025-11-20 05:31:39 -05:00
Author
Owner
Copy Link

@jvanbruegge commented on GitHub (Jan 10, 2025):

I have a similar issue and I found this in the authentik logs:

{"auth_via": "unauthenticated", "domain_url": "authentik.<domain>", "event": "/api/v3/core/users/?page=1", "host": "authentik.<domain>", "level": "info", "logger": "authentik.asgi", "method": "GET", "pid": 3369829, "remote": "127.0.0.1", "request_id": "fbdd3550f688401c98ac5a00af4d1a3f", "runtime": 37, "schema_name": "public", "scheme": "https", "status": 403, "timestamp": "2025-01-10T18:20:50.917033", "user": "", "user_agent": "OpenAPI-Generator/1.0.0/go"}

which would suggest that netbird is trying to get the users without using the service account

@jvanbruegge commented on GitHub (Jan 10, 2025): I have a similar issue and I found this in the authentik logs: ``` {"auth_via": "unauthenticated", "domain_url": "authentik.<domain>", "event": "/api/v3/core/users/?page=1", "host": "authentik.<domain>", "level": "info", "logger": "authentik.asgi", "method": "GET", "pid": 3369829, "remote": "127.0.0.1", "request_id": "fbdd3550f688401c98ac5a00af4d1a3f", "runtime": 37, "schema_name": "public", "scheme": "https", "status": 403, "timestamp": "2025-01-10T18:20:50.917033", "user": "", "user_agent": "OpenAPI-Generator/1.0.0/go"} ``` which would suggest that netbird is trying to get the users without using the service account
saavagebueno commented 2025-11-20 05:31:40 -05:00
Author
Owner
Copy Link

@jvanbruegge commented on GitHub (Jan 11, 2025):

Never mind, the solution from here: https://github.com/netbirdio/netbird/issues/2941#issuecomment-2503971240 fixed my issue, I did need to add the API access scope

@jvanbruegge commented on GitHub (Jan 11, 2025): Never mind, the solution from here: https://github.com/netbirdio/netbird/issues/2941#issuecomment-2503971240 fixed my issue, I did need to add the API access scope
saavagebueno commented 2025-11-20 05:31:40 -05:00
Author
Owner
Copy Link

@nazarewk commented on GitHub (Apr 28, 2025):

Hello @reginaldosoares,

We're currently reviewing our open issues and would like to verify if this problem still exists in the latest NetBird version.

Could you please confirm if the issue is still there?

We may close this issue temporarily if we don't hear back from you within 2 weeks, but feel free to reopen it with updated information.

Thanks for your contribution to improving the project!

@nazarewk commented on GitHub (Apr 28, 2025): Hello @reginaldosoares, We're currently reviewing our open issues and would like to verify if this problem still exists in the [latest NetBird version](https://github.com/netbirdio/netbird/releases). Could you please confirm if the issue is still there? We may close this issue temporarily if we don't hear back from you within **2 weeks**, but feel free to reopen it with updated information. Thanks for your contribution to improving the project!
saavagebueno commented 2025-11-20 05:31:40 -05:00
Author
Owner
Copy Link

@kreativmonkey commented on GitHub (May 19, 2025):

after upgrades this problem is still present on my setup.

Authentik version 2025.4.1 and Netbird version 0.44.0.

What i've tried:

  1. updated the Access Token
  2. added a new provider and user for netbird.
  3. Checked all settings again and again.

i don't know what to try next. devices that are connected are fine but i can not manage anything at all!

@kreativmonkey commented on GitHub (May 19, 2025): after upgrades this problem is still present on my setup. Authentik version 2025.4.1 and Netbird version 0.44.0. What i've tried: 1. updated the Access Token 2. added a new provider and user for netbird. 3. Checked all settings again and again. i don't know what to try next. devices that are connected are fine but i can not manage anything at all!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label dashboard idp self-hosting waiting-feedback
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#1491
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?