[GH-ISSUE #4978] Hitting nginx 404 with Caddy and PocketID #10484

Open
opened 2026-08-05 01:26:05 -04:00 by saavagebueno · 3 comments
Owner

Originally created by @ujaandas on GitHub (Dec 20, 2025).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/4978

Describe the problem

I'm having a little trouble figuring out how I'm supposed to serve Netbird with PocketID on NixOS. I have it running inside a VM, and have all the enableNginx options enabled. Then, in another VM (acts like a nameserver/proxy) which is running Caddy, I forward all Netbird requests to my Netbird VM, ie; reverse_proxy 192.168.100.5:80.

What happens is that when I access netbird.domain, I see the dashboard load, it forwards to PocketID which asks me to authenticate, then it goes to this URL: https://netbird.domain.tld/auth?code=k21CjtEE4PxYhPgyXcmRlgMgMKmpRPrk&state=8ZWS2F3KCd7eP9Cq&iss=https%3A%2F%2Fpocketid.domain.tld, which leads to a 404 from nginx.

I've tried doing it with enableNginx disabled and serving all the routes myself, but I cannot for the life of me get it to work - how do I even serve the frontend? AFAIK, it's served with the mgmt server, but for some reason, I can't access anything.

In any case, I am 99% sure my Netbird configuration is fine, and that it's just something to do with how I'm forwarding requests. In my Caddyfile, all I do is literally just forward all requests to the Netbird VM:

"netbird.${domain}".extraConfig = ''
  reverse_proxy 192.168.100.5:80
'';

Again, my best guess is that ^^^ is forwarding the request to /auth to the wrong place. I've tried more complex ones, like I saw some on closed issues where they forward /api/* to one place, or gRPC requests to another, but (obviously) none of those helped here.

I've also attached my generated management.json file below.

management.json
{
  "DataStoreEncryptionKey": "REDACTED_DATASTORE_KEY",
  "Datadir": "/var/lib/netbird-mgmt/data",
  "DeviceAuthorizationFlow": {
    "Provider": "none",
    "ProviderConfig": {
      "Audience": "netbird",
      "ClientID": "netbird",
      "DeviceAuthEndpoint": "",
      "Domain": null,
      "Scope": "openid profile email groups",
      "TokenEndpoint": null,
      "UseIDToken": true
    }
  },
  "HttpConfig": {
    "Address": "127.0.0.1:8011",
    "AuthAudience": "REDACTED_AUTH_AUDIENCE",
    "IdpSignKeyRefreshEnabled": true,
    "OIDCConfigEndpoint": "https://pocketid.domain.tld/.well-known/openid-configuration"
  },
  "IdpManagerConfig": {
    "Auth0ClientCredentials": null,
    "AzureClientCredentials": null,
    "ClientConfig": {
      "ClientID": "netbird",
      "ClientSecret": "",
      "GrantType": "client_credentials",
      "Issuer": "",
      "TokenEndpoint": ""
    },
    "ExtraConfig": {
      "ApiToken": "REDACTED_API_TOKEN",
      "ManagementEndpoint": "https://pocketid.domain.tld"
    },
    "KeycloakClientCredentials": null,
    "ManagerType": "pocketid",
    "ZitadelClientCredentials": null
  },
  "PKCEAuthorizationFlow": {
    "ProviderConfig": {
      "Audience": "REDACTED_PKCE_AUDIENCE",
      "AuthorizationEndpoint": "",
      "ClientID": "REDACTED_PKCE_CLIENT_ID",
      "ClientSecret": "",
      "RedirectURLs": [
        "http://localhost:53000"
      ],
      "Scope": "openid profile email",
      "TokenEndpoint": "",
      "UseIDToken": false
    }
  },
  "Relay": {
    "Addresses": [
      "rels://netbird.domain.tld:33080"
    ],
    "CredentialsTTL": "24h",
    "Secret": "REDACTED_RELAY_SECRET"
  },
  "ReverseProxy": {
    "TrustedHTTPProxies": [],
    "TrustedHTTPProxiesCount": 0,
    "TrustedPeers": [
      "0.0.0.0/0"
    ]
  },
  "Signal": {
    "Password": null,
    "Proto": "https",
    "URI": "netbird.domain.tld:443",
    "Username": ""
  },
  "StoreConfig": {
    "Engine": "sqlite"
  },
  "Stuns": [
    {
      "Password": null,
      "Proto": "udp",
      "URI": "stun:netbird.domain.tld:3478",
      "Username": ""
    }
  ],
  "TURNConfig": {
    "CredentialsTTL": "12h",
    "Secret": "REDACTED_TURN_SECRET",
    "TimeBasedCredentials": false,
    "Turns": [
      {
        "Password": "REDACTED_TURN_PASSWORD",
        "Proto": "udp",
        "URI": "turn:netbird.domain.tld:3478",
        "Username": "netbird"
      }
    ]
  }
}

Any help at all is greatly appreciated!

UPDATE: Apparently, no route other than the base route works, is this expected?

Some more logs below:

[default@vpn:~]$ curl -v https://netbird.domain.tld/auth
* Host netbird.domain.tld:443 was resolved.
* IPv6: 2606:4700:3033::6815:3a75, 2606:4700:3036::ac43:9f69
* IPv4: 192.168.100.4
*   Trying [2606:4700:3033::6815:3a75]:443...
* Immediate connect fail for 2606:4700:3033::6815:3a75: Network is unreachable
*   Trying 192.168.100.4:443...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* SSL Trust Anchors:
*   OpenSSL default paths (fallback)
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519MLKEM768 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*   subject: CN=*.domain.tld
*   start date: Dec  5 10:38:30 2025 GMT
*   expire date: Mar  5 10:38:29 2026 GMT
*   issuer: C=US; O=Let's Encrypt; CN=E8
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
*   subjectAltName: "netbird.domain.tld" matches cert's "*.domain.tld"
* SSL certificate verified via OpenSSL.
* Established connection to netbird.domain.tld (192.168.100.4 port 443) from 192.168.100.5 port 43832 
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://netbird.domain.tld/auth
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: netbird.domain.tld]
* [HTTP/2] [1] [:path: /auth]
* [HTTP/2] [1] [user-agent: curl/8.17.0]
* [HTTP/2] [1] [accept: */*]
> GET /auth HTTP/2
> Host: netbird.domain.tld
> User-Agent: curl/8.17.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Request completely sent off
< HTTP/2 404 
< alt-svc: h3=":443"; ma=2592000
< content-type: text/html
< date: Mon, 22 Dec 2025 01:44:30 GMT
< server: nginx
< via: 1.1 Caddy
< content-length: 146
< 
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host netbird.domain.tld:443 left intact
[default@vpn:~]$ curl -v http://192.168.100.5:80/auth
*   Trying 192.168.100.5:80...
* Established connection to 192.168.100.5 (192.168.100.5 port 80) from 192.168.100.5 port 58326 
* using HTTP/1.x
> GET /auth HTTP/1.1
> Host: 192.168.100.5
> User-Agent: curl/8.17.0
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 404 Not Found
< Server: nginx
< Date: Mon, 22 Dec 2025 01:44:50 GMT
< Content-Type: text/html
< Content-Length: 146
< Connection: keep-alive
< 
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host 192.168.100.5:80 left intact

Expected behavior

I assume this should redirect to the Netbird dashboard?

Are you using NetBird Cloud?

No, I'm self hosting

NetBird version

I'm using an overlay while I wait for my PR.

Is any other VPN software installed?

Nope.

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • Disabled other VPN software
  • Checked firewall settings
Originally created by @ujaandas on GitHub (Dec 20, 2025). Original GitHub issue: https://github.com/netbirdio/netbird/issues/4978 **Describe the problem** I'm having a little trouble figuring out how I'm supposed to serve Netbird with PocketID on NixOS. I have it running inside a VM, and have all the `enableNginx` options enabled. Then, in another VM (acts like a nameserver/proxy) which is running Caddy, I forward all Netbird requests to my Netbird VM, ie; `reverse_proxy 192.168.100.5:80`. What happens is that when I access `netbird.domain`, I see the dashboard load, it forwards to PocketID which asks me to authenticate, then it goes to this URL: `https://netbird.domain.tld/auth?code=k21CjtEE4PxYhPgyXcmRlgMgMKmpRPrk&state=8ZWS2F3KCd7eP9Cq&iss=https%3A%2F%2Fpocketid.domain.tld`, which leads to a 404 from nginx. I've tried doing it with `enableNginx` disabled and serving all the routes myself, but I cannot for the life of me get it to work - how do I even serve the frontend? AFAIK, it's served with the mgmt server, but for some reason, I can't access anything. In any case, I am 99% sure my Netbird configuration is fine, and that it's just something to do with how I'm forwarding requests. In my Caddyfile, all I do is literally just forward all requests to the Netbird VM: ```nix "netbird.${domain}".extraConfig = '' reverse_proxy 192.168.100.5:80 ''; ``` Again, my best guess is that ^^^ is forwarding the request to `/auth` to the wrong place. I've tried more complex ones, like I saw some on closed issues where they forward `/api/*` to one place, or gRPC requests to another, but (obviously) none of those helped here. I've also attached my generated `management.json` file below. <details> <summary>management.json</summary> ```json { "DataStoreEncryptionKey": "REDACTED_DATASTORE_KEY", "Datadir": "/var/lib/netbird-mgmt/data", "DeviceAuthorizationFlow": { "Provider": "none", "ProviderConfig": { "Audience": "netbird", "ClientID": "netbird", "DeviceAuthEndpoint": "", "Domain": null, "Scope": "openid profile email groups", "TokenEndpoint": null, "UseIDToken": true } }, "HttpConfig": { "Address": "127.0.0.1:8011", "AuthAudience": "REDACTED_AUTH_AUDIENCE", "IdpSignKeyRefreshEnabled": true, "OIDCConfigEndpoint": "https://pocketid.domain.tld/.well-known/openid-configuration" }, "IdpManagerConfig": { "Auth0ClientCredentials": null, "AzureClientCredentials": null, "ClientConfig": { "ClientID": "netbird", "ClientSecret": "", "GrantType": "client_credentials", "Issuer": "", "TokenEndpoint": "" }, "ExtraConfig": { "ApiToken": "REDACTED_API_TOKEN", "ManagementEndpoint": "https://pocketid.domain.tld" }, "KeycloakClientCredentials": null, "ManagerType": "pocketid", "ZitadelClientCredentials": null }, "PKCEAuthorizationFlow": { "ProviderConfig": { "Audience": "REDACTED_PKCE_AUDIENCE", "AuthorizationEndpoint": "", "ClientID": "REDACTED_PKCE_CLIENT_ID", "ClientSecret": "", "RedirectURLs": [ "http://localhost:53000" ], "Scope": "openid profile email", "TokenEndpoint": "", "UseIDToken": false } }, "Relay": { "Addresses": [ "rels://netbird.domain.tld:33080" ], "CredentialsTTL": "24h", "Secret": "REDACTED_RELAY_SECRET" }, "ReverseProxy": { "TrustedHTTPProxies": [], "TrustedHTTPProxiesCount": 0, "TrustedPeers": [ "0.0.0.0/0" ] }, "Signal": { "Password": null, "Proto": "https", "URI": "netbird.domain.tld:443", "Username": "" }, "StoreConfig": { "Engine": "sqlite" }, "Stuns": [ { "Password": null, "Proto": "udp", "URI": "stun:netbird.domain.tld:3478", "Username": "" } ], "TURNConfig": { "CredentialsTTL": "12h", "Secret": "REDACTED_TURN_SECRET", "TimeBasedCredentials": false, "Turns": [ { "Password": "REDACTED_TURN_PASSWORD", "Proto": "udp", "URI": "turn:netbird.domain.tld:3478", "Username": "netbird" } ] } } ``` </details> Any help at all is greatly appreciated! UPDATE: Apparently, no route other than the base route works, is this expected? Some more logs below: <details> ```sh [default@vpn:~]$ curl -v https://netbird.domain.tld/auth * Host netbird.domain.tld:443 was resolved. * IPv6: 2606:4700:3033::6815:3a75, 2606:4700:3036::ac43:9f69 * IPv4: 192.168.100.4 * Trying [2606:4700:3033::6815:3a75]:443... * Immediate connect fail for 2606:4700:3033::6815:3a75: Network is unreachable * Trying 192.168.100.4:443... * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * SSL Trust Anchors: * OpenSSL default paths (fallback) * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS change cipher, Change cipher spec (1): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519MLKEM768 / id-ecPublicKey * ALPN: server accepted h2 * Server certificate: * subject: CN=*.domain.tld * start date: Dec 5 10:38:30 2025 GMT * expire date: Mar 5 10:38:29 2026 GMT * issuer: C=US; O=Let's Encrypt; CN=E8 * Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384 * Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption * subjectAltName: "netbird.domain.tld" matches cert's "*.domain.tld" * SSL certificate verified via OpenSSL. * Established connection to netbird.domain.tld (192.168.100.4 port 443) from 192.168.100.5 port 43832 * using HTTP/2 * [HTTP/2] [1] OPENED stream for https://netbird.domain.tld/auth * [HTTP/2] [1] [:method: GET] * [HTTP/2] [1] [:scheme: https] * [HTTP/2] [1] [:authority: netbird.domain.tld] * [HTTP/2] [1] [:path: /auth] * [HTTP/2] [1] [user-agent: curl/8.17.0] * [HTTP/2] [1] [accept: */*] > GET /auth HTTP/2 > Host: netbird.domain.tld > User-Agent: curl/8.17.0 > Accept: */* > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * Request completely sent off < HTTP/2 404 < alt-svc: h3=":443"; ma=2592000 < content-type: text/html < date: Mon, 22 Dec 2025 01:44:30 GMT < server: nginx < via: 1.1 Caddy < content-length: 146 < <html> <head><title>404 Not Found</title></head> <body> <center><h1>404 Not Found</h1></center> <hr><center>nginx</center> </body> </html> * Connection #0 to host netbird.domain.tld:443 left intact ``` ```sh [default@vpn:~]$ curl -v http://192.168.100.5:80/auth * Trying 192.168.100.5:80... * Established connection to 192.168.100.5 (192.168.100.5 port 80) from 192.168.100.5 port 58326 * using HTTP/1.x > GET /auth HTTP/1.1 > Host: 192.168.100.5 > User-Agent: curl/8.17.0 > Accept: */* > * Request completely sent off < HTTP/1.1 404 Not Found < Server: nginx < Date: Mon, 22 Dec 2025 01:44:50 GMT < Content-Type: text/html < Content-Length: 146 < Connection: keep-alive < <html> <head><title>404 Not Found</title></head> <body> <center><h1>404 Not Found</h1></center> <hr><center>nginx</center> </body> </html> * Connection #0 to host 192.168.100.5:80 left intact ``` </details> **Expected behavior** I assume this should redirect to the Netbird dashboard? **Are you using NetBird Cloud?** No, I'm self hosting **NetBird version** I'm using an [overlay ](https://github.com/ujaandas/netbird/tree/pprof-configurable-signal-mgmt)while I wait for my [PR](https://github.com/netbirdio/netbird/pull/4971). **Is any other VPN software installed?** Nope. **Have you tried these troubleshooting steps?** - [x] Reviewed [client troubleshooting](https://docs.netbird.io/how-to/troubleshooting-client) (if applicable) - [x] Checked for newer NetBird versions - [x] Searched for similar issues on GitHub (including closed ones) - [x] Restarted the NetBird client - [x] Disabled other VPN software - [x] Checked firewall settings
saavagebueno added the triage-needed label 2026-08-05 01:26:05 -04:00
Author
Owner

@coderabbitai[bot] commented on GitHub (Dec 20, 2025):

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

  • Create Plan
Examples

🔗 Similar Issues

Possible Duplicates

  • ISSUE-4829

Related Issues

🔗 Related PRs

netbirdio/netbird#4717 - [client] Add quick actions window [merged]
netbirdio/netbird#4807 - [management] pass config to controller [merged]

👤 Suggested Assignees

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!

<!-- gh-comment-id:3678109191 --> @coderabbitai[bot] commented on GitHub (Dec 20, 2025): <!-- This is an auto-generated issue plan by CodeRabbit --> ### 📝 CodeRabbit Plan Mode Generate an implementation plan and prompts that you can use with your favorite coding agent. - [ ] <!-- {"checkboxId": "8d4f2b9c-3e1a-4f7c-a9b2-d5e8f1c4a7b9"} --> Create Plan <details> <summary>Examples</summary> - [Example 1](https://github.com/coderabbitai/git-worktree-runner/issues/29#issuecomment-3589134556) - [Example 2](https://github.com/coderabbitai/git-worktree-runner/issues/12#issuecomment-3606665167) </details> --- <details> <summary><b>🔗 Similar Issues</b></summary> **Possible Duplicates** - ISSUE-4829 **Related Issues** - https://github.com/netbirdio/netbird/issues/3834 - https://github.com/netbirdio/netbird/issues/4851 - https://github.com/netbirdio/netbird/issues/4829 </details> <details> <summary><b>🔗 Related PRs</b></summary> netbirdio/netbird#4717 - [client] Add quick actions window [merged] netbirdio/netbird#4807 - [management] pass config to controller [merged] </details> <details> <summary><b>👤 Suggested Assignees</b></summary> - [mrll](https://github.com/mrll) - [TheDoDoo](https://github.com/TheDoDoo) - [afonsofrancof](https://github.com/afonsofrancof) - [doromaraujo](https://github.com/doromaraujo) - [pascal-fischer](https://github.com/pascal-fischer) </details> --- <details> <summary> 🧪 Issue enrichment is currently in open beta.</summary> You can configure auto-planning by selecting labels in the issue_enrichment configuration. To disable automatic issue enrichment, add the following to your `.coderabbit.yaml`: ```yaml issue_enrichment: auto_enrich: enabled: false ``` </details> 💬 Have feedback or questions? Drop into our [discord](https://discord.gg/coderabbit) or [schedule a call](https://calendly.com/parth-coderabbit/issues-feedback)!
Author
Owner

@ujaandas commented on GitHub (Dec 21, 2025):

Ah, got it - the issue wasn’t Caddy or NetBird, it was the Nginx config generated by nixpkgs. The dashboard is an SPA, but the module’s default tryFiles doesn’t fall back to index.html (which I assume doesn't let the SPA handle it properly), so routes like /auth and /silent-auth give back a 404.

Overriding the location fixed it:

nginx.virtualHosts."netbird.${domain}".locations."/" = {
  tryFiles = lib.mkForce "$uri $uri/ /index.html";
};
Old

Previously, the tryFiles was:

tryFiles = "$uri $uri.html $uri/ =404";

With this, it works as expected. Can I open a PR for this in nixpkgs?

<!-- gh-comment-id:3679992193 --> @ujaandas commented on GitHub (Dec 21, 2025): Ah, got it - the issue wasn’t Caddy or NetBird, it was the Nginx config generated by nixpkgs. The dashboard is an SPA, but the module’s default `tryFiles` doesn’t fall back to `index.html` (which I assume doesn't let the SPA handle it properly), so routes like `/auth` and `/silent-auth` give back a 404. Overriding the location fixed it: ```nix nginx.virtualHosts."netbird.${domain}".locations."/" = { tryFiles = lib.mkForce "$uri $uri/ /index.html"; }; ``` <details> <summary>Old</summary> Previously, the `tryFiles` was: ```nix tryFiles = "$uri $uri.html $uri/ =404"; ``` </details> With this, it works as expected. Can I open a PR for this in nixpkgs?
Author
Owner

@jasonmoore2k commented on GitHub (Jan 4, 2026):

This explains an issue I've been having putting a WAF (BunkerWeb) in front of NetBird. It was intercepting those 404 errors and causing problems for mee, but I found that the underlying issue was really with the 404 on the /auth and /silent-auth. Falling back to the index.html page was my hack too, by overridinng the default.conf:

try_files $uri $uri.html $uri/ /index.html;

Looks like it was raised over on the dashbaord repo, and now has a fix in flight?
https://github.com/netbirdio/dashboard/issues/398
https://github.com/netbirdio/dashboard/pull/496

<!-- gh-comment-id:3708025262 --> @jasonmoore2k commented on GitHub (Jan 4, 2026): This explains an issue I've been having putting a WAF (BunkerWeb) in front of NetBird. It was intercepting those 404 errors and causing problems for mee, but I found that the underlying issue was really with the 404 on the /auth and /silent-auth. Falling back to the index.html page was my hack too, by overridinng the default.conf: ``` try_files $uri $uri.html $uri/ /index.html; ``` Looks like it was raised over on the dashbaord repo, and now has a fix in flight? https://github.com/netbirdio/dashboard/issues/398 https://github.com/netbirdio/dashboard/pull/496
Sign in to join this conversation.
No Label triage-needed
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#10484