Install script doesn't provide a working enviroment - All clients fails intantly. #833

Closed
opened 2025-11-20 05:18:19 -05:00 by saavagebueno · 14 comments
Owner

Originally created by @zzecool on GitHub (Apr 30, 2024).

Describe the problem

A clear and concise description of what the problem is.

To Reproduce

Steps to reproduce the behavior:
Trying to deply docker client,

Container fails instantly and shuts down.

Logs are the following :

2024-04-30T13:43:32.453860580Z 2024-04-30T13:43:32Z ERRO client/internal/login.go:105: failed while getting Management Service public key: rpc error: code = PermissionDenied desc = unexpected HTTP status code received from server: 403 (Forbidden); transport: received unexpected content-type "text/html"
2024-04-30T13:43:32.456290315Z panic: runtime error: invalid memory address or nil pointer dereference
2024-04-30T13:43:32.456476020Z [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf34db6]
2024-04-30T13:43:32.456559316Z 
2024-04-30T13:43:32.456598309Z goroutine 1 [running]:
2024-04-30T13:43:32.456759022Z github.com/netbirdio/netbird/client/internal.Login({0x15df9f0, 0xc0007f08a0}, 0xc0007ca8f0, {0x0, 0x0}, {0x0, 0x0})
2024-04-30T13:43:32.456850029Z 	/home/runner/work/netbird/netbird/client/internal/login.go:73 +0x236
2024-04-30T13:43:32.456905564Z github.com/netbirdio/netbird/client/cmd.foregroundLogin.func1()
2024-04-30T13:43:32.456957623Z 	/home/runner/work/netbird/netbird/client/cmd/login.go:137 +0x36
2024-04-30T13:43:32.457239463Z github.com/cenkalti/backoff/v4.RetryNotifyWithTimer(0xc000c3fa28, {0x15d38a0, 0x1fadd40}, 0x1470638, {0x0?, 0x0?})
2024-04-30T13:43:32.457363781Z 	/home/runner/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.3/retry.go:55 +0x118
2024-04-30T13:43:32.457433628Z github.com/cenkalti/backoff/v4.RetryNotify(...)
2024-04-30T13:43:32.457482434Z 	/home/runner/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.3/retry.go:34
2024-04-30T13:43:32.457530752Z github.com/netbirdio/netbird/client/cmd.WithBackOff(0x4b76ec?)
2024-04-30T13:43:32.457578556Z 	/home/runner/work/netbird/netbird/client/cmd/root.go:213 +0x2d
2024-04-30T13:43:32.457626540Z github.com/netbirdio/netbird/client/cmd.foregroundLogin({0x15df9f0?, 0xc0007f08a0}, 0xc0007ca8f0?, 0xc0007ca8f0, {0xc00004800d, 0x24})
2024-04-30T13:43:32.457676811Z 	/home/runner/work/netbird/netbird/client/cmd/login.go:136 +0x98
2024-04-30T13:43:32.457743025Z github.com/netbirdio/netbird/client/cmd.runInForegroundMode({0x15df9f0, 0xc0007f08a0}, 0x1131580?)
2024-04-30T13:43:32.457795865Z 	/home/runner/work/netbird/netbird/client/cmd/up.go:142 +0x42a
2024-04-30T13:43:32.457847561Z github.com/netbirdio/netbird/client/cmd.upFunc(0x1fb49e0, {0x13cb066?, 0x4?, 0x13cb06a?})
2024-04-30T13:43:32.457896384Z 	/home/runner/work/netbird/netbird/client/cmd/up.go:70 +0x1cd
2024-04-30T13:43:32.457942432Z github.com/spf13/cobra.(*Command).execute(0x1fb49e0, {0x2a48240, 0x0, 0x0})
2024-04-30T13:43:32.458017189Z 	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x87c
2024-04-30T13:43:32.458071316Z github.com/spf13/cobra.(*Command).ExecuteC(0x1fb2760)
2024-04-30T13:43:32.458115385Z 	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5
2024-04-30T13:43:32.458165728Z github.com/spf13/cobra.(*Command).Execute(...)
2024-04-30T13:43:32.458207491Z 	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
2024-04-30T13:43:32.458252960Z github.com/netbirdio/netbird/client/cmd.Execute(...)

Expected behavior

To connect wit the server

Are you using NetBird Cloud?

No

NetBird version

I run the latest version using the install script i cant find any place where i can spot the version number.

NetBird status -d output:

If applicable, add the `netbird status -d' command output.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

Originally created by @zzecool on GitHub (Apr 30, 2024). **Describe the problem** A clear and concise description of what the problem is. **To Reproduce** Steps to reproduce the behavior: Trying to deply docker client, Container fails instantly and shuts down. Logs are the following : ``` 2024-04-30T13:43:32.453860580Z 2024-04-30T13:43:32Z ERRO client/internal/login.go:105: failed while getting Management Service public key: rpc error: code = PermissionDenied desc = unexpected HTTP status code received from server: 403 (Forbidden); transport: received unexpected content-type "text/html" 2024-04-30T13:43:32.456290315Z panic: runtime error: invalid memory address or nil pointer dereference 2024-04-30T13:43:32.456476020Z [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf34db6] 2024-04-30T13:43:32.456559316Z 2024-04-30T13:43:32.456598309Z goroutine 1 [running]: 2024-04-30T13:43:32.456759022Z github.com/netbirdio/netbird/client/internal.Login({0x15df9f0, 0xc0007f08a0}, 0xc0007ca8f0, {0x0, 0x0}, {0x0, 0x0}) 2024-04-30T13:43:32.456850029Z /home/runner/work/netbird/netbird/client/internal/login.go:73 +0x236 2024-04-30T13:43:32.456905564Z github.com/netbirdio/netbird/client/cmd.foregroundLogin.func1() 2024-04-30T13:43:32.456957623Z /home/runner/work/netbird/netbird/client/cmd/login.go:137 +0x36 2024-04-30T13:43:32.457239463Z github.com/cenkalti/backoff/v4.RetryNotifyWithTimer(0xc000c3fa28, {0x15d38a0, 0x1fadd40}, 0x1470638, {0x0?, 0x0?}) 2024-04-30T13:43:32.457363781Z /home/runner/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.3/retry.go:55 +0x118 2024-04-30T13:43:32.457433628Z github.com/cenkalti/backoff/v4.RetryNotify(...) 2024-04-30T13:43:32.457482434Z /home/runner/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.3/retry.go:34 2024-04-30T13:43:32.457530752Z github.com/netbirdio/netbird/client/cmd.WithBackOff(0x4b76ec?) 2024-04-30T13:43:32.457578556Z /home/runner/work/netbird/netbird/client/cmd/root.go:213 +0x2d 2024-04-30T13:43:32.457626540Z github.com/netbirdio/netbird/client/cmd.foregroundLogin({0x15df9f0?, 0xc0007f08a0}, 0xc0007ca8f0?, 0xc0007ca8f0, {0xc00004800d, 0x24}) 2024-04-30T13:43:32.457676811Z /home/runner/work/netbird/netbird/client/cmd/login.go:136 +0x98 2024-04-30T13:43:32.457743025Z github.com/netbirdio/netbird/client/cmd.runInForegroundMode({0x15df9f0, 0xc0007f08a0}, 0x1131580?) 2024-04-30T13:43:32.457795865Z /home/runner/work/netbird/netbird/client/cmd/up.go:142 +0x42a 2024-04-30T13:43:32.457847561Z github.com/netbirdio/netbird/client/cmd.upFunc(0x1fb49e0, {0x13cb066?, 0x4?, 0x13cb06a?}) 2024-04-30T13:43:32.457896384Z /home/runner/work/netbird/netbird/client/cmd/up.go:70 +0x1cd 2024-04-30T13:43:32.457942432Z github.com/spf13/cobra.(*Command).execute(0x1fb49e0, {0x2a48240, 0x0, 0x0}) 2024-04-30T13:43:32.458017189Z /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x87c 2024-04-30T13:43:32.458071316Z github.com/spf13/cobra.(*Command).ExecuteC(0x1fb2760) 2024-04-30T13:43:32.458115385Z /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 2024-04-30T13:43:32.458165728Z github.com/spf13/cobra.(*Command).Execute(...) 2024-04-30T13:43:32.458207491Z /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 2024-04-30T13:43:32.458252960Z github.com/netbirdio/netbird/client/cmd.Execute(...) ``` **Expected behavior** To connect wit the server **Are you using NetBird Cloud?** No **NetBird version** I run the latest version using the install script i cant find any place where i can spot the version number. **NetBird status -d output:** If applicable, add the `netbird status -d' command output. **Screenshots** If applicable, add screenshots to help explain your problem. **Additional context** Add any other context about the problem here.
saavagebueno added the serverzitadelself-hosting labels 2025-11-20 05:18:19 -05:00
Author
Owner

@zzecool commented on GitHub (Apr 30, 2024):

I also tried to connect using android,

It also fails with "invalid server address" but it the same https:// address that im using to login and setup peers - keys.

TCP ports 80, 443, 33073 and 10000; and UDP ports: 3478, 49152-65535 are open.

This is a curl on the server address:

StatusCode        : 200                                                                                                                                                                                              StatusDescription : OK                                                                                                                                                                                               Content           : <!DOCTYPE html><html id="__next_error__"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="preload" as="s
                    cript" fetchPriority="low" href="/_n...
RawContent        : HTTP/1.1 200 OK
                    Transfer-Encoding: chunked
                    Connection: keep-alive
                    alt-svc: h3=":443"; ma=86400
                    referrer-policy: strict-origin-when-cross-origin
                    strict-transport-security: max-age=15552000; includ...
Forms             : {}
Headers           : {[Transfer-Encoding, chunked], [Connection, keep-alive], [alt-svc, h3=":443"; ma=86400], [referrer-policy, strict-origin-when-cross-origin]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 4754

dead end.

@zzecool commented on GitHub (Apr 30, 2024): I also tried to connect using android, It also fails with "invalid server address" but it the same https:// address that im using to login and setup peers - keys. TCP ports 80, 443, 33073 and 10000; and UDP ports: 3478, 49152-65535 are open. This is a curl on the server address: ``` StatusCode : 200 StatusDescription : OK Content : <!DOCTYPE html><html id="__next_error__"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="preload" as="s cript" fetchPriority="low" href="/_n... RawContent : HTTP/1.1 200 OK Transfer-Encoding: chunked Connection: keep-alive alt-svc: h3=":443"; ma=86400 referrer-policy: strict-origin-when-cross-origin strict-transport-security: max-age=15552000; includ... Forms : {} Headers : {[Transfer-Encoding, chunked], [Connection, keep-alive], [alt-svc, h3=":443"; ma=86400], [referrer-policy, strict-origin-when-cross-origin]...} Images : {} InputFields : {} Links : {} ParsedHtml : mshtml.HTMLDocumentClass RawContentLength : 4754 ``` dead end.
Author
Owner

@mlsmaycon commented on GitHub (Apr 30, 2024):

Hello, @zzecool, the log with received unexpected content-type "text/html" indicates a proxy configuration issue. Can you confirm if you are running the NetBird host behind a reverse proxy? If so, can you share the configuration guide you used?

@mlsmaycon commented on GitHub (Apr 30, 2024): Hello, @zzecool, the log with `received unexpected content-type "text/html"` indicates a proxy configuration issue. Can you confirm if you are running the NetBird host behind a reverse proxy? If so, can you share the configuration guide you used?
Author
Owner

@zzecool commented on GitHub (Apr 30, 2024):

@mlsmaycon

Hello, @zzecool, the log with received unexpected content-type "text/html" indicates a proxy configuration issue. Can you confirm if you are running the NetBird host behind a reverse proxy? If so, can you share the configuration guide you used?

I just used the script on a fresh ubuntu vps nothing fancy. This is the only thing that is running on the vps :

image

@zzecool commented on GitHub (Apr 30, 2024): @mlsmaycon > Hello, @zzecool, the log with `received unexpected content-type "text/html"` indicates a proxy configuration issue. Can you confirm if you are running the NetBird host behind a reverse proxy? If so, can you share the configuration guide you used? I just used the script on a fresh ubuntu vps nothing fancy. This is the only thing that is running on the vps : ![image](https://github.com/netbirdio/netbird/assets/2816897/175c2930-c6a5-4137-beb5-835ab53f316c)
Author
Owner

@mlsmaycon commented on GitHub (Apr 30, 2024):

If this is a new install something may have failed. Can you get the logs with:

docker compose logs > /tmp/netbird.logs 2>&1

Then you can retry the installation again( the script will guide you) and see if that works, after that please share the logs of the failed install with us.

@mlsmaycon commented on GitHub (Apr 30, 2024): If this is a new install something may have failed. Can you get the logs with: ``` docker compose logs > /tmp/netbird.logs 2>&1 ``` Then you can retry the installation again( the script will guide you) and see if that works, after that please share the logs of the failed install with us.
Author
Owner

@zzecool commented on GitHub (Apr 30, 2024):

If this is a new install something may have failed. Can you get the logs with:

docker compose logs > /tmp/netbird.logs 2>&1

Then you can retry the installation again( the script will guide you) and see if that works, after that please share the logs of the failed install with us.

Are you sure about the command ? This just exports :
ubuntu@synovps:/tmp$ cat netbird.logs
no configuration file provided: not found

@zzecool commented on GitHub (Apr 30, 2024): > If this is a new install something may have failed. Can you get the logs with: > > ``` > docker compose logs > /tmp/netbird.logs 2>&1 > ``` > > Then you can retry the installation again( the script will guide you) and see if that works, after that please share the logs of the failed install with us. Are you sure about the command ? This just exports : ubuntu@synovps:/tmp$ cat netbird.logs no configuration file provided: not found
Author
Owner

@mlsmaycon commented on GitHub (Apr 30, 2024):

You need to run this from the same directory where you ran the script the first time. You should see a docker-compose.yml file there

@mlsmaycon commented on GitHub (Apr 30, 2024): You need to run this from the same directory where you ran the script the first time. You should see a docker-compose.yml file there
Author
Owner

@zzecool commented on GitHub (Apr 30, 2024):

You need to run this from the same directory where you ran the script the first time. You should see a docker-compose.yml file there

Well @mlsmaycon thank you very much.

We got something now.

It is full of Caddy errors about the

api.domain.com
broker.domain.com
and
*.domain.com

*where domain my actuall domain

So the problem is that the subdomain.domain used in the install script is not enough and you either have to manual DNS every subdomain ( User need to be informed about the subdomains that will be used ) or add a wildcard DNS for *.subdomain.domain

@zzecool commented on GitHub (Apr 30, 2024): > You need to run this from the same directory where you ran the script the first time. You should see a docker-compose.yml file there Well @mlsmaycon thank you very much. We got something now. It is full of Caddy errors about the api.domain.com broker.domain.com and *.domain.com *where domain my actuall domain So the problem is that the subdomain.domain used in the install script is not enough and you either have to manual DNS every subdomain ( User need to be informed about the subdomains that will be used ) or add a wildcard DNS for *.subdomain.domain
Author
Owner

@mlsmaycon commented on GitHub (Apr 30, 2024):

Yeah, that's the expectation. The domain you set with NETBIRD_DOMAIN needs to point to the server. After the install is complete your peers would use this domain to connect with --management-url https//<domain>

In case of docker you can set it with the environment variable NB_MANAGEMENT_URL=https<domain>

With that said the bug with container is valid and will be fixed in the next release

@mlsmaycon commented on GitHub (Apr 30, 2024): Yeah, that's the expectation. The domain you set with NETBIRD_DOMAIN needs to point to the server. After the install is complete your peers would use this domain to connect with `--management-url https//<domain>` In case of docker you can set it with the environment variable `NB_MANAGEMENT_URL=https<domain>` With that said the bug with container is valid and will be fixed in the next release
Author
Owner

@zzecool commented on GitHub (Apr 30, 2024):

@mlsmaycon

Nothing worked

1: I installed it from scratch 5 times, *.mydomain.com is forwared to the server. Every time removed everything containers volumes etc and files.
2: I can login make keys etc just fine.
3: Ports are forwarded.
4: Docker client fails, android client fails.

This is the caddy config that gets created :

image

Does it looks ok ?

this is from inside Caddy

image

Please tell me what links should caddy reply to ? So i can test from outside.
Help me identify this.

Or you can spinup a vps and check it please.

@zzecool commented on GitHub (Apr 30, 2024): @mlsmaycon Nothing worked 1: I installed it from scratch 5 times, *.mydomain.com is forwared to the server. Every time removed everything containers volumes etc and files. 2: I can login make keys etc just fine. 3: Ports are forwarded. 4: Docker client fails, android client fails. This is the caddy config that gets created : ![image](https://github.com/netbirdio/netbird/assets/2816897/598f1981-42f8-4253-b967-1b4084c6f614) Does it looks ok ? this is from inside Caddy ![image](https://github.com/netbirdio/netbird/assets/2816897/5d325d99-4065-46b5-9c29-89f837dbd8a3) Please tell me what links should caddy reply to ? So i can test from outside. Help me identify this. Or you can spinup a vps and check it please.
Author
Owner

@mlsmaycon commented on GitHub (May 1, 2024):

Hello @zzecool the only domain to be used with this setup is the one you provide to the script.

When it fails it is usually a certificate problem or a new bug.

Can you confirm if you can access the netbird dashboard?

Also, can you share the logs for when you try to access via client?

@mlsmaycon commented on GitHub (May 1, 2024): Hello @zzecool the only domain to be used with this setup is the one you provide to the script. When it fails it is usually a certificate problem or a new bug. Can you confirm if you can access the netbird dashboard? Also, can you share the logs for when you try to access via client?
Author
Owner

@zzecool commented on GitHub (May 1, 2024):

@mlsmaycon

Yes i have access to the netbird dashboard
Yes i have access to the zitadel dashboard

The logs are shared on the first post. Its always the same.

I tried on other distro
I tried on other vps

It just fails with every client trying to join. For real it will be much faster if you just run the script and verify the behaviour on a random vps.

That can't be only me.

@zzecool commented on GitHub (May 1, 2024): @mlsmaycon Yes i have access to the netbird dashboard Yes i have access to the zitadel dashboard The logs are shared on the first post. Its always the same. I tried on other distro I tried on other vps It just fails with every client trying to join. For real it will be much faster if you just run the script and verify the behaviour on a random vps. That can't be only me.
Author
Owner

@zzecool commented on GitHub (May 1, 2024):

@mlsmaycon

.............

Impossible but possible.

Problem is cloudflare, if you disable the proxy on the DNS it works, i dont know why.

  • note that even though i had it proxied i had also developer mode on so there was no caching.
    Any ideas?
@zzecool commented on GitHub (May 1, 2024): @mlsmaycon ............. Impossible but possible. Problem is cloudflare, if you disable the proxy on the DNS it works, i dont know why. * note that even though i had it proxied i had also developer mode on so there was no caching. Any ideas?
Author
Owner

@mlsmaycon commented on GitHub (May 1, 2024):

The management and signal servers will be using grpc, so you would need to configure cloudflare to handle grpc and http2, but the coturn server will be using UDP with dynamic UDP ports for allocation which also gets affected by cloudflare's proxy

@mlsmaycon commented on GitHub (May 1, 2024): The management and signal servers will be using grpc, so you would need to configure cloudflare to handle grpc and http2, but the coturn server will be using UDP with dynamic UDP ports for allocation which also gets affected by cloudflare's proxy
Author
Owner

@zzecool commented on GitHub (May 1, 2024):

Thank you very much,

UDP is not supported in cloudflare. So there is no way i can think of.

Lets hope this will help other mofos like me in the future.

@zzecool commented on GitHub (May 1, 2024): Thank you very much, UDP is not supported in cloudflare. So there is no way i can think of. Lets hope this will help other mofos like me in the future.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#833