[GH-ISSUE #4591] Upgrade of signal server (docker) from v.0.58.2 to v0.59.2 leads to no TLS #9471

Open
opened 2026-08-05 01:22:16 -04:00 by saavagebueno · 5 comments
Owner

Originally created by @glaz123 on GitHub (Oct 6, 2025).
Original GitHub issue: https://github.com/netbirdio/netbird/issues/4591

Describe the problem

After upgrade docker release of signal service from v.0.58.2 to higher version (0.59 and higher) TLS is switched off and HTTP websock proxy is running on 443 port.

To Reproduce

first deploy container with following config

x-default: &default
  restart: 'unless-stopped'
  logging:
    driver: 'json-file'
    options:
      max-size: '500m'
      max-file: '2'
services:
  # Signal
  signal:
    <<: *default
    command: ["--cert-file","/etc/cert.pem","--cert-key","/etc/cert.key.pem","--log-file","console"]
    image: netbirdio/signal:0.58.2
    environment:
      - NB_CERTFILE=/etc/cert.pem
      - NB_CERTKEY=/etc/cert.key.pem
    volumes:
      - netbird-signal:/var/lib/netbird
      - /home/someuser/certs/somewhere.outside/fullchain.pem:/etc/cert.pem:ro
      - /home/someuser/certs/somewhere.outside/key.pem:/etc/cert.key.pem:ro
    ports:
      - 8080:443

got following log on start everything works as expected.

2025-10-03T14:05:17Z INFO signal/cmd/run.go:217: setting up TLS with custom certificates.
2025-10-03T14:05:17Z INFO signal/cmd/run.go:104: running metrics server: :9090/metrics
2025-10-03T14:05:17Z INFO signal/cmd/run.go:132: running gRPC server: [::]:443
2025-10-03T14:05:17Z INFO signal/cmd/run.go:142: running gRPC backward compatibility server: [::]:10000
2025-10-03T14:05:17Z INFO signal/cmd/run.go:145: signal server version 0.58.2
2025-10-03T14:05:17Z INFO signal/cmd/run.go:146: started Signal Service

quick test with curl shows that TLS is running

root@host01:~# curl https://signal.netbird:8080 -v
*   Trying 1.2.3.4:8080...
* Connected to signal.netbird (1.2.3.4) port 8080 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=netbird
*  start date: Sep  7 20:04:09 2025 GMT
*  expire date: Dec  6 20:04:08 2025 GMT
*  subjectAltName: host "signal.netbird" matched cert's "*.netbird"
*  issuer: C=US; O=Let's Encrypt; CN=R13
*  SSL certificate verify ok.
* using HTTP/2
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: signal.netbird:8080]
* h2h3 [user-agent: curl/7.88.1]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x5640ead312b0)
> GET / HTTP/2
> Host: signal.netbird:8080
> user-agent: curl/7.88.1
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 415
< content-type: application/grpc
< grpc-status: 3
< grpc-message: invalid gRPC request content-type ""
<
* Connection #0 to host signal.netbird left intact

change version of service image as followed (tested with version 0.59.1, 0.59.2, 0.59.4)
image: netbirdio/signal:latest
redeploy container with new version and got following log on start. No TLS Server only plain HTTP.

2025-10-03T14:06:05Z INFO signal/cmd/run.go:234: setting up TLS with custom certificates. 
2025-10-03T14:06:05Z INFO signal/cmd/run.go:110: running metrics server: :9090/metrics
2025-10-03T14:06:05Z INFO signal/cmd/run.go:140: running HTTP server with WebSocket proxy (no TLS): [::]:443
2025-10-03T14:06:05Z INFO signal/cmd/run.go:159: running gRPC backward compatibility server: [::]:10000
2025-10-03T14:06:05Z INFO signal/cmd/run.go:162: signal server version 0.59.2
2025-10-03T14:06:05Z INFO signal/cmd/run.go:163: started Signal Service

quick test with curl; shows that no TLS is running

root@host01:~# curl https://signal.netbird:8080 -v
*   Trying 1.2.3.4:8080...
* Connected to tail.rtall.ru (1.2.3.4) port 8080 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* OpenSSL/3.0.17: error:0A00010B:SSL routines::wrong version number
* Closing connection 0
curl: (35) OpenSSL/3.0.17: error:0A00010B:SSL routines::wrong version number

Expected behavior

working gRPC server with custom TLS certificates and TLS.

Are you using NetBird Cloud?

self-host NetBird's control plane.

NetBird version

tested every version starting from 0.59. For the moment runs infrastructure with management plane at 0.59.2 and signal service with 0.58.2.

Is any other VPN software installed?

none

Debug output

No netbird clients running on management plane.

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • Disabled other VPN software
  • Checked firewall settings
Originally created by @glaz123 on GitHub (Oct 6, 2025). Original GitHub issue: https://github.com/netbirdio/netbird/issues/4591 **Describe the problem** After upgrade docker release of signal service from v.0.58.2 to higher version (0.59 and higher) TLS is switched off and HTTP websock proxy is running on 443 port. **To Reproduce** first deploy container with following config ``` x-default: &default restart: 'unless-stopped' logging: driver: 'json-file' options: max-size: '500m' max-file: '2' services: # Signal signal: <<: *default command: ["--cert-file","/etc/cert.pem","--cert-key","/etc/cert.key.pem","--log-file","console"] image: netbirdio/signal:0.58.2 environment: - NB_CERTFILE=/etc/cert.pem - NB_CERTKEY=/etc/cert.key.pem volumes: - netbird-signal:/var/lib/netbird - /home/someuser/certs/somewhere.outside/fullchain.pem:/etc/cert.pem:ro - /home/someuser/certs/somewhere.outside/key.pem:/etc/cert.key.pem:ro ports: - 8080:443 ``` got following log on start everything works as expected. ``` 2025-10-03T14:05:17Z INFO signal/cmd/run.go:217: setting up TLS with custom certificates. 2025-10-03T14:05:17Z INFO signal/cmd/run.go:104: running metrics server: :9090/metrics 2025-10-03T14:05:17Z INFO signal/cmd/run.go:132: running gRPC server: [::]:443 2025-10-03T14:05:17Z INFO signal/cmd/run.go:142: running gRPC backward compatibility server: [::]:10000 2025-10-03T14:05:17Z INFO signal/cmd/run.go:145: signal server version 0.58.2 2025-10-03T14:05:17Z INFO signal/cmd/run.go:146: started Signal Service ``` quick test with curl shows that TLS is running ``` root@host01:~# curl https://signal.netbird:8080 -v * Trying 1.2.3.4:8080... * Connected to signal.netbird (1.2.3.4) port 8080 (#0) * ALPN: offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: /etc/ssl/certs * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 * ALPN: server accepted h2 * Server certificate: * subject: CN=netbird * start date: Sep 7 20:04:09 2025 GMT * expire date: Dec 6 20:04:08 2025 GMT * subjectAltName: host "signal.netbird" matched cert's "*.netbird" * issuer: C=US; O=Let's Encrypt; CN=R13 * SSL certificate verify ok. * using HTTP/2 * h2h3 [:method: GET] * h2h3 [:path: /] * h2h3 [:scheme: https] * h2h3 [:authority: signal.netbird:8080] * h2h3 [user-agent: curl/7.88.1] * h2h3 [accept: */*] * Using Stream ID: 1 (easy handle 0x5640ead312b0) > GET / HTTP/2 > Host: signal.netbird:8080 > user-agent: curl/7.88.1 > accept: */* > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): < HTTP/2 415 < content-type: application/grpc < grpc-status: 3 < grpc-message: invalid gRPC request content-type "" < * Connection #0 to host signal.netbird left intact ``` change version of service image as followed (tested with version 0.59.1, 0.59.2, 0.59.4) `image: netbirdio/signal:latest` redeploy container with new version and got following log on start. No TLS Server only plain HTTP. ``` 2025-10-03T14:06:05Z INFO signal/cmd/run.go:234: setting up TLS with custom certificates. 2025-10-03T14:06:05Z INFO signal/cmd/run.go:110: running metrics server: :9090/metrics 2025-10-03T14:06:05Z INFO signal/cmd/run.go:140: running HTTP server with WebSocket proxy (no TLS): [::]:443 2025-10-03T14:06:05Z INFO signal/cmd/run.go:159: running gRPC backward compatibility server: [::]:10000 2025-10-03T14:06:05Z INFO signal/cmd/run.go:162: signal server version 0.59.2 2025-10-03T14:06:05Z INFO signal/cmd/run.go:163: started Signal Service ``` quick test with curl; shows that no TLS is running ``` root@host01:~# curl https://signal.netbird:8080 -v * Trying 1.2.3.4:8080... * Connected to tail.rtall.ru (1.2.3.4) port 8080 (#0) * ALPN: offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: /etc/ssl/certs * OpenSSL/3.0.17: error:0A00010B:SSL routines::wrong version number * Closing connection 0 curl: (35) OpenSSL/3.0.17: error:0A00010B:SSL routines::wrong version number ``` **Expected behavior** working gRPC server with custom TLS certificates and TLS. **Are you using NetBird Cloud?** self-host NetBird's control plane. **NetBird version** tested every version starting from `0.59`. For the moment runs infrastructure with management plane at 0.59.2 and signal service with 0.58.2. **Is any other VPN software installed?** none **Debug output** No netbird clients running on management plane. **Have you tried these troubleshooting steps?** - [ ] Reviewed [client troubleshooting](https://docs.netbird.io/how-to/troubleshooting-client) (if applicable) - [X] Checked for newer NetBird versions - [X] Searched for similar issues on GitHub (including closed ones) - [ ] Restarted the NetBird client - [ ] Disabled other VPN software - [ ] Checked firewall settings
saavagebueno added the triage-needed label 2026-08-05 01:22:16 -04:00
Author
Owner

@flotpg commented on GitHub (Oct 13, 2025):

Same here. Wanted to use the new browser rdp/ssh feature and tried to enable TLS for signal (was unencrypted before): https://github.com/netbirdio/netbird/issues/4580#issuecomment-3394049817

signal-1 | 2025-10-12T07:55:15Z INFO signal/cmd/run.go:235: setting up TLS with custom certificates.
signal-1 | 2025-10-12T07:55:16Z INFO signal/cmd/run.go:141: running HTTP server with WebSocket proxy (no TLS): [::]:443

signal:
    depends_on:
     - dashboard
    image: netbirdio/signal:latest
    restart: unless-stopped
    volumes:
      - netbird-signal:/var/lib/netbird
      - netbird-letsencrypt:/etc/letsencrypt:ro
    ports:
      - 10000:80
  #      # port and command for Let's Encrypt validation
  #      - 443:443
    command: ["--cert-file", "/etc/letsencrypt/live/my-domain.com/fullchain.pem",  "--cert-key", "/etc/letsencrypt/live/my-domain.com/privkey.pem","--log-file", "console"]

management.json:
   "Signal": {
        "Proto": "https",
        "URI": "my-domain.com:10000",
        "Username": "",
        "Password": ""
    },
<!-- gh-comment-id:3397531410 --> @flotpg commented on GitHub (Oct 13, 2025): Same here. Wanted to use the new browser rdp/ssh feature and tried to enable TLS for signal (was unencrypted before): https://github.com/netbirdio/netbird/issues/4580#issuecomment-3394049817 signal-1 | 2025-10-12T07:55:15Z INFO signal/cmd/run.go:235: setting up TLS with custom certificates. signal-1 | 2025-10-12T07:55:16Z INFO signal/cmd/run.go:141: running HTTP server with WebSocket proxy **(no TLS)**: [::]:443 ``` signal: depends_on: - dashboard image: netbirdio/signal:latest restart: unless-stopped volumes: - netbird-signal:/var/lib/netbird - netbird-letsencrypt:/etc/letsencrypt:ro ports: - 10000:80 # # port and command for Let's Encrypt validation # - 443:443 command: ["--cert-file", "/etc/letsencrypt/live/my-domain.com/fullchain.pem", "--cert-key", "/etc/letsencrypt/live/my-domain.com/privkey.pem","--log-file", "console"] ``` ``` management.json: "Signal": { "Proto": "https", "URI": "my-domain.com:10000", "Username": "", "Password": "" }, ```
Author
Owner

@glaz123 commented on GitHub (Oct 15, 2025):

Well, I figure out the core of the problem. Previously before commit b5daec3 (Oct 1) in file netbird/signal/cmd/run.go the grpc server was started on port 443 with TLS when no let's encrypt is configured and signalPort is 443. But after that commit, the logic for starting grpc server with TLS when no LE and 443 port configured was ommited. And grpc server with TLS is only started on legacyGrpcPort. As a confirmation I change docker-compose.yml to point to port 10000 instead of 443 (8080:443 -> 8080:10000), and after recreate container to latest version (0.59.6) everything is back to work with correct TLS config and certificates.

<!-- gh-comment-id:3406606673 --> @glaz123 commented on GitHub (Oct 15, 2025): Well, I figure out the core of the problem. Previously before commit b5daec3 (Oct 1) in file netbird/signal/cmd/run.go the grpc server was started on port 443 with TLS when no let's encrypt is configured and signalPort is 443. But after that commit, the logic for starting grpc server with TLS when no LE and 443 port configured was ommited. And grpc server with TLS is only started on legacyGrpcPort. As a confirmation I change docker-compose.yml to point to port 10000 instead of 443 (8080:443 -> 8080:10000), and after recreate container to latest version (0.59.6) everything is back to work with correct TLS config and certificates.
Author
Owner

@flotpg commented on GitHub (Oct 15, 2025):

thanks @glaz123
can you pls post your compose section for signal and ports? I'm unsure how it should look like and want to make sure ;)
thanks a lot.

<!-- gh-comment-id:3406745583 --> @flotpg commented on GitHub (Oct 15, 2025): thanks @glaz123 can you pls post your compose section for signal and ports? I'm unsure how it should look like and want to make sure ;) thanks a lot.
Author
Owner

@glaz123 commented on GitHub (Oct 16, 2025):

The final version is look like this

x-default: &default
  restart: 'unless-stopped'
  logging:
    driver: 'json-file'
    options:
      max-size: '500m'
      max-file: '2'
services:
  # Signal
  signal:
    <<: *default
    command: ["--cert-file","/etc/cert.pem","--cert-key","/etc/cert.key.pem","--log-file","console"]
    image: netbirdio/signal:0.58.2
    environment:
      - NB_CERTFILE=/etc/cert.pem
      - NB_CERTKEY=/etc/cert.key.pem
    volumes:
      - netbird-signal:/var/lib/netbird
      - /home/someuser/certs/somewhere.outside/fullchain.pem:/etc/cert.pem:ro
      - /home/someuser/certs/somewhere.outside/key.pem:/etc/cert.key.pem:ro
    ports:
      - 8080:10000
<!-- gh-comment-id:3409436496 --> @glaz123 commented on GitHub (Oct 16, 2025): The final version is look like this ``` x-default: &default restart: 'unless-stopped' logging: driver: 'json-file' options: max-size: '500m' max-file: '2' services: # Signal signal: <<: *default command: ["--cert-file","/etc/cert.pem","--cert-key","/etc/cert.key.pem","--log-file","console"] image: netbirdio/signal:0.58.2 environment: - NB_CERTFILE=/etc/cert.pem - NB_CERTKEY=/etc/cert.key.pem volumes: - netbird-signal:/var/lib/netbird - /home/someuser/certs/somewhere.outside/fullchain.pem:/etc/cert.pem:ro - /home/someuser/certs/somewhere.outside/key.pem:/etc/cert.key.pem:ro ports: - 8080:10000 ```
Author
Owner

@flotpg commented on GitHub (Oct 16, 2025):

Got it, so in my case:

 signal:
    depends_on:
     - dashboard
    image: netbirdio/signal:latest
    restart: unless-stopped
    volumes:
      - netbird-signal:/var/lib/netbird
      - netbird-letsencrypt:/etc/letsencrypt:ro
    ports:
      #- 10000:80
      - 10000:10000
      #- 10000:443
  #      # port and command for Let's Encrypt validation
  #      - 443:443
    command: ["--cert-file", "/etc/letsencrypt/live/netbird.domain.com/fullchain.pem",  "--cert-key", "/etc/letsencrypt/live/netbird.domain.com/privkey.pem","--log-file", "console"]

  • management.json proto = https
    "Signal": {
        "Proto": "https",
        "URI": "netbird.domain.com:10000",
        "Username": "",
        "Password": ""
    },
<!-- gh-comment-id:3410285117 --> @flotpg commented on GitHub (Oct 16, 2025): Got it, so in my case: ``` signal: depends_on: - dashboard image: netbirdio/signal:latest restart: unless-stopped volumes: - netbird-signal:/var/lib/netbird - netbird-letsencrypt:/etc/letsencrypt:ro ports: #- 10000:80 - 10000:10000 #- 10000:443 # # port and command for Let's Encrypt validation # - 443:443 command: ["--cert-file", "/etc/letsencrypt/live/netbird.domain.com/fullchain.pem", "--cert-key", "/etc/letsencrypt/live/netbird.domain.com/privkey.pem","--log-file", "console"] ``` + management.json proto = https ``` "Signal": { "Proto": "https", "URI": "netbird.domain.com:10000", "Username": "", "Password": "" }, ```
Sign in to join this conversation.
No Label triage-needed
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#9471