SVI/netbird
1
0
Fork 0
mirror of https://github.com/netbirdio/netbird.git synced 2026-07-29 17:32:35 -04:00
Code Issues 3.0k Packages Projects Releases 10 Wiki Activity

V0.27.3 No access to dashboard #796

New Issue
Closed
opened 2025-11-20 05:17:42 -05:00 by saavagebueno · 5 comments
saavagebueno commented 2025-11-20 05:17:42 -05:00
Owner
Copy Link

Originally created by @xcstatus on GitHub (Apr 13, 2024).

Describe the problem

No access to dashboard。crdb, management have been restarting

crdb-1 | CockroachDB node starting at 2024-04-13 10:52:00.377480896 +0000 UTC m=+38.865661448 (took 35.8s)
crdb-1 | build: CCL v22.2.2 @ 2023/01/04 17:50:17 (go1.19.1)
crdb-1 | webui: https://crdb:8080
crdb-1 | sql: postgresql://root@crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt
crdb-1 | sql (JDBC): jdbc:postgresql://crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt&user=root
crdb-1 | RPC client flags: /cockroach/cockroach --host=crdb:26257 --certs-dir=certs
crdb-1 | logs: /cockroach/cockroach-data/logs
crdb-1 | temp dir: /cockroach/cockroach-data/cockroach-temp1272495510
crdb-1 | external I/O path: /cockroach/cockroach-data/extern
crdb-1 | store[0]: path=/cockroach/cockroach-data
crdb-1 | storage engine: pebble
crdb-1 | clusterID: ba60a6f0-9de2-4c8a-9414-258815587aa3
crdb-1 | status: restarted pre-existing node
crdb-1 | nodeID: 1
crdb-1 | *
crdb-1 | * ERROR: a panic has occurred!
crdb-1 | * pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781)
crdb-1 | * (1) attached stack trace
crdb-1 | * -- stack trace:
crdb-1 | * | runtime.gopanic
crdb-1 | * | GOROOT/src/runtime/panic.go:884
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).destroy
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:955
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).Close
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:312
crdb-1 | * | [...repeated from below...]
crdb-1 | * Wraps: (2) forced error mark
crdb-1 | * | "pebble: corruption"
crdb-1 | * | github.com/cockroachdb/errors/withstack/*withstack.withStack::
crdb-1 | * Wraps: (3) attached stack trace
crdb-1 | * -- stack trace:
crdb-1 | * | github.com/cockroachdb/pebble/internal/base.CorruptionErrorf
crdb-1 | * | github.com/cockroachdb/pebble/internal/base/external/com_github_cockroachdb_pebble/internal/base/error.go:27
crdb-1 | * | github.com/cockroachdb/pebble/sstable.checkChecksum
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2753
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*Reader).readBlock
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2817
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).readBlockWithStats
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:583
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).loadBlock
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:498
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).seekGEHelper
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:790
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:702
crdb-1 | * | github.com/cockroachdb/pebble.(*levelIter).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/level_iter.go:648
crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).seekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:908
crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:970
crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1144
crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1028
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).SeekGE
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:325
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).findTimeSeries
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/pruning.go:69
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).MaintainTimeSeries
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/maintenance.go:57
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*timeSeriesMaintenanceQueue).process
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/ts_maintenance_queue.go:154
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:997
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:91
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:956
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2.1
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:874
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489
crdb-1 | * | runtime.goexit
crdb-1 | * | src/runtime/asm_arm64.s:1172
crdb-1 | * Wraps: (4) pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781)
crdb-1 | * Error types: (1) *withstack.withStack (2) *markers.withMark (3) *withstack.withStack (4) *errutil.leafError
crdb-1 | *
crdb-1 | *
crdb-1 | * ERROR: Queued as error 145e0dceddd441f08f8e5c1fd9519bb4
crdb-1 | *
crdb-1 exited with code 0

management-1 | 2024-04-13T18:59:08+08:00 INFO management/server/telemetry/app_metrics.go:177: enabled application metrics and exposing on http://0.0.0.0:8081
management-1 | 2024-04-13T18:59:08+08:00 INFO management/server/store.go:92: using SQLite store engine
management-1 | 2024-04-13T18:59:10+08:00 INFO management/cmd/management.go:173: geo location service has been initialized from /var/lib/netbird/
management-1 | 2024-04-13T18:59:10+08:00 INFO management/server/account.go:885: single account mode enabled, accounts number 1
management-1 | 2024-04-13T18:59:10+08:00 WARN management/cmd/management.go:191: TrustedPeers are configured to default value '0.0.0.0/0', '::/0'. This allows connection IP spoofing.
management-1 | Error: failed creating JWT validator: invalid character 'u' looking for beginning of value
management-1 | 2024-04-13T18:59:11+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration
management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF
management-1 |
management-1 | 2024-04-13T18:59:13+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration
management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF
management-1 |
management-1 | 2024-04-13T18:59:15+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration
management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF
management-1 |
management-1 | 2024-04-13T18:59:17+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration
management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF
management-1 |
management-1 | 2024-04-13T18:59:20+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration
management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF
management-1 |
management-1 | 2024-04-13T18:59:25+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration
management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF
management-1 |
management-1 | 2024-04-13T18:59:32+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration

Originally created by @xcstatus on GitHub (Apr 13, 2024). **Describe the problem** No access to dashboard。crdb, management have been restarting crdb-1 | CockroachDB node starting at 2024-04-13 10:52:00.377480896 +0000 UTC m=+38.865661448 (took 35.8s) crdb-1 | build: CCL v22.2.2 @ 2023/01/04 17:50:17 (go1.19.1) crdb-1 | webui: https://crdb:8080 crdb-1 | sql: postgresql://root@crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt crdb-1 | sql (JDBC): jdbc:postgresql://crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt&user=root crdb-1 | RPC client flags: /cockroach/cockroach <client cmd> --host=crdb:26257 --certs-dir=certs crdb-1 | logs: /cockroach/cockroach-data/logs crdb-1 | temp dir: /cockroach/cockroach-data/cockroach-temp1272495510 crdb-1 | external I/O path: /cockroach/cockroach-data/extern crdb-1 | store[0]: path=/cockroach/cockroach-data crdb-1 | storage engine: pebble crdb-1 | clusterID: ba60a6f0-9de2-4c8a-9414-258815587aa3 crdb-1 | status: restarted pre-existing node crdb-1 | nodeID: 1 crdb-1 | * crdb-1 | * ERROR: a panic has occurred! crdb-1 | * pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781) crdb-1 | * (1) attached stack trace crdb-1 | * -- stack trace: crdb-1 | * | runtime.gopanic crdb-1 | * | GOROOT/src/runtime/panic.go:884 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).destroy crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:955 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).Close crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:312 crdb-1 | * | [...repeated from below...] crdb-1 | * Wraps: (2) forced error mark crdb-1 | * | "pebble: corruption" crdb-1 | * | github.com/cockroachdb/errors/withstack/*withstack.withStack:: crdb-1 | * Wraps: (3) attached stack trace crdb-1 | * -- stack trace: crdb-1 | * | github.com/cockroachdb/pebble/internal/base.CorruptionErrorf crdb-1 | * | github.com/cockroachdb/pebble/internal/base/external/com_github_cockroachdb_pebble/internal/base/error.go:27 crdb-1 | * | github.com/cockroachdb/pebble/sstable.checkChecksum crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2753 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*Reader).readBlock crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2817 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).readBlockWithStats crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:583 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).loadBlock crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:498 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).seekGEHelper crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:790 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:702 crdb-1 | * | github.com/cockroachdb/pebble.(*levelIter).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/level_iter.go:648 crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).seekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:908 crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:970 crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1144 crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1028 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).SeekGE crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:325 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).findTimeSeries crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/pruning.go:69 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).MaintainTimeSeries crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/maintenance.go:57 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*timeSeriesMaintenanceQueue).process crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/ts_maintenance_queue.go:154 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:997 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:91 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:956 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2.1 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:874 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489 crdb-1 | * | runtime.goexit crdb-1 | * | src/runtime/asm_arm64.s:1172 crdb-1 | * Wraps: (4) pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781) crdb-1 | * Error types: (1) *withstack.withStack (2) *markers.withMark (3) *withstack.withStack (4) *errutil.leafError crdb-1 | * crdb-1 | * crdb-1 | * ERROR: Queued as error 145e0dceddd441f08f8e5c1fd9519bb4 crdb-1 | * crdb-1 exited with code 0 management-1 | 2024-04-13T18:59:08+08:00 INFO management/server/telemetry/app_metrics.go:177: enabled application metrics and exposing on http://0.0.0.0:8081 management-1 | 2024-04-13T18:59:08+08:00 INFO management/server/store.go:92: using SQLite store engine management-1 | 2024-04-13T18:59:10+08:00 INFO management/cmd/management.go:173: geo location service has been initialized from /var/lib/netbird/ management-1 | 2024-04-13T18:59:10+08:00 INFO management/server/account.go:885: single account mode enabled, accounts number 1 management-1 | 2024-04-13T18:59:10+08:00 WARN management/cmd/management.go:191: TrustedPeers are configured to default value '0.0.0.0/0', '::/0'. This allows connection IP spoofing. management-1 | Error: failed creating JWT validator: invalid character 'u' looking for beginning of value management-1 | 2024-04-13T18:59:11+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF management-1 | management-1 | 2024-04-13T18:59:13+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF management-1 | management-1 | 2024-04-13T18:59:15+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF management-1 | management-1 | 2024-04-13T18:59:17+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF management-1 | management-1 | 2024-04-13T18:59:20+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF management-1 | management-1 | 2024-04-13T18:59:25+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration management-1 | Error: failed reading provided config file: /etc/netbird/management.json: OIDC configuration request returned status 404 with response: unexpected EOF management-1 | management-1 | 2024-04-13T18:59:32+08:00 INFO management/cmd/management.go:455: loading OIDC configuration from the provided IDP configuration endpoint https://my.domain.com/.well-known/openid-configuration
saavagebueno added the triage-needed label 2025-11-20 05:17:42 -05:00
saavagebueno commented 2025-11-20 05:17:43 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Apr 13, 2024):

Hello @xcstatus it seems like the cockroach database has exited. Can you try to restart it and share the logs for it?

@mlsmaycon commented on GitHub (Apr 13, 2024): Hello @xcstatus it seems like the cockroach database has exited. Can you try to restart it and share the logs for it?
saavagebueno commented 2025-11-20 05:17:43 -05:00
Author
Owner
Copy Link

@xcstatus commented on GitHub (Apr 13, 2024):

The cockroach keeps restarting

crdb-1 | *
crdb-1 | * WARNING: Running a server without --sql-addr, with a combined RPC/SQL listener, is deprecated.
crdb-1 | * This feature will be removed in the next version of CockroachDB.
crdb-1 | *
crdb-1 | *
crdb-1 | * WARNING: The server appears to be unable to contact the other nodes in the cluster. Please try:
crdb-1 | *
crdb-1 | * - starting the other nodes, if you haven't already;
crdb-1 | * - double-checking that the '--join' and '--listen'/'--advertise' flags are set up correctly;
crdb-1 | * - running the 'cockroach init' command if you are trying to initialize a new cluster.
crdb-1 | *
crdb-1 | * If problems persist, please see https://www.cockroachlabs.com/docs/v22.2/cluster-setup-troubleshooting.html.
crdb-1 | *
crdb-1 | CockroachDB node starting at 2024-04-13 12:28:48.607991136 +0000 UTC m=+45.771172393 (took 42.2s)
crdb-1 | build: CCL v22.2.2 @ 2023/01/04 17:50:17 (go1.19.1)
crdb-1 | webui: https://crdb:8080
crdb-1 | sql: postgresql://root@crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt
crdb-1 | sql (JDBC): jdbc:postgresql://crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt&user=root
crdb-1 | RPC client flags: /cockroach/cockroach --host=crdb:26257 --certs-dir=certs
crdb-1 | logs: /cockroach/cockroach-data/logs
crdb-1 | temp dir: /cockroach/cockroach-data/cockroach-temp3464751422
crdb-1 | external I/O path: /cockroach/cockroach-data/extern
crdb-1 | store[0]: path=/cockroach/cockroach-data
crdb-1 | storage engine: pebble
crdb-1 | clusterID: ba60a6f0-9de2-4c8a-9414-258815587aa3
crdb-1 | status: restarted pre-existing node
crdb-1 | nodeID: 1
crdb-1 | *
crdb-1 | * ERROR: a panic has occurred!
crdb-1 | * pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781)
crdb-1 | * (1) attached stack trace
crdb-1 | * -- stack trace:
crdb-1 | * | runtime.gopanic
crdb-1 | * | GOROOT/src/runtime/panic.go:884
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).destroy
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:955
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).Close
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:312
crdb-1 | * | [...repeated from below...]
crdb-1 | * Wraps: (2) forced error mark
crdb-1 | * | "pebble: corruption"
crdb-1 | * | github.com/cockroachdb/errors/withstack/*withstack.withStack::
crdb-1 | * Wraps: (3) attached stack trace
crdb-1 | * -- stack trace:
crdb-1 | * | github.com/cockroachdb/pebble/internal/base.CorruptionErrorf
crdb-1 | * | github.com/cockroachdb/pebble/internal/base/external/com_github_cockroachdb_pebble/internal/base/error.go:27
crdb-1 | * | github.com/cockroachdb/pebble/sstable.checkChecksum
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2753
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*Reader).readBlock
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2817
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).readBlockWithStats
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:583
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).loadBlock
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:498
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).seekGEHelper
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:790
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:702
crdb-1 | * | github.com/cockroachdb/pebble.(*levelIter).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/level_iter.go:648
crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).seekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:908
crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:970
crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1144
crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1028
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).SeekGE
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:325
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).findTimeSeries
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/pruning.go:69
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).MaintainTimeSeries
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/maintenance.go:57
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*timeSeriesMaintenanceQueue).process
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/ts_maintenance_queue.go:154
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:997
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:91
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:956
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2.1
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:874
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489
crdb-1 | * | runtime.goexit
crdb-1 | * | src/runtime/asm_arm64.s:1172
crdb-1 | * Wraps: (4) pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781)
crdb-1 | * Error types: (1) *withstack.withStack (2) *markers.withMark (3) *withstack.withStack (4) *errutil.leafError
crdb-1 | *
crdb-1 | *
crdb-1 | * ERROR: Queued as error 7965494b4cf14195980e878990c0f727
crdb-1 | *
crdb-1 | *
crdb-1 | * WARNING: Running a server without --sql-addr, with a combined RPC/SQL listener, is deprecated.
crdb-1 | * This feature will be removed in the next version of CockroachDB.
crdb-1 | *
crdb-1 | *
crdb-1 | * WARNING: The server appears to be unable to contact the other nodes in the cluster. Please try:
crdb-1 | *
crdb-1 | * - starting the other nodes, if you haven't already;
crdb-1 | * - double-checking that the '--join' and '--listen'/'--advertise' flags are set up correctly;
crdb-1 | * - running the 'cockroach init' command if you are trying to initialize a new cluster.
crdb-1 | *
crdb-1 | * If problems persist, please see https://www.cockroachlabs.com/docs/v22.2/cluster-setup-troubleshooting.html.
crdb-1 | *
crdb-1 | CockroachDB node starting at 2024-04-13 12:32:09.276199547 +0000 UTC m=+51.042282873 (took 38.9s)
crdb-1 | build: CCL v22.2.2 @ 2023/01/04 17:50:17 (go1.19.1)
crdb-1 | webui: https://crdb:8080
crdb-1 | sql: postgresql://root@crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt
crdb-1 | sql (JDBC): jdbc:postgresql://crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt&user=root
crdb-1 | RPC client flags: /cockroach/cockroach --host=crdb:26257 --certs-dir=certs
crdb-1 | logs: /cockroach/cockroach-data/logs
crdb-1 | temp dir: /cockroach/cockroach-data/cockroach-temp3219634163
crdb-1 | external I/O path: /cockroach/cockroach-data/extern
crdb-1 | store[0]: path=/cockroach/cockroach-data
crdb-1 | storage engine: pebble
crdb-1 | clusterID: ba60a6f0-9de2-4c8a-9414-258815587aa3
crdb-1 | status: restarted pre-existing node
crdb-1 | nodeID: 1
crdb-1 | *
crdb-1 | * ERROR: a panic has occurred!
crdb-1 | * pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781)
crdb-1 | * (1) attached stack trace
crdb-1 | * -- stack trace:
crdb-1 | * | runtime.gopanic
crdb-1 | * | GOROOT/src/runtime/panic.go:884
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).destroy
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:955
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).Close
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:312
crdb-1 | * | [...repeated from below...]
crdb-1 | * Wraps: (2) forced error mark
crdb-1 | * | "pebble: corruption"
crdb-1 | * | github.com/cockroachdb/errors/withstack/*withstack.withStack::
crdb-1 | * Wraps: (3) attached stack trace
crdb-1 | * -- stack trace:
crdb-1 | * | github.com/cockroachdb/pebble/internal/base.CorruptionErrorf
crdb-1 | * | github.com/cockroachdb/pebble/internal/base/external/com_github_cockroachdb_pebble/internal/base/error.go:27
crdb-1 | * | github.com/cockroachdb/pebble/sstable.checkChecksum
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2753
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*Reader).readBlock
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2817
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).readBlockWithStats
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:583
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).loadBlock
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:498
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).seekGEHelper
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:790
crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:702
crdb-1 | * | github.com/cockroachdb/pebble.(*levelIter).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/level_iter.go:648
crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).seekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:908
crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:970
crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1144
crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGE
crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1028
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).SeekGE
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:325
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).findTimeSeries
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/pruning.go:69
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).MaintainTimeSeries
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/maintenance.go:57
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*timeSeriesMaintenanceQueue).process
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/ts_maintenance_queue.go:154
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:997
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:91
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:956
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2.1
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:874
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489
crdb-1 | * | runtime.goexit
crdb-1 | * | src/runtime/asm_arm64.s:1172
crdb-1 | * Wraps: (4) pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781)
crdb-1 | * Error types: (1) *withstack.withStack (2) *markers.withMark (3) *withstack.withStack (4) *errutil.leafError
crdb-1 | *
crdb-1 | *
crdb-1 | * ERROR: Queued as error d5993a2c93ac417593a0f2e9cb265514
crdb-1 | *
crdb-1 | *
crdb-1 | * WARNING: Running a server without --sql-addr, with a combined RPC/SQL listener, is deprecated.
crdb-1 | * This feature will be removed in the next version of CockroachDB.
crdb-1 | *

@xcstatus commented on GitHub (Apr 13, 2024): The cockroach keeps restarting crdb-1 | * crdb-1 | * WARNING: Running a server without --sql-addr, with a combined RPC/SQL listener, is deprecated. crdb-1 | * This feature will be removed in the next version of CockroachDB. crdb-1 | * crdb-1 | * crdb-1 | * WARNING: The server appears to be unable to contact the other nodes in the cluster. Please try: crdb-1 | * crdb-1 | * - starting the other nodes, if you haven't already; crdb-1 | * - double-checking that the '--join' and '--listen'/'--advertise' flags are set up correctly; crdb-1 | * - running the 'cockroach init' command if you are trying to initialize a new cluster. crdb-1 | * crdb-1 | * If problems persist, please see https://www.cockroachlabs.com/docs/v22.2/cluster-setup-troubleshooting.html. crdb-1 | * crdb-1 | CockroachDB node starting at 2024-04-13 12:28:48.607991136 +0000 UTC m=+45.771172393 (took 42.2s) crdb-1 | build: CCL v22.2.2 @ 2023/01/04 17:50:17 (go1.19.1) crdb-1 | webui: https://crdb:8080 crdb-1 | sql: postgresql://root@crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt crdb-1 | sql (JDBC): jdbc:postgresql://crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt&user=root crdb-1 | RPC client flags: /cockroach/cockroach <client cmd> --host=crdb:26257 --certs-dir=certs crdb-1 | logs: /cockroach/cockroach-data/logs crdb-1 | temp dir: /cockroach/cockroach-data/cockroach-temp3464751422 crdb-1 | external I/O path: /cockroach/cockroach-data/extern crdb-1 | store[0]: path=/cockroach/cockroach-data crdb-1 | storage engine: pebble crdb-1 | clusterID: ba60a6f0-9de2-4c8a-9414-258815587aa3 crdb-1 | status: restarted pre-existing node crdb-1 | nodeID: 1 crdb-1 | * crdb-1 | * ERROR: a panic has occurred! crdb-1 | * pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781) crdb-1 | * (1) attached stack trace crdb-1 | * -- stack trace: crdb-1 | * | runtime.gopanic crdb-1 | * | GOROOT/src/runtime/panic.go:884 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).destroy crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:955 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).Close crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:312 crdb-1 | * | [...repeated from below...] crdb-1 | * Wraps: (2) forced error mark crdb-1 | * | "pebble: corruption" crdb-1 | * | github.com/cockroachdb/errors/withstack/*withstack.withStack:: crdb-1 | * Wraps: (3) attached stack trace crdb-1 | * -- stack trace: crdb-1 | * | github.com/cockroachdb/pebble/internal/base.CorruptionErrorf crdb-1 | * | github.com/cockroachdb/pebble/internal/base/external/com_github_cockroachdb_pebble/internal/base/error.go:27 crdb-1 | * | github.com/cockroachdb/pebble/sstable.checkChecksum crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2753 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*Reader).readBlock crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2817 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).readBlockWithStats crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:583 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).loadBlock crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:498 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).seekGEHelper crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:790 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:702 crdb-1 | * | github.com/cockroachdb/pebble.(*levelIter).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/level_iter.go:648 crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).seekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:908 crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:970 crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1144 crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1028 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).SeekGE crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:325 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).findTimeSeries crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/pruning.go:69 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).MaintainTimeSeries crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/maintenance.go:57 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*timeSeriesMaintenanceQueue).process crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/ts_maintenance_queue.go:154 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:997 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:91 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:956 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2.1 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:874 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489 crdb-1 | * | runtime.goexit crdb-1 | * | src/runtime/asm_arm64.s:1172 crdb-1 | * Wraps: (4) pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781) crdb-1 | * Error types: (1) *withstack.withStack (2) *markers.withMark (3) *withstack.withStack (4) *errutil.leafError crdb-1 | * crdb-1 | * crdb-1 | * ERROR: Queued as error 7965494b4cf14195980e878990c0f727 crdb-1 | * crdb-1 | * crdb-1 | * WARNING: Running a server without --sql-addr, with a combined RPC/SQL listener, is deprecated. crdb-1 | * This feature will be removed in the next version of CockroachDB. crdb-1 | * crdb-1 | * crdb-1 | * WARNING: The server appears to be unable to contact the other nodes in the cluster. Please try: crdb-1 | * crdb-1 | * - starting the other nodes, if you haven't already; crdb-1 | * - double-checking that the '--join' and '--listen'/'--advertise' flags are set up correctly; crdb-1 | * - running the 'cockroach init' command if you are trying to initialize a new cluster. crdb-1 | * crdb-1 | * If problems persist, please see https://www.cockroachlabs.com/docs/v22.2/cluster-setup-troubleshooting.html. crdb-1 | * crdb-1 | CockroachDB node starting at 2024-04-13 12:32:09.276199547 +0000 UTC m=+51.042282873 (took 38.9s) crdb-1 | build: CCL v22.2.2 @ 2023/01/04 17:50:17 (go1.19.1) crdb-1 | webui: https://crdb:8080 crdb-1 | sql: postgresql://root@crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt crdb-1 | sql (JDBC): jdbc:postgresql://crdb:26257/defaultdb?sslcert=certs%2Fclient.root.crt&sslkey=certs%2Fclient.root.key&sslmode=verify-full&sslrootcert=certs%2Fca.crt&user=root crdb-1 | RPC client flags: /cockroach/cockroach <client cmd> --host=crdb:26257 --certs-dir=certs crdb-1 | logs: /cockroach/cockroach-data/logs crdb-1 | temp dir: /cockroach/cockroach-data/cockroach-temp3219634163 crdb-1 | external I/O path: /cockroach/cockroach-data/extern crdb-1 | store[0]: path=/cockroach/cockroach-data crdb-1 | storage engine: pebble crdb-1 | clusterID: ba60a6f0-9de2-4c8a-9414-258815587aa3 crdb-1 | status: restarted pre-existing node crdb-1 | nodeID: 1 crdb-1 | * crdb-1 | * ERROR: a panic has occurred! crdb-1 | * pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781) crdb-1 | * (1) attached stack trace crdb-1 | * -- stack trace: crdb-1 | * | runtime.gopanic crdb-1 | * | GOROOT/src/runtime/panic.go:884 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).destroy crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:955 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).Close crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:312 crdb-1 | * | [...repeated from below...] crdb-1 | * Wraps: (2) forced error mark crdb-1 | * | "pebble: corruption" crdb-1 | * | github.com/cockroachdb/errors/withstack/*withstack.withStack:: crdb-1 | * Wraps: (3) attached stack trace crdb-1 | * -- stack trace: crdb-1 | * | github.com/cockroachdb/pebble/internal/base.CorruptionErrorf crdb-1 | * | github.com/cockroachdb/pebble/internal/base/external/com_github_cockroachdb_pebble/internal/base/error.go:27 crdb-1 | * | github.com/cockroachdb/pebble/sstable.checkChecksum crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2753 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*Reader).readBlock crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:2817 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).readBlockWithStats crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:583 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).loadBlock crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:498 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).seekGEHelper crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:790 crdb-1 | * | github.com/cockroachdb/pebble/sstable.(*singleLevelIterator).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/sstable/external/com_github_cockroachdb_pebble/sstable/reader.go:702 crdb-1 | * | github.com/cockroachdb/pebble.(*levelIter).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/level_iter.go:648 crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).seekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:908 crdb-1 | * | github.com/cockroachdb/pebble.(*mergingIter).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/merging_iter.go:970 crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1144 crdb-1 | * | github.com/cockroachdb/pebble.(*Iterator).SeekGE crdb-1 | * | github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/iterator.go:1028 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).SeekGE crdb-1 | * | github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:325 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).findTimeSeries crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/pruning.go:69 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts.(*DB).MaintainTimeSeries crdb-1 | * | github.com/cockroachdb/cockroach/pkg/ts/maintenance.go:57 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*timeSeriesMaintenanceQueue).process crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/ts_maintenance_queue.go:154 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:997 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:91 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:956 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2.1 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:874 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 crdb-1 | * | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489 crdb-1 | * | runtime.goexit crdb-1 | * | src/runtime/asm_arm64.s:1172 crdb-1 | * Wraps: (4) pebble/table: invalid table 181191 (checksum mismatch at 1027356/17781) crdb-1 | * Error types: (1) *withstack.withStack (2) *markers.withMark (3) *withstack.withStack (4) *errutil.leafError crdb-1 | * crdb-1 | * crdb-1 | * ERROR: Queued as error d5993a2c93ac417593a0f2e9cb265514 crdb-1 | * crdb-1 | * crdb-1 | * WARNING: Running a server without --sql-addr, with a combined RPC/SQL listener, is deprecated. crdb-1 | * This feature will be removed in the next version of CockroachDB. crdb-1 | *
saavagebueno commented 2025-11-20 05:17:43 -05:00
Author
Owner
Copy Link

@xcstatus commented on GitHub (Apr 13, 2024):

The problem persists even after restarting all containers

@xcstatus commented on GitHub (Apr 13, 2024): The problem persists even after restarting all containers
saavagebueno commented 2025-11-20 05:17:44 -05:00
Author
Owner
Copy Link

@mlsmaycon commented on GitHub (Apr 13, 2024):

Can you check with the cockroach's community? I believe they will be the best ones to help you restore the database.

Maybe Zitadel's community too.

@mlsmaycon commented on GitHub (Apr 13, 2024): Can you check with the cockroach's community? I believe they will be the best ones to help you restore the database. Maybe Zitadel's community too.
saavagebueno commented 2025-11-20 05:17:44 -05:00
Author
Owner
Copy Link

@xcstatus commented on GitHub (Apr 13, 2024):

This issue was resolved by restarting the server. Thank you very much.

Still have to do a good job of data backup, data is priceless.

@xcstatus commented on GitHub (Apr 13, 2024): This issue was resolved by restarting the server. Thank you very much. Still have to do a good job of data backup, data is priceless.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
2021 Q4
2022 Q1
2022 Q1
accessibility
acl
agent
agent
Android
Android
api
authentik
automation
azure
battery-usage
bug
cache
client
client-ui
cloud
cloud-only
cloudflare
community
compatibility
config-idp
config-issue
connection
contribution
coturn
cross-vpn
dashboard
data-usage
distribution
dns
docker
documentation
duplicate
enhancement
enhancement
event-stream
feature-request
freebsd
getting-started
go
good first issue
gui
help wanted
home-assistant
idp
inconsistency
integration
integrations
ios
ipv6
jwt
k8s
keycloak
linux
login
macos
management-service
missing-docs
mobile
moved-internal
needs-review
netbird-ui
networking
new-platform
nginx
notification
okta
openwrt
packaging
peer-management
peer-management
peer-management
performance
postgres
posture-checks
psk
pull-request
Mirrored from GitHub Pull Request
 question
refactor
relay
release
rfc
routes
security
security-related
self-hosting
server
signal
sleep-issue
ssh
ssl
status
store
synology
system-compatibility-issue
test-suite
third-party-integration
triage
triage-needed
troubleshooting
UX
waiting-feedback
windows
wontfix
zitadel
No Label triage-needed
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
saavagebueno
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: SVI/netbird#796
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?