[PR #3890] [management] Add postgres support for activity event store #3980

New Issue

saavagebueno · 2025-11-20T08:07:28-05:00

saavagebueno commented

2025-11-20 08:07:28 -05:00

Original Pull Request: https://github.com/netbirdio/netbird/pull/3890

State: closed
Merged: Yes

Describe your changes

Adds postgres as an optional storage engine for activity events while keeping sqlite as the default. To use postgres, set the environment variable NB_ACTIVITY_EVENT_STORE_ENGINE=postgres and provide a connection string via NB_ACTIVITY_EVENT_POSTGRES_DSN.

Migrating existing events to Postgres

Back up the current sqlite store

mkdir backup
docker compose cp -a management:/var/lib/netbird/. backup/

Load the data into Postgres

pgloader --type sqlite backup/events.db "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>"

Issue ticket number and link

Stack

Checklist

Is it a bug fix
Is a typo/documentation fix
Is a feature enhancement
It is a refactor
Created tests that fail without the change (if possible)
Extended the README / documentation, if necessary

By submitting this pull request, you confirm that you have read and agree to the terms of the Contributor License Agreement.

**Original Pull Request:** https://github.com/netbirdio/netbird/pull/3890 **State:** closed **Merged:** Yes --- ## Describe your changes Adds postgres as an optional storage engine for activity events while keeping sqlite as the default. To use postgres, set the environment variable `NB_ACTIVITY_EVENT_STORE_ENGINE=postgres` and provide a connection string via `NB_ACTIVITY_EVENT_POSTGRES_DSN`. Migrating existing events to Postgres 1. Back up the current sqlite store ```sh mkdir backup docker compose cp -a management:/var/lib/netbird/. backup/ ``` 2. Load the data into Postgres ```sh pgloader --type sqlite backup/events.db "postgresql://<PG_USER>:<PG_PASSWORD>@<PG_HOST>:<PG_PORT>/<PG_DB_NAME>" ``` ## Issue ticket number and link ## Stack  ### Checklist - [ ] Is it a bug fix - [ ] Is a typo/documentation fix - [ ] Is a feature enhancement - [x] It is a refactor - [ ] Created tests that fail without the change (if possible) - [ ] Extended the README / documentation, if necessary > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md).

saavagebueno added the pull-request label 2025-11-20 08:07:28 -05:00

saavagebueno referenced this issue

2025-11-20 08:07:36 -05:00

[PR #3980] [client] Feat/multiple profile #4035

Sign in to join this conversation.

Branches Tags

main

feature/fleetdm

feat/byod-proxy

fix/pat-target-user-account-valdation

client-ipv6-android-ui

dependabot/go_modules/golang.org/x/image-0.38.0

client-ipv6-iptables

combined-migration-2

feature/use-local-keys-embedded

entire/checkpoints/v1

update-embedded-idp-user

mgmt-ipv6-addressing

crowdsec-integration

pcp-support

nat-pmp-upnp

refactor/unexport-getserverpublickey-add-healthcheck

fix/module-check-for-posturecheck-delete

client-ipv6-nftables

client-ipv6-routing

refactor/permissions-manager

feat/reseller-openapi-spec

client-ipv6-acl-usp

dependabot/go_modules/github.com/russellhaering/goxmldsig-1.6.0

fix-ssh-stop-deadlock

client-ipv6-ssh-netflow

client-ipv6-dns

client-ipv6-iface

proto-ipv6-overlay

dependabot/npm_and_yarn/proxy/web/picomatch-4.0.4

iptables-mangle-dnat-guard

fix/ssh-proxy-command-quoting

fix/userspace-native-firewall

dependabot/go_modules/filippo.io/edwards25519-1.1.1

dependabot/npm_and_yarn/proxy/web/multi-770cfcd984

dependabot/npm_and_yarn/proxy/web/rollup-4.60.0

dependabot/npm_and_yarn/proxy/web/flatted-3.4.2

chore/proxy-web-packages

set-env-only-if-not-fork

feature/expose-has-channel

fix/connection-status-race

nmap/cleanup

fix/filter-cgnat-cni-ice-candidates

feature/check-cert-locker-before-acme

test/proxy-fixes

test/proxy-mtu

prototype/ui-tauri

test/proxy-speed

fix-reused-ports

feat/migrate-to-embedded-idp

feature/add-serial-to-proxy-merged

deploy/proxy-serial

fix/policy-upd

test/connection

feature/disable-legacy-port

feature/flag-to-disable-legacy-port

test/perftest

dependabot/go_modules/github.com/pion/dtls/v3-3.0.11

fix/http-redirect

poc-token-command

dn-reverse-proxy

prototype/reverse-proxy-rename

prototype/reverse-proxy-logs-pagination

feature/client-metrics

local-dns-listener

prototype/reverse-proxy-clusters

debug-dns-route

fix/win-dns-batch

add-extra-route-logs

job-stream-notify-disconnection-eof

deploy/secrets-manager

trigger-proxy-update

bug/update-ios-client-code-build-tags

sync-client-netmap-serial

log/conn-disconn

nmap/compaction-deploy

ci-win-test

feature/disk-encryption-check

wasm-debug

swap-dns-prio

fix/dex-config

feature/migrate-auto-groups-to-table

dependabot/go_modules/github.com/quic-go/quic-go-0.57.0

nmap/compaction

dex-nocgo-stub

feature/exclude-terraform-from-rate-limiting

test-freebsd

fix/login-cmd-root-flags

retries-refactor

coderabbitai/docstrings/b7e98ac

feat/integrate-zitadel

bug/ios-hanging-reconection

zitadel-idp

feat/network-map-serial

refactor/get-account-no-users

feat/auto-upgrade

feature/report-high-pat-id

feature/temporary-access-for-resource

fix/nmap-fwrules

dont-restart-dns

prototype/ui

update-gomobile

go-dns-for-ice

wasm-ldflags

test-ldflags

wasmbuild-test

feature/networks-s2s

vk/compare-nmaps

dbg/bothmaps

feature/changeset

reorder-dns-shutdown

fix/relay-reconnection-race

fix/nmap-exitnodes

vk/debug/nmap-both

move-licensed-code

feat/better-daemon-connection-lost-message

feat/auto-update-2

test/timings

refactor/getaccount-raw

tests/nmap-getaccount

refactor/nmap

refactor/nmap-limit-buffer

feature/detect-mac-wakeup

feature/extract-modules

quick-setings

feat/sync-limiter

feature/store-cache-impl

fix-install-version

feature/store-metrics

feature/metrics-on-store

feature/use-gorm-cache

loadtest-signal

unsymmetrical-squash

refactor/reducate-signaling

test/update-reduce

feature/store-cache

feature/remote-debug

cli-ws-proxy-backend-addr

feat/mgmt-map-serial

snyk-fix-d9d0081a4c7f9137bdb59d0d50a141a2

snyk-fix-7415cea5a11acd66753540ca2c598c63

job-yml-update

feature/android-allow-selecting-routes

fix/up-sequence

fix/dns-hash-update

snyk-fix-967adae9863f17f108ce8948d9117b8d

log/getaccount-by-peer

signal-suppressor

dns-exit-node

feature/auto-updates

feature/cache-srv-key

merged-fixes

fix/missed-offers-and-debug

debug-and-fixes

poc-wasm-clean-backend-s2s

test/remote-debug

debug-api

dependabot/go_modules/github.com/docker/docker-28.0.0incompatible

fix/remove-gpo-if-empty

fix/test-freebsd

fix/mysql-setup

fix/remove-logout-btn

handle-existing-domain-user

chore/unify-domain-validation

snyk-fix-c5fafc8a50ce1f29046e25a1fc346185

feat/profile-edit-btn

snyk-fix-a54966211e18d4cf67e5a2757cc006d1

log-short-id

feat/logout-ephemeral

log-checks

batch-wg-ops

nb-interface-default

feat/aws-integration

add/race-test

feature/relay-feature-versioning

fix/systemd-service-logs

poc/preprocessed-map

add-account-onboarding

bind-ipv6

fix/merge-main

logs/peerlogs-addpeer

feature/net-297-network-migration

feature/support-skip-auto-apply-exit-node-routes

set-cmd

set-command-with-cursor

feature/limit-update-channel

stop-using-locking-share

feature/poc-lazy-detection

feature/net-248-removal-of-sync-mutex-locks

test/multiple-peer-logging

preresolve

add-ns-punnycode-support

apply-routes-early

windows-search-domains

fix/connecting-route-filter

feature/management/rest-client/impersonate

debug-local-records

resource-fields-snake-case

test/grpc-rate-limit

traffic-correlation-policy

feature/rest-client-options

feat/events-metrics

feature/buf-cli

test/add-ratelimiter

test/remove-write-lock-on-add-peer

fix/add-peer-semaphore

feature/users-roles-endpoint

mlsmaycon-patch-1

debug-user-role

chore/primary-key-on-networks

feature/update-account-peers-buffer-startup

remove-ubuntu2004-runners

refactor/permissions-no-pat-allowed

ref/logrus-factory

use-conntrack-zone

deploy/permissions-account

feature/lazy-connection-idle

ref/improve-test-cov

restore-pr-3440

test/increase-grpc-timeouts

feat/buffer-account-peers-update

test/networkmapgeneration-changes

feature/base-manager

feature/flow-receiver

chore/benchmark-with-large-runner

refactor/handshake-initiator

client/ui-update-systray-icons

userspace-router

wgwatcher-test

output-if-key-already-exists

fix/relay-reconnection

feature/port-forwarding-client-codecleaning

detached2

test/callbacks-nil-iceconninfo

refactor/optimize-peer-expiration

enable-udp-port-for-docker-template

fix/relay-update

feature/apply-posture-netmap

fix/group-update-existing-resource

conntrack-stats

upgrade-okta-sdk

multi-price

test/conn-stat

set-min-parallel-tests-for-management

dns-interceptor

debug-dns

router-dns

add-static-system-info

debug-0.29.4

debug-0.33.0

account-refactoring

relay/2800_quic

route-get-account-refactoring

test/seed-random-routes

feature/get-account-refactoring

test/reconnect-race-condition

refactor/get-account-usage

feature/add-session-id-to-update-channel

improve-ipv4conn

fix/async-pion-event-handling

debug

add-offload

feature/validate-group-association-debug

fix/limit-conn-for-sqlite

test/engine-iface

test/transaction-for-jwt-sync

fix/engine-stop-in-foreground

feature/add-mysql-support

test-migration

refactor/header-size-values

relay/eliminate-gob

test/signal-dispatcher-with-relay

relay/debug

validate-icon

feature/ipv6-support

use-pre-expanded-peers-map

feature/use-signal-dispatcher

validate/peer-status

add-read-write-times

fix/sync-peer-race

feature/relay-status

netmap

evaluate/network-map-hash

fix/lower-dns-resolve-interval-on-fail

feature/relay

fix/go-mod-version

upgrade-nftables

synology-userspace-mode

fix/use-ip-for-default-routes-on-darwin

fix/proxy_close

enable-release-workflow-on-pr

deploy/peer-performance

feature/permanent-turn

feature/permanent-turn-proxy

deploy/posture-check-sqlite

feature/optimize_sqlite_save

debug-ios-behavior

fix/delete-route-only-after-adding

tshoot/windows-logger

remove-new-routing

refactor/eliminate-repo-dependency

add-arm-to-ci

refactor-demo-account-object

test/abc2

test/abc

send-ssh-rosenpass-config-meta

refactor-demo

ensure-schedule-never-runs-non-positive

feature/peer-validator-groupmgm

feature/peer-validator-fix

fix/include-active-dashboard-users

fix/handle-canceling-schedule

fix/geo-download

debug-google-workspace

yury/resolve-ip-to-location

feature/extend-sysinfo

sqlite-async-peer-status

yury/add-postgresql-store

fix/route

test-build

posture-checks-poc

debug-keycloak-idp

poc/netstack

for-pascal-tmp

peer-logout-management

manual-peer-logout

detached

chore/refactor-management

test/dns-bind

fix/enforce-acl-for-containers

yury/use-sync-map-in-updatechannel

fix/events-key-handling

filter-cache-on-load-account

fix/user-expiration

handle-user-context-cancellation

nb-client-k8s-statefulset

fake-addr

fix/iptables_in_docker

ebpf-debug

update-getting-started-flow-use-postgres

fix/peer_list_notification

feature/device-authentication-with-client-secret

feature/keep_alive

feat-groups-from-jwt

separate_proxy_from_wgconfig

fix/wg_conn

wg_conn_fix

wg_bind_parallel_processing

fix-rollback-get-acls

proxy_cfg_cleanup

performance-improvement-rego

update-lock-log-level

feat-client-side-acl

refactor/move_grpcserver_logic_to_account_manager

feature/event-storage

feature/update-idp-redeeming-invite

feature/api-peer-info

return-groupminimum-setupkey

feature/interface-bind

documentation_enhancement

fix-peer-registration

ssh

users_cache

pass-client-caller

client_caller_type

revert-283-feat-fix-windows-installer

periodic-peer-updates

ebpf

braginini/wasm

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: SVI/netbird#3980