[PR #6199] [misc, infrastructure] proto: unify toolchain & gate CI on drift #27743

Open
opened 2026-08-05 07:09:10 -04:00 by saavagebueno · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/netbirdio/netbird/pull/6199
Author: @riccardomanfrin
Created: 5/19/2026
Status: 🔄 Open

Base: mainHead: task/align_protobuff_toolset


📝 Commits (6)

  • e1b6864 Aligns go protoc toolset
  • 46ec42b Regenerated pb go files
  • 0151cbd GH action gate on generated PB output
  • 0920fa1 Address SonarCloud quality gate
  • 274a184 Detect untracked files in proto drift gate
  • 63e7a01 Merge branch 'main' into task/align_protobuff_toolset

📊 Changes

16 files changed (+2084 additions, -4016 deletions)

View changed files

📝 .github/workflows/proto-version-check.yml (+65 -51)
📝 client/proto/generate.sh (+15 -3)
📝 encryption/testprotos/generate.sh (+27 -1)
📝 encryption/testprotos/testproto.pb.go (+20 -39)
📝 flow/proto/flow.pb.go (+119 -215)
📝 flow/proto/flow_grpc.pb.go (+38 -56)
📝 flow/proto/generate.sh (+15 -3)
proto-tools.env (+14 -0)
📝 shared/management/proto/generate.sh (+15 -3)
📝 shared/management/proto/management.pb.go (+925 -2044)
📝 shared/management/proto/management_grpc.pb.go (+102 -122)
📝 shared/management/proto/proxy_service.pb.go (+488 -1101)
📝 shared/management/proto/proxy_service_grpc.pb.go (+88 -110)
📝 shared/signal/proto/generate.sh (+16 -4)
📝 shared/signal/proto/signalexchange.pb.go (+94 -205)
📝 shared/signal/proto/signalexchange_grpc.pb.go (+43 -59)

📄 Description

Describe your changes

Unifies the protobuf code-generation toolchain across the repo and adds a CI
gate that fails on drift between .proto sources and committed *.pb.go.

Why

The five proto generate.sh scripts pinned different (and stale) tool
versions, and none pinned protoc itself. The committed *.pb.go files were
generated with at least four different protoc versions (v3.12.4,
v3.21.9, v6.33.1, v7.34.1) and a protoc-gen-go-grpc newer than the
pinned v1.1.

What changed

  • New proto-tools.env at repo root — single source of truth for
    PROTOC_VERSION, PROTOC_GEN_GO_VERSION, PROTOC_GEN_GO_GRPC_VERSION.
  • All five proto generate.sh scripts (client/proto,
    shared/signal/proto, shared/management/proto, flow/proto,
    encryption/testprotos) now source proto-tools.env, install pinned
    plugin versions, and refuse to run if the local protoc does not match
    the pinned release.
  • All *.pb.go regenerated with the pinned toolchain
    (protoc v6.33.1, protoc-gen-go v1.36.6, protoc-gen-go-grpc v1.6.1).
    Headers now match across every generated file.
  • .github/workflows/proto-version-check.yml rewritten: installs the
    pinned protoc and plugins, runs every generate.sh, and fails via
    git diff --exit-code on any drift. Trigger paths extended to
    **/*.proto, **/*.pb.go, **/generate.sh, proto-tools.env, and the
    workflow itself.

Holes in the previous CI check

The old proto-version-check.yml parsed the PR diff with a regex on
header comments. It missed drift in several ways:

  • Regex did not match protoc-gen-go-grpc. The pattern
    protoc(?:-gen-go)? accepts protoc or protoc-gen-go, never the
    -grpc plugin. Version changes there passed silently.
  • protoc-gen-go-grpc v1.5+ stopped emitting the version header.
    Nothing left to match — total bypass.
  • Trigger restricted to **/*.pb.go. A PR that edits .proto but
    forgets to regenerate never runs the check.
  • No canonical pin. Check compared old vs new in the same PR, not
    against a repo-wide pinned toolchain. Two PRs introducing two
    different tool versions both pass.
  • Patch truncation on large PRs. API omits patch data, check
    explicitly bails with "verify manually".

The new workflow regenerates with the pinned toolchain and runs
git diff --exit-code. Byte-for-byte comparison against canonical
output. None of the above gaps remain.

Compatibility

  • Wire format: unchanged (.proto sources untouched). Cross-version peers
    interoperate.
  • In-repo Go API: regenerated together, no internal call sites broken.
  • External consumers importing these proto packages must embed
    UnimplementedXxxServer (already required by protoc-gen-go-grpc v1.5+);
    the in-repo implementations already do.

Verification

  • proxy/web UI build green (tsc && vite build, 1602 modules,
    no TS errors).
  • go build on every proto package green (client/proto,
    shared/signal/proto, shared/management/proto, flow/proto,
    encryption/testprotos).

N/A

Stack

Checklist

  • Is it a bug fix
  • Is a typo/documentation fix
  • Is a feature enhancement
  • It is a refactor
  • Created tests that fail without the change (if possible)

The new CI workflow itself is the regression test: if a future PR commits
*.pb.go generated with a different toolchain, the job fails.

By submitting this pull request, you confirm that you have read and agree to the terms of the Contributor License Agreement.

Documentation

Select exactly one:

  • I added/updated documentation for this change
  • Documentation is not needed for this change (explain why)

Internal build/CI infrastructure only. No user-facing API or behavior change.

Docs PR URL (required if "docs added" is checked)

N/A

Summary by CodeRabbit

  • Chores
    • Centralized pinning of protobuf compiler and Go code-gen plugin versions for proto generation.
    • CI now regenerates proto artifacts and fails PRs if generated outputs drift from pinned toolchain or sources.
    • Hardened proto generation scripts with robust path handling, environment sourcing, stricter error checks, and runtime version verification.

Review Change Stack


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/netbirdio/netbird/pull/6199 **Author:** [@riccardomanfrin](https://github.com/riccardomanfrin) **Created:** 5/19/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `task/align_protobuff_toolset` --- ### 📝 Commits (6) - [`e1b6864`](https://github.com/netbirdio/netbird/commit/e1b686420a0b6af6210cbfcb757108bc2143a28c) Aligns go protoc toolset - [`46ec42b`](https://github.com/netbirdio/netbird/commit/46ec42b9a07e4783cce5a36d4c17a9018026af7e) Regenerated pb go files - [`0151cbd`](https://github.com/netbirdio/netbird/commit/0151cbd3e3422d9b85319fa0b115bbe5764d8df9) GH action gate on generated PB output - [`0920fa1`](https://github.com/netbirdio/netbird/commit/0920fa1fa91f48a32e3960f788bd5c5fa5e68a60) Address SonarCloud quality gate - [`274a184`](https://github.com/netbirdio/netbird/commit/274a1841357262344118f430158dd00bc80c7729) Detect untracked files in proto drift gate - [`63e7a01`](https://github.com/netbirdio/netbird/commit/63e7a018c10567f3af5826c56cfa3060a88528a1) Merge branch 'main' into task/align_protobuff_toolset ### 📊 Changes **16 files changed** (+2084 additions, -4016 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/proto-version-check.yml` (+65 -51) 📝 `client/proto/generate.sh` (+15 -3) 📝 `encryption/testprotos/generate.sh` (+27 -1) 📝 `encryption/testprotos/testproto.pb.go` (+20 -39) 📝 `flow/proto/flow.pb.go` (+119 -215) 📝 `flow/proto/flow_grpc.pb.go` (+38 -56) 📝 `flow/proto/generate.sh` (+15 -3) ➕ `proto-tools.env` (+14 -0) 📝 `shared/management/proto/generate.sh` (+15 -3) 📝 `shared/management/proto/management.pb.go` (+925 -2044) 📝 `shared/management/proto/management_grpc.pb.go` (+102 -122) 📝 `shared/management/proto/proxy_service.pb.go` (+488 -1101) 📝 `shared/management/proto/proxy_service_grpc.pb.go` (+88 -110) 📝 `shared/signal/proto/generate.sh` (+16 -4) 📝 `shared/signal/proto/signalexchange.pb.go` (+94 -205) 📝 `shared/signal/proto/signalexchange_grpc.pb.go` (+43 -59) </details> ### 📄 Description ## Describe your changes Unifies the protobuf code-generation toolchain across the repo and adds a CI gate that fails on drift between `.proto` sources and committed `*.pb.go`. ### Why The five proto `generate.sh` scripts pinned different (and stale) tool versions, and none pinned `protoc` itself. The committed `*.pb.go` files were generated with at least four different `protoc` versions (`v3.12.4`, `v3.21.9`, `v6.33.1`, `v7.34.1`) and a `protoc-gen-go-grpc` newer than the pinned `v1.1`. ### What changed - New `proto-tools.env` at repo root — single source of truth for `PROTOC_VERSION`, `PROTOC_GEN_GO_VERSION`, `PROTOC_GEN_GO_GRPC_VERSION`. - All five proto `generate.sh` scripts (`client/proto`, `shared/signal/proto`, `shared/management/proto`, `flow/proto`, `encryption/testprotos`) now `source` `proto-tools.env`, install pinned plugin versions, and refuse to run if the local `protoc` does not match the pinned release. - All `*.pb.go` regenerated with the pinned toolchain (`protoc v6.33.1`, `protoc-gen-go v1.36.6`, `protoc-gen-go-grpc v1.6.1`). Headers now match across every generated file. - `.github/workflows/proto-version-check.yml` rewritten: installs the pinned `protoc` and plugins, runs every `generate.sh`, and fails via `git diff --exit-code` on any drift. Trigger paths extended to `**/*.proto`, `**/*.pb.go`, `**/generate.sh`, `proto-tools.env`, and the workflow itself. ### Holes in the previous CI check The old `proto-version-check.yml` parsed the PR diff with a regex on header comments. It missed drift in several ways: - **Regex did not match `protoc-gen-go-grpc`.** The pattern `protoc(?:-gen-go)?` accepts `protoc` or `protoc-gen-go`, never the `-grpc` plugin. Version changes there passed silently. - **`protoc-gen-go-grpc` v1.5+ stopped emitting the version header.** Nothing left to match — total bypass. - **Trigger restricted to `**/*.pb.go`.** A PR that edits `.proto` but forgets to regenerate never runs the check. - **No canonical pin.** Check compared old vs new in the same PR, not against a repo-wide pinned toolchain. Two PRs introducing two different tool versions both pass. - **Patch truncation on large PRs.** API omits `patch` data, check explicitly bails with "verify manually". The new workflow regenerates with the pinned toolchain and runs `git diff --exit-code`. Byte-for-byte comparison against canonical output. None of the above gaps remain. ### Compatibility - Wire format: unchanged (`.proto` sources untouched). Cross-version peers interoperate. - In-repo Go API: regenerated together, no internal call sites broken. - External consumers importing these proto packages must embed `UnimplementedXxxServer` (already required by `protoc-gen-go-grpc` v1.5+); the in-repo implementations already do. ### Verification - `proxy/web` UI build green (`tsc && vite build`, 1602 modules, no TS errors). - `go build` on every proto package green (`client/proto`, `shared/signal/proto`, `shared/management/proto`, `flow/proto`, `encryption/testprotos`). ## Issue ticket number and link N/A ## Stack <!-- branch-stack --> ### Checklist - [ ] Is it a bug fix - [ ] Is a typo/documentation fix - [ ] Is a feature enhancement - [ ] It is a refactor - [x] Created tests that fail without the change (if possible) The new CI workflow itself is the regression test: if a future PR commits `*.pb.go` generated with a different toolchain, the job fails. > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md). ## Documentation Select exactly one: - [ ] I added/updated documentation for this change - [x] Documentation is **not needed** for this change (explain why) Internal build/CI infrastructure only. No user-facing API or behavior change. ### Docs PR URL (required if "docs added" is checked) N/A <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Centralized pinning of protobuf compiler and Go code-gen plugin versions for proto generation. * CI now regenerates proto artifacts and fails PRs if generated outputs drift from pinned toolchain or sources. * Hardened proto generation scripts with robust path handling, environment sourcing, stricter error checks, and runtime version verification. <!-- review_stack_entry_start --> [![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/netbirdio/netbird/pull/6199?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
saavagebueno added the pull-request label 2026-08-05 07:09:10 -04:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#27743