[PR #6900] [MERGED] [management, proxy] Add prompt-cache token and cost accounting to agent network usage #27452

Closed
opened 2026-08-05 07:08:44 -04:00 by saavagebueno · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/netbirdio/netbird/pull/6900
Author: @mlsmaycon
Created: 7/25/2026
Status: Merged
Merged: 7/26/2026
Merged by: @braginini

Base: mainHead: anet-cache-cost-accounting


📝 Commits (9)

  • 99ad564 [proxy] parse Bedrock Converse camelCase cache token fields
  • d9bf667 [proxy] split prompt-cache cost out of the metered total
  • 0c26e36 [management] expose prompt-cache accounting in the agent-network REST API
  • 6faffe5 [management] persist prompt-cache tokens and cache cost per request
  • 8bc2851 [proxy] add end-to-end cost calculation matrix test
  • 4e624a2 [e2e] validate stored totals and cache costs against published rates
  • 691953b [ci] drop the temporary e2e push trigger
  • b67ea93 Store per-bucket LLM cost, derive totals on read.
  • 75e695d Fix failing tests

📊 Changes

29 files changed (+1582 additions, -170 deletions)

View changed files

📝 .github/workflows/agent-network-e2e.yml (+9 -0)
📝 e2e/agentnetwork/chat_test.go (+227 -7)
📝 e2e/harness/client.go (+5 -5)
📝 e2e/harness/combined.go (+23 -0)
📝 management/internals/modules/agentnetwork/accesslog_ingest.go (+56 -41)
📝 management/internals/modules/agentnetwork/accesslog_ingest_realstore_test.go (+37 -12)
📝 management/internals/modules/agentnetwork/accesslog_sessions_realstore_test.go (+3 -3)
📝 management/internals/modules/agentnetwork/types/accesslog.go (+105 -36)
📝 management/internals/modules/agentnetwork/types/accesslogfilter.go (+2 -2)
management/internals/modules/agentnetwork/types/cost_test.go (+124 -0)
📝 management/internals/modules/agentnetwork/types/usage.go (+24 -2)
📝 management/internals/modules/agentnetwork/types/usageoverview.go (+40 -11)
📝 management/server/migration/migration.go (+78 -0)
📝 management/server/migration/migration_test.go (+97 -0)
📝 management/server/store/sql_store_agentnetwork.go (+1 -1)
📝 management/server/store/sql_store_agentnetwork_accesslog_test.go (+6 -6)
📝 management/server/store/store.go (+8 -0)
📝 proxy/internal/accesslog/logger.go (+15 -8)
📝 proxy/internal/llm/bedrock.go (+11 -7)
📝 proxy/internal/llm/bedrock_test.go (+12 -0)

...and 9 more files

📄 Description

Describe your changes

Bedrock Converse responses report prompt-cache usage in camelCase fields (cacheReadInputTokens/cacheWriteInputTokens) the proxy didn't parse, so cached Converse requests were under-billed. This fixes the parsing and makes cache accounting first-class: the cost meter emits the cache portion of each request's cost (cost.usd_cache), and cached_input_tokens, cache_creation_tokens, and cache_cost_usd are persisted in the agent-network access-log and request-usage tables and exposed in the REST API (entries, sessions, usage buckets), so cache-write-heavy requests are explainable instead of hiding inside total_tokens.

N/A

Stack

Checklist

  • Is it a bug fix
  • Is a typo/documentation fix
  • Is a feature enhancement
  • It is a refactor
  • Created tests that fail without the change (if possible)
  • This change does not modify the public API, gRPC protocols, functionality behavior, CLI / service flags, or introduce a new feature — OR I have discussed it with the NetBird team beforehand (link the issue / Slack thread in the description). See CONTRIBUTING.md.

By submitting this pull request, you confirm that you have read and agree to the terms of the Contributor License Agreement.

Documentation

Select exactly one:

  • I added/updated documentation for this change
  • Documentation is not needed for this change (explain why)

Docs PR URL (required if "docs added" is checked)

Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/886

Summary by CodeRabbit

  • New Features

    • Added prompt-cache token counts and a full cost breakdown (input, cached-input, cache-creation, output) plus cache-attributed totals across access logs, sessions, and usage summaries.
    • Updated Bedrock/Converse parsing and streaming to include cache token fields.
    • Extended cost reporting metadata to include per-component USD values and cost.usd_cache; updated request token limits for e2e chat.
    • Added optional Bedrock model override for e2e workflow runs.
  • Migration

    • Migrates legacy aggregate cost data into the new per-bucket cost format while preserving historical totals.
  • Bug Fixes

    • Improved pricing, cost aggregation, and cost computation accuracy across providers.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/netbirdio/netbird/pull/6900 **Author:** [@mlsmaycon](https://github.com/mlsmaycon) **Created:** 7/25/2026 **Status:** ✅ Merged **Merged:** 7/26/2026 **Merged by:** [@braginini](https://github.com/braginini) **Base:** `main` ← **Head:** `anet-cache-cost-accounting` --- ### 📝 Commits (9) - [`99ad564`](https://github.com/netbirdio/netbird/commit/99ad5642b9c8d840dc177cce2b7e6f7d15a4b8c8) [proxy] parse Bedrock Converse camelCase cache token fields - [`d9bf667`](https://github.com/netbirdio/netbird/commit/d9bf667275def7241e524343abaf923ce47f483e) [proxy] split prompt-cache cost out of the metered total - [`0c26e36`](https://github.com/netbirdio/netbird/commit/0c26e364eb906777db8775ef44ce9b6e63c50552) [management] expose prompt-cache accounting in the agent-network REST API - [`6faffe5`](https://github.com/netbirdio/netbird/commit/6faffe5716fcc411ccb980d67a3d55964536e690) [management] persist prompt-cache tokens and cache cost per request - [`8bc2851`](https://github.com/netbirdio/netbird/commit/8bc2851f70ec45e4c28696af525c4ed103803afb) [proxy] add end-to-end cost calculation matrix test - [`4e624a2`](https://github.com/netbirdio/netbird/commit/4e624a25c2ad223f9bd0e7bb5ec78df592e0e713) [e2e] validate stored totals and cache costs against published rates - [`691953b`](https://github.com/netbirdio/netbird/commit/691953bacf16993e0fcc1610329c22fb9fbaa604) [ci] drop the temporary e2e push trigger - [`b67ea93`](https://github.com/netbirdio/netbird/commit/b67ea93c03c7b4d90d94f521a2663876d9af3385) Store per-bucket LLM cost, derive totals on read. - [`75e695d`](https://github.com/netbirdio/netbird/commit/75e695d0ea5f07497fb293b8e30e4d80c3d39060) Fix failing tests ### 📊 Changes **29 files changed** (+1582 additions, -170 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/agent-network-e2e.yml` (+9 -0) 📝 `e2e/agentnetwork/chat_test.go` (+227 -7) 📝 `e2e/harness/client.go` (+5 -5) 📝 `e2e/harness/combined.go` (+23 -0) 📝 `management/internals/modules/agentnetwork/accesslog_ingest.go` (+56 -41) 📝 `management/internals/modules/agentnetwork/accesslog_ingest_realstore_test.go` (+37 -12) 📝 `management/internals/modules/agentnetwork/accesslog_sessions_realstore_test.go` (+3 -3) 📝 `management/internals/modules/agentnetwork/types/accesslog.go` (+105 -36) 📝 `management/internals/modules/agentnetwork/types/accesslogfilter.go` (+2 -2) ➕ `management/internals/modules/agentnetwork/types/cost_test.go` (+124 -0) 📝 `management/internals/modules/agentnetwork/types/usage.go` (+24 -2) 📝 `management/internals/modules/agentnetwork/types/usageoverview.go` (+40 -11) 📝 `management/server/migration/migration.go` (+78 -0) 📝 `management/server/migration/migration_test.go` (+97 -0) 📝 `management/server/store/sql_store_agentnetwork.go` (+1 -1) 📝 `management/server/store/sql_store_agentnetwork_accesslog_test.go` (+6 -6) 📝 `management/server/store/store.go` (+8 -0) 📝 `proxy/internal/accesslog/logger.go` (+15 -8) 📝 `proxy/internal/llm/bedrock.go` (+11 -7) 📝 `proxy/internal/llm/bedrock_test.go` (+12 -0) _...and 9 more files_ </details> ### 📄 Description ## Describe your changes Bedrock Converse responses report prompt-cache usage in camelCase fields (`cacheReadInputTokens`/`cacheWriteInputTokens`) the proxy didn't parse, so cached Converse requests were under-billed. This fixes the parsing and makes cache accounting first-class: the cost meter emits the cache portion of each request's cost (`cost.usd_cache`), and `cached_input_tokens`, `cache_creation_tokens`, and `cache_cost_usd` are persisted in the agent-network access-log and request-usage tables and exposed in the REST API (entries, sessions, usage buckets), so cache-write-heavy requests are explainable instead of hiding inside `total_tokens`. ## Issue ticket number and link N/A ## Stack <!-- branch-stack --> ### Checklist - [x] Is it a bug fix - [ ] Is a typo/documentation fix - [x] Is a feature enhancement - [ ] It is a refactor - [x] Created tests that fail without the change (if possible) - [x] This change does **not** modify the public API, gRPC protocols, functionality behavior, CLI / service flags, or introduce a new feature — **OR** I have discussed it with the NetBird team beforehand (link the issue / Slack thread in the description). See [CONTRIBUTING.md](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTING.md#discuss-changes-with-the-netbird-team-first). > By submitting this pull request, you confirm that you have read and agree to the terms of the [Contributor License Agreement](https://github.com/netbirdio/netbird/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT.md). ## Documentation Select exactly one: - [x] I added/updated documentation for this change - [ ] Documentation is **not needed** for this change (explain why) ### Docs PR URL (required if "docs added" is checked) Paste the PR link from https://github.com/netbirdio/docs here: https://github.com/netbirdio/docs/pull/886 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added prompt-cache token counts and a full cost breakdown (input, cached-input, cache-creation, output) plus cache-attributed totals across access logs, sessions, and usage summaries. * Updated Bedrock/Converse parsing and streaming to include cache token fields. * Extended cost reporting metadata to include per-component USD values and `cost.usd_cache`; updated request token limits for e2e chat. * Added optional Bedrock model override for e2e workflow runs. * **Migration** * Migrates legacy aggregate cost data into the new per-bucket cost format while preserving historical totals. * **Bug Fixes** * Improved pricing, cost aggregation, and cost computation accuracy across providers. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
saavagebueno added the pull-request label 2026-08-05 07:08:44 -04:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DYNR/netbird#27452