mirror of
https://github.com/netbirdio/netbird.git
synced 2026-07-28 08:52:37 -04:00
3fac269506
Field report: a Bedrock Claude Sonnet 4.6 request showing 3 input / 1514 output tokens displayed a cost of $0.1372 instead of the expected $0.0227. Root cause analysis: the calculation is correct — the request's first call also wrote a ~30.5k-token prompt cache (cache_creation_input_tokens, billed at 1.25x input per AWS pricing), which is folded into total_tokens and the cost but not visible next to the input/output counts. Lock the pipeline down with tests so any real calculation regression fails loudly: - Add a provider cost matrix test driving the real proxy pipeline (llm_request_parser -> llm_response_parser -> cost_meter) with realistic wire fixtures for every metered surface (OpenAI JSON/SSE incl. cached subset, Anthropic JSON/SSE incl. cache buckets, Bedrock InvokeModel and Converse in both buffered and streaming form, Vertex path-routed, Kimi Anthropic-shape, unpriced gateway-prefixed ids) against the embedded default pricing table, asserting exact USD amounts derived from the published per-million prices. Covers the reported scenario byte-for-byte ($0.022719 bare, $0.137199 with the 30,528-token cache write) and would catch a per-token-instead-of-per-1k-chunk regression as a 1000x blowup. - Pin the management catalog (dashboard-displayed prices) to the proxy's embedded pricing table so the two can never drift apart silently. - Validate cost end-to-end in the live e2e provider matrix: each provider's ingested access-log row must carry a cost_usd matching the vendor's published per-1k rates applied to the row's token counts (cache-aware for the additive Anthropic/Bedrock buckets, zero for gateway-prefixed model ids the meter deliberately skips). - Fix a real under-billing bug the audit surfaced: the Bedrock Converse shapes report prompt-cache usage as camelCase cacheReadInputTokens / cacheWriteInputTokens, which neither the buffered parser nor the converse-stream metadata handler read - cached Converse traffic was metered without its cache buckets. Parse both fields into the same Usage buckets as the InvokeModel snake_case fields.