Billing & Usage Metering
Why this exists
Section titled “Why this exists”Logrite spends real money on every AI call — LLM-backed analysis, /code-fixes,
/analyze-exceptions, Cortex chat with tool calling, scheduled agents and the AI Warden judge.
Before this module, none of it was metered, attributed to a workspace, or charged for.
Two measured facts shaped the design:
-
Log generation is not where the money goes.
POST /analyzeand/analyze/auto— the 27-strategy engine covering 28 languages — are deterministic AST parsing with no LLM call. A repo scan costs CPU, not tokens. Spend concentrates in chat (up to ~15 LLM calls per turn),/fix-codeand/code-fixes(max_tokens65 000 and 32 000), the backend’s unboundedPOST /api/v1/codeFixes/autorepair loop, and the Warden judge ladder. -
Most spend was invisible. No streaming call requested usage — there is no
stream_options: {include_usage: true}anywhere — andAnthropicProvider.streamChatCompletiondroppedmessage_delta.usage. Non-streaming usage was captured intometadata.tokens_usedand then discarded.
The unit: micro-USD, not credits
Section titled “The unit: micro-USD, not credits”Usage is presented to customers as raw input/output/cached token counts and a dollar figure — no abstract credit, and no per-model multiplier table. The industry moved away from multipliers in 2026 (GitHub retired its 0.25x–57x table on 2026-06-01 in favour of direct token billing; Windsurf dropped credits entirely in March) because cost varies ~100x within a single model on agentic workloads — which is exactly the shape of a repo scan, where a 4-file service and a 500-file monorepo are the same “action”.
The ledger unit is an integer micro-USD:
- Integer, never floating-point. A float ledger does not add up.
- Micro-USD, not tokens, because a token is not a unit of value: input and output differ ~5x, and models differ ~100x. Storing tokens would force re-derivation on every read against a rate card that changes.
- Displayed as tokens + dollars, which is what “no abstraction” means to the user.
A tier’s allowance is therefore “$40 of included AI usage per engineer”, not “N credits”.
Data model
Section titled “Data model”| Table | Role |
|---|---|
billing_plan | Plan catalog — prices in cents, allowance in micro-USD, provider product ids |
billing_entitlement | (plan_code, feature_key, type) → BOOLEAN | LIMIT | METERED |
workspace_subscription | One row per workspace: plan, status, enforcement_mode, overage_policy, spend cap |
billing_period | UNIQUE(workspace_id, period_start) — this constraint is the grant lock |
usage_ledger_entry | Append-only record of truth; signed micro_usd |
usage_balance | Derived projection, so the gate is one indexed row read |
usage_event | Raw meter rows; doubles as the provider outbox via provider_synced_at |
ai_model | Model registry — capabilities and params as data |
ai_model_role | Role → model, ranked. Ladders become reviewable rows |
ai_model_rate | Effective-dated rate card, with a basis of TOKEN or ENERGY |
Schema arrives via ddl-auto: update; the reviewable mirror is
logwrite-api/src/main/resources/migrations/2026-billing.sql.
Grant burn-down order
Section titled “Grant burn-down order”Grants carry priority, expires_at, and rollover bounds. Burn order is:
priority ASC → expires_at ASC NULLS LAST → created_at ASC → id ASCWithout it, an expiring promotional grant outlives the paid balance it should have been spent before, and the customer loses credit they bought.
The spend gate
Section titled “The spend gate”An LLM call’s cost is unknown until it finishes, so metering uses the payments authorize → capture → void pattern:
reserve(estimate)— atomically holds budget, or refusescapture(actual)— converts the hold to consumption, releasing the differencerelease(estimate)— voids the hold when the call produced nothing
The gate itself is a single atomic conditional UPDATE:
UPDATE usage_balance SET reserved_micro_usd = reserved_micro_usd + :amount WHERE workspace_id = :ws AND billing_period_id = :periodId AND (included_micro_usd + purchased_micro_usd - consumed_micro_usd - reserved_micro_usd) >= :amountZero rows updated means insufficient allowance.
There is no Redis in the backend (ten Spring starters, none of them data-redis). Adding one
for this would buy a new dependency, a new failure mode and a cache-coherence bug. Postgres already
does it correctly.
A cancelled stream is not a void
Section titled “A cancelled stream is not a void”The provider generated those tokens and billed us for them. Capture what was actually produced — otherwise “cancel every request” is a free ride.
Enforcement is per-workspace, and starts in observe mode
Section titled “Enforcement is per-workspace, and starts in observe mode”enforcement_mode defaults to OBSERVE, which logs what would have been blocked without
blocking it. This mirrors how the OpenSearch tenant-isolation cutover was de-risked
(opensearch.proxy.enforce-jwt shadow mode).
It matters because every existing workspace is dropped into a trial when billing goes live — a global hard cutover would lock real users out on day 15.
overage_policy is an explicit field rather than inferred from status: a trial has no card and
therefore cannot auto-buy overage, while a paid tier can.
Mode flag
Section titled “Mode flag”logrite: billing: mode: ${LOGRITE_BILLING_MODE:disabled} # disabled | saas | onprem enforcement: ${LOGRITE_BILLING_ENFORCEMENT:observe} # observe | enforce provider: ${LOGRITE_BILLING_PROVIDER:noop} # noop | polar| Mode | Metering | Gate | Provider | Plan source |
|---|---|---|---|---|
disabled | off | open | noop | unlimited internal plan |
saas | on | per enforcement | polar | workspace_subscription |
onprem | local only | per licence | noop | signed licence file |
When disabled, endpoints return 200 with status: "DISABLED" rather than 404, per the house
convention — so the frontend degrades to demo fixtures instead of erroring.
The browser-facing API
Section titled “The browser-facing API”BillingController, @RequestMapping("/api/v1/billing"). There is no servlet context path, so
these are the full paths. All five are what Settings → Billing calls.
| Method | Path | Notes |
|---|---|---|
GET | /api/v1/billing/overview | everything the tab renders, in one call |
GET | /api/v1/billing/usage | the usage block alone |
PUT | /api/v1/billing/spend-cap | Maintainer only |
POST | /api/v1/billing/checkout | {planCode, billingCycle} → {url} to redirect to |
POST | /api/v1/billing/portal | → {url} for the provider’s portal |
POST | /api/v1/billing/webhooks/polar | provider → us; PolarWebhookController |
Authorization is inside the controller, not the filter chain. /api/v1/** is permitAll() in
SecurityConfiguration, and requireWorkspace() throws 401 for a missing bearer token or a
token carrying no workspaceId claim when the user has more than one workspace. So a 401 here
means “sign in again to refresh the session” at least as often as it means “not logged in”.
overview — which keys are optional
Section titled “overview — which keys are optional”This is the part that broke a UI, so it is worth stating precisely. Only these keys are always present:
status · mode · plans · rateCard · usage
subscriptionandbalanceare added only if a row exists (subscriptionRepo.findByWorkspaceId(ws).ifPresent(...)). No subscription → the keys are absent, not null. A client must not assumesubscription.periodStart,activeEngineers,daysElapsed,daysInPeriod,nextInvoiceAmountornextInvoiceDateexist. The billing period is available regardless asusage.periodStart/usage.periodEnd, which are derived from the calendar month.invoicesandpaymentMethodare never sent by any endpoint. They exist only inbillingDemoData.js. Billing history and the card-on-file footer are therefore permanently empty against a live backend, and the UI renders empty states for both.GET /api/v1/billing/invoices/{id}/url— there is no such endpoint.billingApi.jshas agetInvoiceUrl()for it; it would 404, and it is not wired to any button.usageusesMap.of(...), so its keys are always present;features,dailyandescalationare empty arrays when nothing was metered./usageignores theperiodStartquery param the frontend sends and always reportsYearMonth.now().
rateCard is grouped by path, two levels deep
Section titled “rateCard is grouped by path, two levels deep”"rateCard": [ { "path": "standard", "label": "Standard", "models": [ { "model": "deepseek-v4-flash", "provider": "neuralwatt", "inputPerM": 0.70, "cachedPerM": 0.15, "outputPerM": 1.40 }, { "model": "gemma-4-31b", "provider": "neuralwatt", "inputPerM": 0.70, "cachedPerM": 0.05, "outputPerM": 2.10 } ]}, { "path": "advanced", "label": "Advanced", "models": [ ... ] }]The price triple lives on each model, not on the group — a path publishes several models at
different prices, and blending them into one row per path both hides that and cannot be
reconciled against an invoice line. The group carries no description; that copy is fixture-only.
The provider seam
Section titled “The provider seam”| Logrite owns, permanently | Delegated behind BillingProvider |
|---|---|
| Plans, entitlements | Payment methods, card vaulting |
| Spend ledger and balances | Tax calculation, filing, remittance |
| Usage events, meters, rollups | Invoices, receipts, dunning |
| Seat counting, quota enforcement | Hosted checkout + customer portal |
Three separate requirements all need a subscription the provider knows nothing about — air-gapped
on-prem, enterprise deals on PO/net-30 terms (the provider is card-only), and a card-less trial
(its checkout always collects a card). One mechanism serves all three: a subscription whose
provider is NoopBillingProvider.
Model registry — changing models without a deploy
Section titled “Model registry — changing models without a deploy”Model choice used to be scattered across env vars in three services (LLM_MODEL,
LLM_CHEAP_MODEL, LLM_FALLBACK_MODELS, AI_PROVIDER, WARDEN_JUDGE_LADDER), so a change meant
editing several Coolify apps and redeploying each — and nothing knew what any model cost.
Resolution is a four-layer ladder, each layer complete on its own:
1. per-workspace BYO credential (ByoLlmService — already exists)2. env var override (today's mechanism, kept)3. registry rows (ai_model_role, scoped global | plan | workspace)4. compiled-in seed defaults (BillingCatalogSeeder)| Situation | Resolves from |
|---|---|
| Local dev / tests | seeds or env — no backend required |
| Deploy / bootstrap | seeds ship in the image; registry supersedes on first fetch |
| SaaS steady state | registry rows — one row update, no redeploy |
| On-prem | the licence file carries the registry snapshot; env overrides for BYO-key or a self-hosted model (ollama / lmstudio are already supported) |
Capabilities become data too: NeuralWattProvider strips markdown fences because that provider
reports json_mode: false, so it becomes capabilities.jsonMode. The determinism contract
(temperature 0, seed 7, top_p 0.9, max_tokens 8000) travels in params with the model,
because those values are per-model, not global.
The escalation ladder is a cost multiplier
Section titled “The escalation ladder is a cost multiplier”Priced against one representative analyzer call (15k input at 75% cache hit, 3k output):
| Model | cost/call | vs default | JSON mode |
|---|---|---|---|
deepseek-v4-flash | $0.00170 | 1.0x | ✗ |
gemma-4-31b | $0.00190 | 1.1x | ✗ |
qwen3.6-35b | $0.00488 | 2.9x | ✓ |
kimi-k2.7-code | $0.01669 | 9.8x | ✓ |
glm-5.2-short | $0.02051 | 12.0x | ✗ |
kimi-k3 | $0.05963 | 35.0x | ✓ |
ResilientProvider retries the whole prompt on the next tier for any 429/5xx, and the judge
escalates whenever confidence falls below threshold — so a transient upstream blip silently
multiplies COGS, and both attempts are paid for. Ladder depth is therefore a plan entitlement
(judge.ladder_depth), not a global env var, and usage_event.escalation_tier makes the
escalation rate visible.
The outbox must not run nightly
Section titled “The outbox must not run nightly”usage_event.provider_synced_at is null until shipped, so the table is its own outbox and there is
no second table to keep consistent.
UsageOutboxDrainService implements this as two schedules. drain() ships a page every 5s.
flushBeforePeriodEnd() runs every minute, does nothing unless an open period actually ends
within the next 15 minutes, and then drains to empty.
The stamping rule is what makes it safe to ship unarmed
Section titled “The stamping rule is what makes it safe to ship unarmed”Rows are marked sent only when the provider accounted for every record, whether inserted or already held:
int accounted = result.accepted() + result.duplicates();if (accounted < records.size()) return 0; // leave the whole batch unsentNoopBillingProvider.ingestUsage returns (0, 0), so with no Polar token configured the drain
runs and marks nothing. Stamping on “the call did not throw” would silently mark every usage
row as shipped to a provider that does not exist, and destroy the evidence that it never was.
Re-sending a whole batch is free — the provider deduplicates on our event_key — so leaving three
records unsent is strictly safer than stamping the two we believe landed.
Reconciliation, because the provider will not tell us
Section titled “Reconciliation, because the provider will not tell us”Polar publishes no usage, meter or balance webhooks — all 41 event types were enumerated and the closest signal in the catalogue is a benefit refill. It also does not notify a merchant when meter-cycle order retries are exhausted (polarsource/polar#12904).
So a customer’s metered usage can stop billing entirely and the only symptom is an invoice that is
quietly too small. PolarReconciliationService runs daily against
GET /v1/meters/{id}/quantities and compares against our own usage_event sums.
Drift is reported in both directions and neither is benign:
| Direction | What it means |
|---|---|
| Polar lower than us | Events were rejected, lost, or landed in another cycle — revenue that will never be invoiced |
| Polar higher than us | Something is sending events we have no record of — a customer is being overcharged |
Significance needs a floor and a ratio. The floor alone lets a large account’s real 1% gap hide under a threshold tuned for small ones; the ratio alone pages on rounding.
The monthly grant, without a distributed lock
Section titled “The monthly grant, without a distributed lock”@EnableScheduling is on and there is no Quartz, no ShedLock and no distributed lock of any
kind, so every replica runs every @Scheduled method. Existing jobs tolerate that by being
idempotent upserts — granting allowance is not naturally idempotent and would multiply by the
replica count.
The UNIQUE(workspace_id, period_start) constraint on billing_period is the lock: the grant
is an insert that only one replica wins, and only the winner credits the balance. Note also
spring.lifecycle.timeout-per-shutdown-phase: 8s — the job must be resumable, not long-running.
BillingPeriodService runs this daily, not monthly. A monthly trigger fires once; if the
process is down at that moment, or a deploy lands on it, the month is skipped and every customer
quietly loses their allowance until somebody notices. Asking “whose period ended?” every day makes a
missed run self-healing, and periods are keyed by date so a repeat pass cannot double-grant.
How much is granted
Section titled “How much is granted”Allowance is per seat, matching how plans are priced — $15 / $40 / $100 a month on Team / Pro / Enterprise — floored at the plan’s committed minimum. A trial gets its flat block instead, because it has no seat commitment; that is the point of the “14 days plus a fixed grant” shape, which teaches the buyer the unit before they have to price it.
Seats are counted retroactively, per the published definition: an active engineer is someone who signed in or had a commit analysed. That is why the number can only be taken at roll time.
PAST_DUE still receives its grant. It is a grace window, not a cancellation, and withholding
allowance the moment a card bounces turns a billing hiccup into an outage in the middle of
someone’s incident. Revocation is the real cutoff.
Failure policy
Section titled “Failure policy”| Situation | Behaviour | Why |
|---|---|---|
| Billing disabled | everything permitted | adding the module changes nothing until turned on |
| Entitlement row missing | fail open | a wrong “no” costs a paying customer access; a wrong “yes” costs an upsell |
| Model has no rate | fail expensive — falls back to the priciest known rate, loudly | a lookup miss returning zero turns an outage or a typo into free inference at scale |
| Registry unreachable | keep last good snapshot, fall through to env then seeds | a billing-database outage must never take down inference |
| BYO-key workspace | metered, charged nothing | the customer already paid their own provider |
Where usage is actually captured
Section titled “Where usage is actually captured”Metering reads token counts at the point each provider hands them over, and attributes them
through a request-scoped AsyncLocalStorage rather than a parameter threaded through every
call site. The pattern is borrowed from retrievalTrace.ts, which already existed for the
same reason.
The streaming fix is three different fixes
Section titled “The streaming fix is three different fixes”Roughly 90% of chat spend used to be invisible, because no streaming path returned usage at
all. “Add stream_options” is only correct for one of the three providers:
| Provider | How usage arrives | What was dropping it |
|---|---|---|
| OpenAI-compatible (platform default) | needs stream_options: {include_usage: true}; the final chunk carries usage with an empty choices array | if (!choice) return discarded exactly that chunk |
| Groq | ⚠️ groq-sdk has no stream_options — usage rides on chunk.x_groq.usage | if (!choice) continue, same shape |
| Anthropic | no flag needed: message_start.usage.input_tokens + message_delta.usage.output_tokens | there was no message_start branch at all, and message_delta read only stop_reason |
All three return usage: undefined when no usage frame arrives — never zeros. A zero asserts
a measured free call; undefined says “unmeasured”, and a ledger must be able to tell those apart.
Meter in exactly one place
Section titled “Meter in exactly one place”AgentLoop already sums usage across its iterations. Metering at the provider decorator
and reading AgentLoopResult.usage counts everything twice, so the decorator is the only
writer and the loop’s total is a cross-check.
The decorator wraps at AIProviderFactory.buildResilient() (outside the nested
ResilientProvider chain, so one logical call is one record) and at
providerResolver.buildBundle() for BYO. Escalation tier is derived by matching
result.model against the ladder position rather than plumbing a counter — every provider
already propagates model faithfully.
⚠️ generateCompletion returns a bare string, so a decorator cannot see its usage: the
provider discards the SDK response first. In the analyzer, each provider reports usage
directly instead — four one-line additions, no interface change. Guardrail screening and the
summarizers spend on that path in the ai-service and remain unmetered for now.
The relay and the judge
Section titled “The relay and the judge”- Non-streaming usage was already in memory and thrown away: the relay parses the body to
extract completion text and never looked at
parsed.usage. It now writes exactlyresponse.usage.total_tokensas an integer, which is the pathWardenQueryService.getStats()has always aggregated — so the existing stats endpoint and its UI tile light up with no backend change. - Usage is reported for any body that parses, not only ones with a
choicesarray. Anthropic’s/messagesputs content in an array of blocks, and/embeddingsand/moderationshave a top-levelusageand nochoicesat all. - SSE needed a separate tail buffer. The existing tee keeps the first 64 KB for the compliance scan, and the usage frame arrives last — a head-capped buffer can never see it.
- The judge is metered inside its provider-invocation helper, the only point that sees the ladder loop, the resilience fallback and the 429 retries. One logical judge can be several billable calls, so a period total is a sum across tiers, and cache hits and single-flight joins record zero — they never reach the provider.
Attribution across the service boundary
Section titled “Attribution across the service boundary”The analyzer had no tenant on any LLM endpoint. AIAdapterService.applyAnalyzerAuth() — which
all eight of its analyzer methods route through — now sends X-Logrite-Workspace, and the
analyzer’s auth middleware reads it into an ALS store. An InheritableThreadLocal carries it
on the Java side, because that service is a singleton and LogGeneratorUtil fans calls across
a worker pool.
A request with no workspace header still runs; its usage is recorded unattributed. Scheduled scans and CLI runs have no workspace and still cost money, so an unattributed row is worth more than no row.
Configuration
Section titled “Configuration”| Variable | Service | Default | Purpose |
|---|---|---|---|
METERING_ENABLED | ai-service | true | Kill switch. Removes the decorator rather than disabling it from inside. |
METERING_VERBOSE | ai-service | false | Per-call logging. |
LLM_STREAM_USAGE | ai-service | true | Gates stream_options — the only change that touches a request body, so it can be turned off if a gateway rejects unknown fields. |
LOGRITE_BILLING_MODE | backend | disabled | disabled | saas | onprem |
LOGRITE_BILLING_ENFORCEMENT | backend | observe | observe logs what would have been blocked. |
LOGRITE_BILLING_PROVIDER | backend | noop | noop | polar |
LOGRITE_CODEFIX_MAX_RUN_MICRO_USD | backend | — | Ceiling reserved before an autoFix run starts. |
LOGRITE_CODEFIX_MODEL / _PROVIDER | backend | — | Which rate card prices the pre-flight estimate. |
AI_SERVICE_LEAK_PATTERNS | analyzer | — | Points the warden drift guard at the relay’s copy from a worktree. |
The drain: POST /internal/billing/usage
Section titled “The drain: POST /internal/billing/usage”Both producers buffer in memory and a worker ships batches to the backend. The buffer is bounded (10k in the ai-service, 5k in the analyzer) and drops oldest-first past the cap.
Auth is X-Billing-Internal-Token, constant-time compared, refusing everything when
unconfigured. Deliberately not the relay’s X-Warden-Internal-Token: three services post
here, and reusing that secret would widen what a leak of it reaches.
Body is NDJSON (one record per line) or a JSON array. The endpoint accepts both producer
dialects — the ai-service sends byo + epoch-millis at, the analyzer sends byoKey + ISO
occurredAt — and ignores unknown fields, so a producer ahead of the backend does not lose
its batch.
Response is {status, received, inserted, duplicates, skipped}.
Idempotency is on eventKey, which has a UNIQUE constraint. Neither producer mints one
yet, so the backend derives auto-<sha256[:40]> over the record’s identifying fields — a
replay hashes identically and is counted as a duplicate rather than charged twice. The
honest cost: two genuinely distinct calls identical in every field including the
millisecond collapse into one row. Under-counting a handful of calls beats double-charging a
customer, and it disappears once producers mint their own keys.
⚠️ usage_event.event_key is VARCHAR(128), and the ledger row it writes is keyed on
"usage:" + eventKey against a VARCHAR(128) idempotency_key. The cap is therefore 122,
not 128 — otherwise a 128-character producer key builds a 134-character ledger key, the
DataIntegrityViolationException is swallowed as a routine replay, and the workspace has
already been charged with no ledger row behind it.
Configuration
Section titled “Configuration”| Variable | Where | Purpose |
|---|---|---|
METERING_ENABLED | ai-service, analyzer | Kill switch for recording. |
METERING_DRAIN_ENABLED | producers | Kill switch for shipping, separate from recording. |
METERING_BACKEND_URL | producers | Unset ⇒ buffer only, logged once. |
METERING_INGEST_TOKEN | producers | Must equal LOGRITE_BILLING_INTERNAL_TOKEN. |
METERING_BATCH_SIZE / METERING_FLUSH_MS | producers | ~50 records / ~5 s. |
LOGRITE_BILLING_INTERNAL_TOKEN | backend | Unset ⇒ the endpoint refuses everything. It does not fail open. |
LOGRITE_BILLING_MODE | backend | disabled discards batches and answers status: "DISABLED". |
Bounding codeFixes/auto
Section titled “Bounding codeFixes/auto”autoFix was unbounded in three separate ways: the inner for over failing files had no cap,
caller-supplied maxFiles had no upper clamp, and supplying explicit levels returned before
the clamp was reached, bypassing it entirely. At 32 000 output tokens per call, one request
could buy thousands of them.
A budget is now reserved up front through the existing UsageLedgerService.reserve(), which
already handles observe mode, auto-buy and spend caps. Exhaustion mid-run returns the fixes
already applied with a clear message — never a 500, never a silent truncation.
The pattern is borrowed from AgentLogScanner’s JUDGE_MAX, which was the only LLM budget
anywhere in either codebase.
Testing
Section titled “Testing”mvn test in logwrite-api. The tests that carry the design:
UsageLedgerServiceTest#concurrentReservesCannotOversell— 24 racers, budget for 23, exactly 23 succeedUsageLedgerServiceTest#captureIsIdempotent— a replay releases the hold without double-chargingUsageLedgerServiceTest#grantIsIdempotentAcrossReplicas— two replicas grant onceRateCardServiceTest#unknownModelFailsExpensive— an unpriced model is never freeRateCardServiceTest#effectiveDatingIsRespected— history re-derives at the old rateBillingControllerTest#missingTokenIsUnauthorized—/api/v1/**ispermitAll(), so the controller’s own guard is the authorization