What TOLVYN does

The moat

The Tamper-Evident Ledger

Every AI request metered by TOLVYN is recorded in an append-only ledger. Each entry is hashed with SHA-256, chained to the previous entry, and signed with HMAC-SHA256. The result: a cryptographically verifiable record of every metered call your organization made to every AI provider. Retention never trims it. The retention job redacts request content in place and does not touch ledger records, so the chain stays complete and verifiable however old it is.

Metered is the operative word, and it is what makes the ledger worth having. A request that never touches the proxy is never recorded — TOLVYN can only attest to what it actually saw and charged. That boundary is deliberate: it is exactly what turns the ledger into a measuring instrument. Because the ledger holds precisely the spend your governance layer saw, subtracting it from the provider’s invoice leaves the spend it did not see. That difference is reconciliation, and the shortfall it exposes is shadow AI. A ledger that claimed to hold everything could not perform that subtraction — it would have nothing to compare against, and shadow AI would be undetectable by construction.

How the chain works

Each ledger record holds a prev_hash field pointing at the SHA-256 hash of the previous record's serialized payload. Any retroactive modification of any record breaks the chain. Verification re-walks the range and reports the first sequence number that does not link, so a break is located exactly rather than merely detected.

HMAC-SHA256 signature

On top of the hash chain, each record is signed with HMAC-SHA256 using a server-side signing key. Signature verification proves both integrity (the record wasn't modified) and authenticity (the record was written by TOLVYN, not forged by someone with read access to the database).

Advisory lock for sequence integrity

Sequence numbers are allocated under a Postgres advisory lock per tenant. There are no gaps and no duplicates — even under concurrent writes from multiple proxy workers.

Verification endpoint

# Verify the entire ledger for your tenant
$ tolvyn ledger verify
✓ Chain verified — 4,821 records, all hashes valid

# Or verify one range
$ tolvyn ledger verify --from-seq 4000 --to-seq 4821
✓ Chain verified — 822 records, all hashes valid

What it proves — and what it doesn't

The ledger proves what was billable: which request, at which timestamp, to which model, with which token counts, at which cost. It does not store prompt or response content. There is nothing in the ledger your finance team can't show an auditor.

Attribution

Six dimensions of cost attribution

Every metered request is tagged across six dimensions. Slice cost reports by any combination.

• Team — X-Tolvyn-Team — engineering, marketing, support
• Service — X-Tolvyn-Service — chatbot-api, search-svc, content-gen
• Feature — X-Tolvyn-Feature — autocomplete, summarize, classify
• Agent — X-Tolvyn-Agent — sdr-agent, support-bot, code-reviewer
• User — X-Tolvyn-User — stored exactly as you send it, not hashed. Send a pseudonym or an internal id if you do not want real user identifiers stored with us.
• End-customer — X-Tolvyn-End-Customer — your customer ID for COGS-per-customer

Hierarchy

Dimensions are independent: each is its own column, and you can group or filter by any of them. Only team and service are also written as a stored hierarchy path (org/<tenant>/team/<team>/service/<service>) — the other dimensions are not nested under it.

Budgets

Budgets & Enforcement

Set spending limits at four scopes — organization, team, service and agent. Choose how strictly they're enforced. Get pre-request cost estimation so the proxy can refuse a request before it hits the provider.

Three enforcement modes

Soft mode — the request goes through and an alert fires. Nothing is ever blocked. Hard mode — at the cap the proxy returns HTTP 429 with a x-tolvyn-budget header explaining which budget was hit, before the provider is called. Your application can fall back, queue, or surface an error to the user. Approval mode (approve-and-wait) — at the cap the request is blocked exactly like hard mode, but a pending approval is opened. An administrator grants a bounded extension — an extra dollar amount, valid for a bounded time — after which requests flow again until that grant's amount or time runs out. Governance without a permanent block: route the decision to a human instead of just failing.

Scope hierarchy

Budgets evaluate in order: agent → service → team → org. The first exhausted budget wins. So a $500/mo agent budget will hard-block even when the parent team budget has plenty of headroom — useful for runaway agent protection.

Pre-request cost estimation

Per-model pricing is known before the request goes out; token counts are estimated — input from the request size, output from the request's max_tokens or a default when it is absent. The proxy rejects a request whose estimate would push you over budget rather than incurring the charge and refusing the response.

Spend Quotas

Spend quotas & burn-rate forecast

Budgets enforce. Quotas give you foresight. A spend quota tracks spend along one dimension and alerts as it climbs toward a limit — it never blocks a request. Run them alongside budgets: the budget is the wall, the quota is the early-warning line painted well before it.

Dimensions & periods

Track spend per model, team, service, end-customer, or organization total, over a daily, weekly, or monthly period. Set any number of percentage thresholds — an alert fires as each one is crossed within the period.

Burn-rate forecast

Each quota projects whether you're on track to reach the limit before the period resets, by extrapolating recent spend. The result is one of on track, projected to hit the cap on <date>, already over, or not enough data yet.

It's an estimate — not a guarantee

The forecast is a straight-line extrapolation of a trailing window of spend. Real usage is spiky and non-linear, so the projected date is guidance, not a promise — it exists to warn you early, not to predict the future precisely. Quotas alert; they never block. Enforcement is the budget's job.

Alerts

Four alert types

• Budget threshold — fires at 50% (info), 75% (warning), and 90%/100% (critical) of any budget scope.
• Cost anomaly — fires when a request costs at least 10× the average of the last 100 requests for that service (e.g. someone shipped a code change that switched gpt-4o-mini → gpt-4o).
• Model change — fires when a service's traffic has genuinely moved to a different model, not when a single call differs. It needs at least 10 requests on the new model, mostly that model rather than a mix, still there 15 minutes later, and the cost per request at least 2× up or 50% down. The wait is deliberate: a detector that fires on any change is one you learn to filter, and we watched that happen on our own traffic — five alerts in nine seconds — before we narrowed it. So switching model for one call, or for a quick test, will not alert you.
• Pricing change — fires when a provider you depend on changes a model's price (see Pricing Change Governance).

Severity

Three levels: info, warning and critical. Severity is carried on the alert and on the webhook payload so you can route it in your own on-call tooling — TOLVYN does not page anyone itself.

Delivery

In-app · email · webhook. Webhooks are HMAC-signed (see below).

Webhooks

Webhooks

Subscribe to four event types, or use the alert.all wildcard.

• alert.budget_threshold — any budget threshold crossed
• alert.cost_anomaly — cost anomaly detected
• alert.model_change — service switched models
• alert.pricing_change — provider changed a model price
• alert.all — receive all of the above

HMAC-SHA256 signature verification

Every webhook payload is signed with HMAC-SHA256 using your endpoint secret. Verify the X-Tolvyn-Signature header before processing.

Retry policy with jitter

Failed deliveries (non-2xx response or timeout) are retried on a fixed schedule — 1 minute, 5 minutes, 30 minutes, then 2 hours — with ±10% jitter so a fleet of tenants does not retry at the same instant. Five attempts in total, after which the delivery is marked failed and no further attempt is made.

Reconciliation

Invoice Reconciliation

Every month, you receive an invoice from OpenAI, Anthropic, and Google. TOLVYN tells you whether what they billed matches what your applications actually requested.

Upload your invoice

Upload the CSV export from your provider dashboard. TOLVYN parses it, matches each line item against the ledger by model and date, and computes the gap.

Gap = Invoice − TOLVYN

A positive gap means you were billed for usage that didn't pass through the TOLVYN proxy. That's shadow AI: applications using provider keys without your observability. The reconciliation report names the model, the gap amount, and the date range — so you can hunt down the source.

Savings

Savings Analyzer

Nightly at 02:00 UTC, TOLVYN runs a savings analysis over the previous 30 days of traffic and recommends concrete, dollar-quantified migrations.

Four rules

• Small-token requests on premium models — if more than 100 of your gpt-4o requests in the period used fewer than 500 tokens, and moving them to gpt-4o-mini would save at least $1/month, you get a finding. The saving is computed from our live rate table at the moment the finding is written, not from a stored ratio.
• Duplicate prompts — identical prompt signatures should hit cache, not the provider.
• Underutilized prompt cache — long system prompts that don't trigger Anthropic's cache reuse.
• Idle models — models you provisioned but barely use; consolidation simplifies governance.

Output

Each recommendation includes the affected service and model, its current and optimized cost, the estimated saving, and the recommendation text. You can dismiss a recommendation from the dashboard.

Kill switch

Kill Switch

A panic button for AI traffic. The proxy returns HTTP 451 ("Unavailable For Legal Reasons") for any request matching a kill-switch rule. Checked before budget enforcement, so killing a runaway agent always works even if all budgets are exhausted.

Five scope types

• Org-wide — stop all AI traffic immediately
• Team — stop all traffic from a specific team
• Service — stop all traffic from a specific service
• Agent — stop a specific agent (most common)
• API key — stop a single key without touching the rest

Activation

Dashboard, CLI (tolvyn kill agent sdr-agent), or API. The identity that activated a switch is recorded with it. Deactivation requires an admin — a member who can arm a kill switch cannot necessarily disarm it, which is deliberate.

Pricing changes

Pricing Change Governance

Provider pricing changes silently. Sometimes a new model is cheaper. Sometimes a model gets repriced upward. Sometimes a model is deprecated and routed to a more expensive replacement. TOLVYN monitors all of this and tells your customers before they're surprised.

Daily scrape

Every day TOLVYN automatically checks the public pricing pages of every supported provider. Diffs are captured, persisted, and surface in the operator console.

Operator approval workflow

A detected change does not auto-update billing. A TOLVYN operator reviews the diff, confirms it's a real change (not a transient page error), and approves. Only then is it propagated to customer accounts.

Customer impact notifications

Any customer whose recent usage included the affected model receives an alert with exact dollar impact estimate: "Your projected monthly cost on gpt-4o-mini goes from $1,240 to $1,615 (+30%) at the new $0.30/M output rate."

Cost index

AI Cost Index

Opt-in benchmarks aggregated from real production AI traffic across TOLVYN customers. Free, no signup required. A figure is published only when at least three organisations contribute to that bucket — a privacy floor applied during collection, so the index shows only what clears it. See cost-index.html for the current index and the full methodology.

k-anonymity ≥ 3

A data point is published only if at least three independent tenants contribute to it. Below that threshold, the cell is suppressed.

Opt-in

Off by default. Nothing you send through TOLVYN reaches the index until you turn on “Contribute to AI Cost Index” in account settings, and you can turn it back off there at any time. Turning it off excludes you from every later nightly collection. Figures already published are aggregates of at least three organisations with no tenant identifiers in them, so they stay as published — there is nothing in a published figure to unmix.

Apache 2.0 data

The published aggregates are released under Apache 2.0. Use them in your own analyses, research, blog posts, or vendor evaluations. Attribution appreciated but not required.

Integration

SDKs & Integration

• Python — tolvyn 0.1.8 — drop-in replacement for openai, anthropic, and Google's GenAI client.
• Node.js — tolvyn 1.0.10 — drop-in replacement for openai and @anthropic-ai/sdk.
• Go — tolvyn-go v0.1.4 — idiomatic Go bindings.
• CLI — 56 commands and subcommands across ledger, budgets, alerts, agents, kill-switch, reconciliation, and savings.
• Direct HTTP — change the base URL, keep your existing request code.

Documentation

Full reference at docs.tolvyn.io including the API reference and the CLI reference.

Evaluating TOLVYN against an observability-first gateway? See TOLVYN vs Helicone.