Pricing Engine

Note

Generated from ONBOARDING.md — the design authority for this project. Edit the source file, not this page.

Lookup priority chain

estimate_cost(usage, model, provider="", core_price=None) resolves a price by trying these in order:

1. Custom YAML (~/.hermes/telemetry/pricing.yaml, models: section), only when the entry is _subscription: true (a declared flat rate) or the model id ends in :free — these two are the only entries that outrank a core-sourced snapshot, because the core has no way to know about a user’s flat-rate deal, and there is no verified guarantee the core resolves a :free-suffixed id to $0 rather than its paid base rate.
2. Core-sourced snapshot (core_price) — the tariff Hermes core itself resolved for this exact (provider, model) pair, read from the pricing_snapshots table (a local SQLite read — the caller in post_api_request passes it in; see § Core-sourced pricing snapshots below). Present only once a snapshot has been captured for the pair; a cold-start (never-seen) pair has none yet.
3. Custom YAML (any remaining entry — a plain hand-added override or an auto-fetched OpenRouter entry) — exact match, case-insensitive.
4. Built-in table (_DEFAULT_PRICING) — exact match.
5. Prefix fallback — scans all of the above plus _PREFIX_PRICING, longest prefix wins.
6. Google symmetric form — if the above misses, tries gemini-X ↔ google/gemini-X.
7. Unknown → returns $0.00, logs a one-time WARNING.

:free suffix rule (issue #32, #54): OpenRouter advertises free-tier variants with a :free suffix (e.g. nvidia/nemotron-3-ultra-550b-a55b:free). These are $0 by definition. The rule sits in _lookup_form before both the prefix scan and core_price, for two reasons: (1) otherwise a suffixed free id inherits its paid base’s price via prefix (or, now, via a core snapshot resolved for the paid base) — billing a free call as paid; (2) returning an explicit zero dict (not the unknown-model None) makes the call resolve as known-free — no estimated-price warning, and it’s recorded in known_free_models so the free→paid alert fires when the gateway later drops the :free suffix. A user’s explicit :free entry (step 1) still overrides the rule.

Every candidate below step 2 is still filtered by the provider-aware guard (below); core_price needs no such guard — it was already resolved for this exact provider by the caller.

Why longest-prefix-wins: gpt-4o-mini must not be matched by gpt-4o or gpt-4. o1-mini must not be matched by o1. The sorted-by-length approach (candidates sorted by -len(prefix)) ensures specific prefixes take precedence.

Provider-aware lookup guard (added v0.4.1, issue #24)

Problem: pricing.yaml is auto-populated from the OpenRouter catalog — every fetched entry carries _source: openrouter. But the same model name can be served by a different provider at a different price. Two real cases:

• Nous Portal serves qwen3.7-plus (bare id) on a flat subscription ($0), while OpenRouter lists qwen/qwen3.7-plus (prefixed) at a real per-token rate.
• NVIDIA NIM serves nvidia/nemotron-3-super-120b-a12b at $0.10/$0.50 while OpenRouter lists the same exact id at $0.09/$0.45 (a true key collision).

A provider-blind, model-name-only lookup silently costs a Nous/NIM call with the OpenRouter rate — a plausible-but-wrong number. _source was decorative until this guard made it load-bearing.

Rule (_source_eligible(source, provider)):

Entry source	Eligible for which calls
_source: openrouter	only provider=="" (backward-compat/unknown) or a provider containing "openrouter"
no _source (_DEFAULT_PRICING, _PREFIX_PRICING, hand-added overrides)	always (provider-neutral)
other named source (e.g. google-ai)	always (direct-provider rate, reasonable for any caller of that id)

provider is the verbatim string from post_api_request (e.g. "nous", "openrouter"; NIM canonicalizes to "nvidia" — aliases nim/nvidia-nim/ nemotron are normalized by Hermes normalize_provider() before the hook fires, verified against hermes_cli/providers.py).

Inverted safe-default for the lookup path (added v0.6.1, issue #42): the guard’s original failure mode was “fail to zero” — if the only candidate was a source-ineligible OpenRouter entry, the lookup returned None and the call recorded cost=0. For a cost-tracking plugin this is the worst failure: a real paid call (e.g. Nous Portal reselling moonshotai/kimi-k2.6 at the OpenRouter rate) silently reads as zero spend. The lookup now errs on the side of recording cost: when no eligible candidate matches but a source-ineligible one would have, _lookup_form returns that price tagged _provider_assumed: True, and estimate_cost logs a one-time WARNING per (model, provider) pair advising the user to pin the rate. Eligible matches, _DEFAULT_PRICING seeds, the :free rule, and source-neutral / _subscription overrides all win first — the assumed fallback only fires when the sole available price is one the guard would otherwise reject. This means a genuine flat-sub / wrong-rate collision must be protected by a source-neutral _subscription (or hand-added) entry; without one, the call now over-counts-with-warning instead of under-counting-silently — the deliberately safer default for a cost tracker.

Why NIM seeds live in _DEFAULT_PRICING (code), not the YAML: for the same-id collision, the OpenRouter entry in models: is excluded for a provider="nvidia" call, and the lookup falls through to the source-neutral seed in _DEFAULT_PRICING. Keeping seeds in code also makes them immune to an OpenRouter sync overwriting the shared key. The :free promo variants (nvidia/...:free) carry no seed — they resolve to $0.00 via the :free suffix rule above (issues #12/#32). (Before that rule existed, a :free variant of any seeded model was mis-priced at the seed’s paid rate via prefix match — the earlier “resolve to $0 via the unknown-model fallback” claim only held for models with no seeded paid base.)

Pricing metadata tags (_-prefixed)

Three _-prefixed keys live on pricing entries and are stripped from the price dict by _load_custom_pricing (captured into parallel structures model_sources / subscription_models so the price math never sees them):

Tag	Meaning	Effect
_source	Which source wrote the entry (openrouter, google-ai, …)	Drives the provider-aware guard
_estimated_price	OpenRouter model with no fixed price (negative→$0)	Counted by estimated_price_share; degrades hard budget verdicts to soft under warn_only
_subscription	A declared $0 (flat-sub / free-tier) rate, hand-added	Distinguishes a genuine $0 from a lookup miss; tracked in _meta.subscription_models; excluded from estimated_price_models

A fourth key, _provider_assumed, is not a stored YAML tag — it is a runtime-only marker synthesized by _lookup_form (issue #42) when a source-ineligible entry is applied as a best-effort estimate. It rides on the resolved price dict purely so estimate_cost can fire its one-time provider-assumed warning; _load_custom_pricing never reads or writes it.

The marker is also surfaced (not just logged): post_api_request calls pricing.is_provider_assumed(model, provider) and persists a per-call boolean to the DB — llm_calls.provider_assumed plus a runs.provider_assumed_calls counter (schema v6, db._migrate_v6). It mirrors the per-call estimated flag end to end: /stats providers shows an Asm% column, and the dashboard (both the standalone serve.py and the Hermes plugin) exposes it as an Asm? column in Requests and an Assumed count in Providers (provider_assumed_calls / provider_assumed_pct in the API). Deliberately unlike _estimated_price, a provider-assumed cost does NOT degrade budget verdicts — the resold-at-the- same-rate case is usually the correct number, so it counts as real spend for enforcement (the issue’s “err on the side of recording cost”); the flag exists only to make the assumption visible so the user can pin the rate.

Adding a subscription/flat-rate model: enter it under the provider’s native (bare) id with input: 0.0, output: 0.0, _subscription: true. The bare id is never returned by OpenRouter, so it never collides with an auto-fetched entry and survives every refresh untouched.

/stats models $0 classification (three-way). The Notes column disambiguates every $0.00 row so the footer never nags about a deliberate zero. stats._models_block splits them, in order:

1. subscription/free-tier — exact-id membership in subscription_models (a declared _subscription entry).
2. free tier — pricing.is_explicitly_priced(model, provider) is True but it is not a declared subscription: a :free suffix id (resolved to $0 by the rule above) or a built-in $0 seed. Known-free, not a lookup miss.
3. no price entry — genuine unknown (is_explicitly_priced is False).

Gotcha — _subscription is the wrong tool for a :free-served model. When the gateway serves the model with a :free suffix — and gateways send dated ids, e.g. tencent/hy3-20260706:free — the :free rule already resolves it to $0, records it known-free, and arms the free→paid alert, so no pricing.yaml entry is needed and the row shows as free tier. _subscription is only for a model served free under a bare native id (no :free), and the entry must match the exact id the gateway records: a non-dated key (tencent/hy3:free) never matches the dated id, so it neither prices nor labels the row. Both the cost lookup (subscription_models / custom exact match) and the stats label key on the exact recorded id — there is no prefix or date-stripping normalization for either.

Google-symmetric normalization (added v0.4.0)

Problem: The gateway reports model IDs differently depending on routing path:

• Direct Google AI: gemini-3.5-flash (no prefix)
• Via OpenRouter: google/gemini-3.5-flash

A single pricing YAML entry would only cover one form. Having the user maintain both was error-prone.

Solution: _google_alt_form(model_lc) maps the pair symmetrically:

• gemini-X → try also google/gemini-X
• google/gemini-X → try also gemini-X

Deliberately google-specific: anthropic/, meta-llama/, openrouter/ are NOT normalized. On OpenRouter, anthropic/claude-sonnet-4-6 costs differently than direct Anthropic — stripping the prefix would give wrong prices.

Per-component cost split (R1)

Cost is computed as:

(input_tokens × input_price
 + output_tokens × output_price
 + cache_read_tokens × cache_read_price
 + cache_write_tokens × cache_write_price
 + reasoning_tokens × reasoning_price) / 1_000_000

prompt_tokens is intentionally ignored — it equals input + cache_read + cache_write in the Hermes canonical usage dict, so using it would double-count those components.

Cache prices are derived from multipliers when not explicit:

• cache_read = input × 0.10 (configurable in YAML defaults:)
• cache_write = input × 1.25
• reasoning defaults to output price

Cache-invalidation on hot-reload

_custom_pricing is a module-level None-initialized cache. reload_custom_pricing() sets it back to None, forcing the next read to re-parse the YAML. This is called by the auto-refresh cycle and by the /setup command, so new prices take effect immediately without a gateway restart.

A read/parse failure is never memoized — but IS stat-gated (issue #105/#106). _load_custom_pricing() treats “file missing” and “file present but unreadable/unparseable” as the same kind of failure state, unified under one mechanism. On EITHER failure it returns the empty fallback table for that call only and leaves _custom_pricing as None — so, unlike a successful parse, a failure is never pinned into the permanent cache. Pinning it used to silently drop every custom override — in particular a _subscription: true flat-rate entry — for the rest of that plugin instance’s process life. This does not zero out cost: pricing.yaml sits ABOVE the built-in/prefix table in the lookup priority chain (see § Pricing Engine above), not below, so an overridden model instead falls through to core-snapshot/built-in/prefix pricing and gets billed at list rate instead of its pinned rate — the #106 mis-billing symptom. A successful parse still memoizes exactly as before, so test_cache_is_stale_until_reload (a stale read until an explicit reload_custom_pricing()) is unaffected.

Naively retrying the file on every single call would reintroduce a different cost: estimate_cost() runs on the hot path (every tool call), so a persistently broken or missing file would pay an open()+parse() (or even just a re-stat()) on every one of them. Instead, on a failure _load_custom_pricing() records the file’s stat signature — _pricing_load_failure_sig = None for “missing”, else (st_ino, st_mtime_ns, st_size) — in _pricing_load_failed/_pricing_load_failure_sig. The signature is captured before the open(), not after the failed parse: if a concurrent writer atomically replaces the file mid-parse, a post-failure stat() would record the good replacement’s signature and pin the instance to the empty table forever. The inode is part of the signature because every os.replace commit yields a new one, even when mtime_ns and size collide with the file it replaced. The next call, while still in a failure streak, stat()s the file and compares: if the signature is unchanged, it returns the fallback immediately without opening or parsing anything; only a real change (file appeared, disappeared, or was rewritten) triggers an actual retry. This is also what makes a missing-then-later-created file (or a corrupt file that gets fixed) pick up correctly with no explicit reload — the changed signature falls through to a real parse attempt. reload_custom_pricing() additionally clears _pricing_load_failed/_pricing_load_failure_sig directly, so an explicit reload always forces a real attempt even in the unlikely case the file’s signature happens to be unchanged from the last recorded failure.

Since a genuine retry only happens when something changed, a persistently broken file (same content) logs nothing at all past the first failure — the signature-match short-circuit returns before any log call. Only a REAL retry that still fails logs anything, and even then only once per failure streak: WARNING on the first real failure, DEBUG on any later one, both tracked via _pricing_load_failed_warned and reset by the next successful parse.

Core-sourced pricing snapshots

pricing.py (above) computes cost from our own pricing.yaml. Separately, the plugin records the tariffs Hermes core itself resolves, so we have an auditable ground truth with provenance to diff against pricing.yaml (the Faro over-estimate is the motivating case) and to feed the manual pricing editor.

Also the primary cost source now. estimate_cost() consults the latest row of this same table (via db.get_latest_pricing_snapshot, a local read — no network) before falling through to pricing.yaml/_DEFAULT_PRICING. See § Pricing Engine → Lookup priority chain above for the full precedence. This means pricing drift/pricing backfill are no longer required to keep costs accurate — they remain useful for auditing pricing.yaml itself (e.g. to patch the fallback tier a cold-start pair still uses) but are not the primary mechanism anymore.

• core_pricing.py is the only module that imports the Hermes core (agent.usage_pricing.get_pricing_entry). The import is lazy (inside the call) and fail-open: resolve() returns None on any failure — core absent (the isolated test suite has no agent.*), unknown model, or endpoint fetch error — and never raises. It normalizes PricingEntry to a JSON-safe dict: Decimal rates → float, CostSource (a Literal string) → str, fetched_at datetime → ISO.
• Capture happens in post_api_request, the only hook that carries base_url and api_mode (verified against agent/conversation_loop.py). It is throttled in-memory to at most one resolve per (provider, model) per _PRICING_SNAPSHOT_TTL_S (6h) per process, because get_pricing_entry can do a /models network fetch for custom endpoints.
• pricing_snapshots is append-per-change (db.record_pricing_snapshot): a new row lands only when a rate field, pricing_version, or source differs from the latest stored row for that pair. Context-only fields (base_url, api_mode, source_url, fetched_at) do not trigger a row.
• Dated model names are canonicalized on miss. The provider /models catalog lists canonical (undated) ids, so a dated call like deepseek/deepseek-v4-pro-20260423 resolves to None. When the raw name misses, capture retries under core_pricing.canonical_model_name() (strips a trailing -YYYYMMDD, incl. before :free). The snapshot is stored under the raw dated name (model) with the canonical name in resolved_model (NULL when the raw name resolved directly). Safe because resolve() is fail-open: a wrong strip → None → no row. Verified on the VPS: the dominant deepseek/deepseek-v4-pro-20260423 was priced by the plugin via the generic family prefix (0.27/1.1) while the core’s real rate was 0.435/0.87. Since #121 unprefixed DeepSeek ids have explicit DeepSeek-direct built-ins; vendor-qualified deepseek/... ids are aggregator ids and intentionally keep the pre-#121 fallback, relying on the core snapshot.
• Caveat: post_api_request carries no api_key, so a private OpenAI-compatible endpoint with no cached /models metadata resolves to None (no snapshot). Nous / OpenRouter / official-docs models resolve without a per-call fetch.
• Drift computation shipped (see below). Surfacing snapshots in the dashboard (a tab rendered inside TelemetryPage) remains a follow-up.

Backfill (hermes telemetry pricing backfill)

Live capture only records a snapshot when a model is actually called, and it keys off effective_model = response_model or model — on nous the provider returns the canonical id, so historical dated ids (e.g. deepseek/deepseek-v4-pro-20260423) that dominate llm_calls get no live snapshot. hermes telemetry pricing backfill seeds coverage for them:

• Enumerates distinct (provider, model) in llm_calls with no pricing_snapshots row (db.models_needing_pricing_snapshot).
• Resolves each via core_pricing.resolve_with_fallback (the shared helper the live post_api_request capture also uses — dated ids retry under the canonical name).
• Dry-run by default; --apply writes, --json for machine output.

It is a coverage seed, not historical reconstruction: rows carry the tariff the core resolves today (captured_at = now, real provenance), stored under the raw model name plus resolved_model when the canonical fallback was used. Fail-open and idempotent — re-running skips already-covered keys; unresolvable models (no core price) are retried each run and captured automatically once the core can price them. No schema change; backfilled rows are indistinguishable from same-day live captures.

Drift detection (hermes telemetry pricing drift)

pricing.yaml is the plugin’s cost source; pricing_snapshots is the core’s ground truth. Drift detection diffs them so an over-priced local entry (the Faro case) surfaces instead of silently inflating cost:

• Reads the latest snapshot per (provider, model) via db.list_latest_pricing_snapshots, collapsed to the canonical write-key (resolved_model or model, latest-id wins on collision).
• Resolves each model’s local rate via pricing._resolve_pricing (provider-aware) and flags input/output drift beyond --threshold (default 1%). Cache/ reasoning are derived and request_cost has no pricing.yaml analog, so they are intentionally excluded. A zero snapshot rate vs. a non-zero local price is always drift (the per-side percentage is reported as undefined).
• _subscription: true entries are skipped (a declared $0 is intentional). _provider_assumed best-effort guesses are treated as “no local price”, not drift, since they were never pinned for that provider.
• Coverage-gap aware: reuses db.models_needing_pricing_snapshot — if models in llm_calls have no snapshot, the report says so and points at pricing backfill, so drift never gives a false all-clear. The gap count is DB-wide; the human-facing nudge is suppressed when --model scopes the run.
• Dry-run by default; --apply merges the snapshot input/output rates back into pricing.yaml (never clobber), tags each repaired entry _source: core-snapshot, and hot-reloads. Because pricing.yaml is keyed by model only, a model that drifted under two providers is written once — the most recent snapshot wins (highest snapshot_id), so a stale older capture from one provider never overwrites a fresher one from another sharing the same API (issue #84); a conflicting older rate is logged and skipped. Routes through paths.get_pricing_path() (resolved at call time, so HERMES_TELEMETRY_HOME outranks HERMES_HOME — same as every other pricing.yaml reader/writer). --model limits to one canonical model; --json for machine output. No schema change.

Note: pricing_refresh.py (the remote-source auto-refresh) previously used a module-level PRICING_FILE constant computed from HERMES_HOME at import time — it ignored HERMES_TELEMETRY_HOME and could freeze a stale path. Fixed to resolve paths.get_pricing_path() at call time, matching pricing.py and pricing_drift. Remaining follow-up: __init__.py::_try_pricing_refresh still derives its 24h .pricing_refresh throttle sentinel from HERMES_HOME directly, so on a consolidated (HERMES_TELEMETRY_HOME) install each profile keeps its own timer and may trigger redundant remote refreshes — harmless (all merge into the one shared file), but should also route through paths.get_telemetry_home().