feat(providers): enable OpenRouter response caching with per-epoch salts

[abhinav-pola]

TL;DR

Every OpenRouter request now opts into response caching with a per-epoch cache salt, so Temporal activity restarts replay cached responses instead of paying for regeneration, and parquets record only real generation IDs (cache-hit dummies are resolved back to their source generation).

What changed?

• src/runtime/response-cache.ts: harness-wide constants and salt plumbing
  • always send x-openrouter-cache: true
  • inject an unknown top-level body field cache_salt = "<sessionId>:epoch-<N>[:attempt-<K>]" (unknown fields are hashed into the cache key server-side but stripped before the provider)
  • withRetryAttemptSalt appends :attempt-<K> for in-process retries only, so a cached response the harness rejects (e.g. invalid user-sim content) is not replayed on retry, while attempt 0 stays deterministic for Temporal restarts
• Applied at all 4 request sites: chat-completions provider, Responses client, and both tau user-simulators
• src/runtime/generation-ids.ts: the collector now flags IDs whose response carried x-openrouter-cache-status: HIT
• src/runtime/generation-resolver.ts (new): on cache hits, the recorded dummy generation ID is resolved to the original via GET /api/v1/generation?id=<dummy> → data.response_cache_source_id, polling (default 12 × 5s) because the dummy row lands asynchronously; on timeout/failure the dummy ID is kept and a warning is logged
• src/harness/run.ts resolves collected IDs before building the sample score, so parquet generation_ids contain only real/original generation IDs
• src/runner/run-by-id.ts provides the resolver layer from the run's apiKey/baseUrl; the resolver is an optional service, so library consumers without it keep current behavior

Why?

Benchmark activities restart under Temporal and previously re-generated every completed sample at full cost. Response caching makes restarts free, the epoch salt keeps epochs from sharing cache entries without touching x-session-id, and resolving response_cache_source_id keeps parquets pointing at real generations (cache-hit rows are zeroed dummies with endpoint_id niled out).

How to test

• bun run format:check && bun run check && bun run typecheck && bun test && bun run build (all green, 1209 tests)
• New deterministic tests: src/runtime/generation-resolver.test.ts (endpoint lookup, polling until row lands, failure fallback, hit-only resolution), src/runtime/response-cache.test.ts, src/runtime/generation-ids.test.ts

Benchmark impact

• Fresh runs are unaffected (cache MISS everywhere, salt is stripped before providers)
• Restarted runs replay cached responses for already-completed requests: identical outputs, zeroed usage/cost for replayed generations (reported cost will undercount by the replayed amount)
• Pinned-endpoint runs become cacheable after OpenRouterTeam/openrouter-web#33690 (merged), which partitions the cache key by pinned endpoint

Reviewer focus

• Salt composition and attempt semantics in src/runtime/response-cache.ts
• Resolver polling/fallback behavior in src/runtime/generation-resolver.ts (keeps the dummy ID if the source never resolves)

Checklist

• Tests cover changed behavior
• Public API or configuration changes are backward compatible, or the break is documented
• Benchmark changes document dataset provenance and licensing (n/a)
• No credentials, private results, or restricted dataset contents are included
• Documentation is updated where needed (n/a)

Link to Devin session: https://openrouter.devinenterprise.com/sessions/627adb18f0f54821b186e701d6b7fa22
Requested by: @abhinav-pola

---

[devin-ai-integration]

Original prompt from Abhinav

SYSTEM:
=== BEGIN THREAD HISTORY (in #agents) ===
<most_recent_message>
Abhinav Pola (U090K0G7JF3) [ts=1786578460.070569]: @Devin read this convo <https://openrouter.slack.com/archives/C0BMHG5CG1E/p1786573818433249|https://openrouter.slack.com/archives/C0BMHG5CG1E/p1786573818433249>. we need to introduce the response caching header to the benchmark harness so that activity restarts dont cost us money. i think this will involve a PR toward the external repo + a PR in openrouter-web

@Devin read this convo . we need to introduce the response caching header to the benchmark harness so that activity restarts dont cost us money. i think this will involve a PR toward the external repo + a PR in openrouter-web
</most_recent_message>
=== END THREAD HISTORY ===
Channel ID: C07UF9XLTFF
Thread URL: https://openrouter.slack.com/archives/C07UF9XLTFF/p1786578460070569?thread_ts=1786578460.070569&amp;cid=C07UF9XLTFF

The latest message is the one right above that tagged you. The <most_recent_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR that start with 'DevinAI' or '@devin'.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.