parser: prefix-validated incremental parse for Claude JSONL appends

[avs-io]

Draft, stacked on #743 (its base commit is that PR's branch — the diff shows both commits until #743 merges, at which point this rebases clean). Same deal as #743: stays a draft until you give the go-ahead.

Implements the revised item-5 design (latest comment on #703) — capability-gated v1, Claude only.

What changed

• Claude CachedFile entries gain an optional checkpoint { parsedBytes, prefixSha256, version }. parsedBytes sits at the boundary before the current still-open turn (a turn only closes when the next non-empty user record arrives, and streaming-ID updates may still replace assistant messages inside it), so resume re-reads everything that can still change. The safety argument is written on the type.
• Warm refresh of a changed Claude file: if size grew AND sha256 of bytes [0, parsedBytes) matches, parse only from the checkpoint and splice; anything else — shrunk, same-size change, hash mismatch, no checkpoint, read error — full re-parse of that file, exactly as today. If the suffix supersedes a streaming ID from a retained turn (rare cross-boundary shape), it also falls back to full parse rather than guessing.
• No serialized parser state anywhere: dedup seeding reconstructs from cached turns as it already does; the overlap re-parse re-derives the open turn.
• All other providers keep byte-identical behavior (snapshot/journal/SQLite/aggregate-cost sources per the design's provider audit). Cache schema v5→v6: one full parse to re-checkpoint, then incremental.
• Runs entirely inside cache: strict cross-process gate for the warm session-cache refresh transaction #743's warm transaction gate, closing the shared-checkpoint concurrent-writer hole.

How tested

• Parity suite (tests/parser-claude-incremental-parity.test.ts, 9 tests): incremental result must deep-equal a from-scratch parse across appended complete turns; append while a turn was open; streaming-ID last-wins updates; a later turn superseding a retained-turn ID (→ full parse); same-length prefix mutation (→ full parse); truncation; truncate-then-append-to-larger with matching size but changed bytes (proves the prefix hash is load-bearing — passes only because of the hash check); mtime-only touch; mcpInventory equality.
• Full suite: 2171 passed with dist built.

Measured effect (synthetic, honest caveats)

52.4MB Claude JSONL + 100KB appended turn: warm incremental refresh 102ms vs 207ms full parse of the same bytes. This synthetic ratio is a floor, and it is sha256-bound (hashing the 52MB prefix is most of the 102ms): real Claude session files parse far slower per byte than they hash (object construction, turn grouping, dedup), and per-file today-corpora are typically well under this fixture's size, so the real-world win per tick is larger. Happy to attach real-corpus numbers if useful.

[iamtoruk]

Full adversarial review completed (parity harness over 29 real files, 301 truncate-append boundary comparisons, adversarial synthetic attacks, perf and migration measurement). Verdict: the design is faithful to the #703 revised item-5 sketch and the fundamentals are strong (all 301 boundary comparisons deep-equal, checkpoint never lands mid-record, prefix hash load-bearing and mutation-verified, 2.4x CPU / 2.6-3.0x wall on real large files, clean v5-to-v6 migration with identical totals), but it cannot land as-is, for two reasons.

First, the branch predates two overlapping features now on main: the incremental-append parser (8326aee) and the rich session capture (title/prLinks/isSidechain/toolResultMeta/sessionMeta with new parseClaudeEntries and groupIntoTurns signatures). Force-merging would regress those captured fields; this needs a rework against current main rather than a resolution.

Second, the parity invariant breaks on the streamed-id straddle shape (one assistant id restated across a non-empty user record; 5 of 1672 real local files): a multi-poll refresh that checkpoints mid-stream loses a call and a turn or corrupts call timestamps at some alignments, because the supersede guard only covers ids in retained turns, not ids whose first occurrence drifted into the committed prefix. Important context: this shape diverges WORSE on main's own append path (over-count, reproduced), and that main-side bug is now fixed conservatively in #772 (full re-parse on any appended-vs-cached id overlap), which also sets the bar the checkpoint design should meet natively when reworked - your turn-boundary resume is better positioned than main's line-offset resume to handle it without a fallback.

Staying a draft per its own framing. For GO after rework: rebase onto current main preserving the captured fields, extend the supersede guard to ids from the dropped open turn (or adopt the #772 overlap fallback), and add a real-data straddle fixture to the parity suite. The 29-file parity harness methodology from this review is documented in the review record for reuse.