parser: full re-parse when an appended region restates a cached message id#772
Merged
Conversation
…ge id The incremental-append path groups the appended region before the file's own cached keys join the dedup set, so a streamed assistant id restated across the append boundary counted twice, and the boundary merge could splice it into the wrong turn (reproduced on real image-heavy sessions: one id streamed across several records over seconds, 111 calls vs the 110 a full re-parse yields). Suppressing the restatement instead would freeze the stale first emission, so neither shortcut matches a full re-parse. On any id overlap between the appended entries and the cached turns the shortcut is abandoned and the file re-parses from byte 0. Rare (about 0.3 percent of real files) and bounded to one file per refresh. Regression test uses the cold-reparse oracle; mutation-verified against the unguarded code.
This was referenced Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found during the adversarial review of #744: the incremental-append parser on main over-counts when a streamed assistant message id straddles the append boundary (real shape: image tool-results streaming one id across several records; 5 of 1672 local files; reproduced at 111 calls vs the full-parse 110 with turn-splice corruption at some alignments). The appended region is grouped before the file's own cached keys join the dedup set, and the boundary merge can splice the restated id into the wrong turn.
Fix is the conservative one: on any id overlap between appended entries and cached turns, abandon the shortcut and re-parse that file from byte 0. Costs a full parse on ~0.3% of files; exact everywhere. Regression test compares against a cold-reparse oracle and is mutation-verified. This also defines the bar #744's checkpoint design must meet when it is reworked (its turn-boundary resume can eventually handle this shape without the fallback).