Skip to content

feat: hard-exclude important user messages from compression#79

Open
ranxianglei wants to merge 2 commits into
masterfrom
2026-07-09_user-message-protection
Open

feat: hard-exclude important user messages from compression#79
ranxianglei wants to merge 2 commits into
masterfrom
2026-07-09_user-message-protection

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

  • Add regex-based importance detector (lib/messages/importance-detector.ts) that classifies user messages as important using 5 categories of markers in both Chinese and English
  • Hard-exclude important user messages from compression ranges — they survive intact in visible context, never enter summary blocks, never subject to GC truncation
  • New config option: compress.protectImportantUserMessages (default: true)

Problem (issue #16)

Important user messages (instructions, corrections, constraints) could be compressed into summaries, then progressively lost across successive compression cycles. The "似是而非" (seems right but isn't) problem: content degrades with each compression round.

Solution

Two-stage classifier:

  1. Gate: isLikelyDataInput() excludes logs, code blocks, diffs, tool output, structured data
  2. Detect: 10 compiled regexes (5 zh + 5 en) check remaining natural language for:
    • Importance: 记住, 重要, 关键, remember this, crucial, essential...
    • Imperative: 必须, 务必, you must, ensure that, make sure to...
    • Correction: 不对, 错了, that's wrong, you misunderstood...
    • Reminder: 我之前说过, as I mentioned, like I said before...
    • Non-negotiable: 硬性要求, non-negotiable, deal breaker...

Code blocks are stripped before detection so keywords inside code don't false-positive.

Architecture

Follows the exact same hard-exclusion pattern as protected tools (PR #75):

  • filterImportantUserMessages() in protected-content.ts — same structure as filterProtectedToolMessages()
  • Applied in range.ts after filterProtectedToolMessages()
  • Applied in message-utils.ts alongside protected tool check
  • Config-driven: compress.protectImportantUserMessages: true (default on)

Test plan

  • npm run typecheck — 0 errors
  • ✅ 24 new tests in tests/importance-detector.test.ts
  • ✅ All 610 existing tests pass (2 pre-existing Bun runner failures unrelated to this change)
  • npm run build succeeds

Closes #16

ranxianglei and others added 2 commits July 9, 2026 01:30
Add regex-based importance detector (lib/messages/importance-detector.ts) that
classifies user messages as important based on 5 categories of markers in both
Chinese and English: importance, imperative, correction, reminder, non-negotiable.

The detector uses a two-stage approach:
1. Gate: exclude data input (logs, code, diffs, tool output, structured data)
2. Detect: check remaining natural-language text for importance markers

When a user message is classified as important, it is hard-excluded from the
compression range — the message survives intact in visible context, never
enters a summary block, and is never subject to GC truncation.

New config option: compress.protectImportantUserMessages (default: true).

This prevents the gradual degradation problem where important instructions
get compressed into summaries, then progressively lost across successive
compression cycles.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant