Skip to content

hypignore usage policy: .hypignore capture-seam drop (LLP 0049/0052/0053)#211

Draft
philcunliffe wants to merge 13 commits into
masterfrom
integration/hypignore-usage-policy
Draft

hypignore usage policy: .hypignore capture-seam drop (LLP 0049/0052/0053)#211
philcunliffe wants to merge 13 commits into
masterfrom
integration/hypignore-usage-policy

Conversation

@philcunliffe

Copy link
Copy Markdown
Contributor

Implements the .hypignore folder-scoped usage policy — LLP 0049 (spec), designed in LLP 0052, planned in LLP 0053.

What

A .hypignore file (gitignore-style, ancestor-walked) maps a directory subtree to a usage class. V1 ships one class — ignore (never recorded). Enforced at the capture seam in the client adapters (LLP 0050), so the live LLM call is untouched and only persistence is suppressed.

  • T1 src/core/usage-policy/ — shared cwd-agnostic matcher (format.js parser + privacy fail-safe, matcher.js ancestor-walk + per-cwd cache).
  • T2/T3 claude + codex adapter drops — live projector returns [] for an ignored cwd; backfill skips ignored sessions (the only 4 sites that resolve a cwd, per LLP 0050).
  • T4 hyp ignore / hyp unignore / hyp ignore --check CLI verbs.
  • T5 hermetic smoke hypignore_capture_drop — proves the drop end-to-end.

No cache schema, export driver, or gateway change (capture-seam only). local-only + session opt-out are deferred (LLP 0051).

🤖 Generated by neutral (design→plan→implement), held for human review + merge.

Change-Set: hypignore-usage-policy

philcunliffe and others added 13 commits June 29, 2026 19:04
Technical design for the .hypignore folder-scoped usage policy: a shared
cwd-agnostic matcher in src/core/usage-policy/ (ancestor walk + per-cwd cache +
file-format fail-safe), capture-seam enforcement at the four claude/codex
adapter drop-sites (live projector returns [], backfill skips), and the
hyp ignore/unignore/--check CLI verbs.

@ref LLP 0049 [implements] — covers the hypignore-usage-policy spec
@ref LLP 0050 [constrained-by] — adapter-enforced, shared matcher in core
@ref LLP 0051 [constrained-by] — local-only/session opt-out deferred, format stays forward-compatible

Generated-by: neutral

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five independently-mergeable tasks: T1 shared core matcher (src/core/usage-policy/),
then T2 claude + T3 codex adapter capture-seam drops + T4 hyp ignore/unignore/--check
CLI in parallel, then T5 hermetic smoke. Capture-seam only (LLP 0050) — no cache schema,
export driver, or gateway change.

Related: LLP 0052 (design), LLP 0049 (spec), LLP 0050 (enforcement point)
Generated-by: neutral

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…49/0050)

Add `src/core/usage-policy/`, the single shared, cwd-agnostic matcher for the
`.hypignore` folder-scoped usage policy. V1 enforces only the capture seam; this
task lands the core module the Claude/Codex adapters (T2/T3) and the CLI (T4)
will import.

- `format.js` `parseHypignore`: strip `#`/blank lines, first token is the class;
  empty/comment-only => `ignore`; unknown/unimplemented token => `ignore` + a
  `warn` string (the privacy fail-safe). `@ref LLP 0049#file-format`/`#fail-safe`.
- `matcher.js` `createUsagePolicyResolver({readFileSync,existsSync})`:
  gitignore-style ancestor walk from a cwd to the nearest `.hypignore`, per-cwd
  memo cache, fs injected. `resolve(cwd)` -> `{class,governedBy,declared}`,
  `isIgnored(cwd)`. `@ref LLP 0050`/`LLP 0049#scope`.
- `index.js` barrel + `types.d.ts` (`UsageClass`, `ParseResult`, `ResolveResult`,
  `UsagePolicyResolver`).
- Tests `test/core/usage-policy.test.js`: empty=>ignore, unknown=>ignore+warn,
  nearest-ancestor wins, no file=>full, cache stable + reads once.

Task-Id: T1

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the shared usage-policy resolver (LLP 0050 / src/core/usage-policy)
into both Claude capture-seam drop-sites so an exchange whose resolved
cwd is governed by an ancestor `.hypignore` is never written to the
cache, for live capture and backfill alike (LLP 0049 R1/R2):

- projector.js: createClaudeExchangeProjector holds one resolver per
  projector. Once the exchange cwd is resolved from the session-context
  record, an ignored cwd returns no rows BEFORE building any, so the
  ai-gateway source write guard persists nothing. Returning `undefined`
  is the projector's existing no-rows signal (a literal `[]` is an
  invalid projection that would log a spurious warning); the live LLM
  call already streamed, so it is untouched.
- backfill.js: createClaudeBackfillProvider holds one resolver per run
  and skips ignored sessions before projecting/writing, keyed on the
  same cwd the row would carry (record cwd, else first transcript line).
- Both drop-sites emit a structured usage_policy_drop event.
- Resolver is injectable on both factories for hermetic tests.

New test/plugins/claude-usage-policy-drop.test.js drives the real
matcher (injected fs) through the gateway dispatcher and the backfill
provider: ignored cwd => no rows / session skipped; clean cwd and
no-cwd unaffected.

@ref LLP 0050 [implements]
@ref LLP 0049#requirements [constrained-by]

Task-Id: T2
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Symmetric to the @hypaware/claude T2 drop: enforce the `.hypignore`
folder-scoped `ignore` usage policy at the Codex capture seam, the only
place that resolves a `cwd`, so an ignored exchange never reaches the cache.

- `codex/src/exchange-projector.js`: `createCodexExchangeProjector` holds one
  shared `createUsagePolicyResolver()` (per listener; injectable for tests).
  Once the exchange `cwd` is resolved, an ancestor `.hypignore` of class
  `ignore` returns no projection, so the gateway source's
  `messageRows.length > 0` write guard persists nothing. The response has
  already streamed, so the live LLM call is untouched (LLP 0049 R1/R2). Emits
  a structured `usage_policy_drop` log (hashed cwd, governing path).
- `codex/src/backfill.js`: `createCodexBackfillProvider` holds one resolver and
  skips a session whose recorded `cwd` is ignored before projecting/yielding,
  so `hyp backfill` never re-imports sessions ignored live (LLP 0049 R1). Adds
  `sessions_ignored` to the scan-complete telemetry.

The projector returns `undefined` (its established skip signal — the dispatcher
maps it to an empty rows array, the "return []" of LLP 0050 §Live); returning a
literal `[]` would trip the gateway's invalid-projection warning.

No cache schema, export driver, gateway, or settlement change — capture-seam
only, per LLP 0050. Tests mirror T2: ignored cwd -> no rows/skip, clean cwd
unaffected, drop telemetry emitted; the real shared matcher is exercised via
injected fs.

@ref LLP 0050 [implements]

Task-Id: T3

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#cli)

Implement the usage-policy CLI verbs on top of the shared core matcher
(T1). These are imperative, cwd-relative, filesystem-mutating commands, so
they register as plain `ctx.commands` CommandRegistrations alongside the
existing `ignore`/`attach`/`detach` commands — not LLP 0034 verbs (whose
`VerbOperationContext` has no `cwd` and whose tools are remotely
invokable). LLP 0009 itself draws this line: "imperative/interactive
commands stay `ctx.commands.register`".

- `hyp ignore [path]` writes a self-documenting `.hypignore` (comment
  header + `ignore` token) at the git repo root, else the cwd; an explicit
  `path` overrides. Idempotent (R5): a path already governed by an
  ancestor `.hypignore` is a no-op success, never a redundant nested file.
- `hyp unignore [path]` removes the nearest governing `.hypignore`.
  Idempotent: a no-op when nothing governs.
- `hyp ignore --check [path]` reports ignored?/governing file/residual
  already-cached row count; prospective-only, never purges
  (LLP 0049 #prospective-only). The residual count pushes a superset LIKE
  filter into `ai_gateway_messages` then refines with an exact subtree
  match in JS (squirreling LIKE treats `_`/`%` as wildcards), and degrades
  to `unknown` when the dataset is unavailable.

Repo-root resolution is a new `findRepoRoot` in the core usage-policy
module: a dependency-free, fs-injectable `.git` ancestor walk mirroring the
adapters' `git rev-parse --show-toplevel`, kept in core so the CLI need not
spawn git and stays hermetically testable.

Tests: idempotency for ignore/unignore, repo-root vs cwd vs explicit-path
placement, `--check` output (ignored/clean/--json/graceful-unknown), a real
residual count proving the LIKE-superset + exact-refine, and direct
findRepoRoot unit tests.

Task-Id: T4
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…P 0053 T5)

Hermetic smoke that drives one Claude exchange from a `.hypignore`'d cwd and
one from a clean cwd through the daemon (ai-gateway + claude), then asserts:

- only the clean session's rows land in `ai_gateway_messages` (no ignored cwd,
  no ignored session id), proving R1;
- the gateway returned 200 for the ignored exchange and its `aigw.exchange`
  log recorded `rows_written = 0`, proving R2 (live call untouched);
- the claude projector emitted a `usage_policy_drop` event naming the governing
  `.hypignore`.

Each phase runs under a stable `smoke_name`/`smoke_step` root span so a failure
names the broken step. Registered in the README smoke battery (the flow index).

@ref LLP 0049#requirements [tests]
@ref LLP 0050 [tests]
@ref LLP 0053#tasks

Task-Id: T5

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A relative `path` argument to `hyp ignore`/`unignore`/`ignore --check`
resolved against the Node process cwd instead of the command-context
cwd, so injected/remote/test dispatch could write, remove, or check the
wrong tree — a privacy-relevant misfire for this control. Resolve against
`ctx.cwd ?? process.cwd()` like the sibling verbs (plugin scaffold/init)
already do. No-path and absolute-path behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@philcunliffe

Copy link
Copy Markdown
Contributor Author

Dual-agent review — request_changes

  • Verdict: request_changes
  • Risk class: medium
  • Auto-merge advisory: 👎 thumbs down — verdict is request_changes; needs human-gated follow-up
  • Reviewed head: 7a201bf · fix pushed: 77bfab9

Advisory only: no merge was attempted.

What changed during review

One actionable major was fixed and pushed (77bfab9): relative path args to hyp ignore/unignore/--check now resolve against the command-context cwd, not the Node process cwd (matching the sibling plugin verbs). Full CI mirror (build:types + typecheck + lint + test, 1521 pass) green. The remaining major below is spec-sanctioned and left for a human decision — not silently patched.

Headline finding (open, human decision)

Process-lifetime negative (full) policy cache makes hyp ignore silently ineffective on a running daemon. A cwd already resolved+cached as full by the live projector keeps recording after the user writes a .hypignore / runs hyp ignore, until the daemon restarts — a silent leak window against R1. Faithfully implemented per LLP 0052 #matcher (cache is process-lifetime), so it is a documented design gap, not a code deviation. Decide: accept-as-documented (+ surface a restart hint to the user) or harden (bounded TTL / mtime re-stat that still satisfies R6). Raised independently by both reviewers.

Risk capstone

Cross-reference: reviewer findings vs blast radius

Finding Reviewer(s) Severity On hot path? Status
Process-lifetime full cache → hyp ignore ineffective on running daemon Claude + Codex major yes (live resolver) OPEN — spec-sanctioned (LLP 0052); human decision
Relative path arg resolved vs process.cwd not ctx.cwd Codex major no (CLI only) FIXED + pushed (77bfab9)
Claude backfill cwd derived from windowed (post-time-window) entries Codex major (med conf) backfill only OPEN-note — likely unreachable ("cwd rides every line"); drop mirrors row stamp; below Claude ≥80 bar
Fail-safe warn produced then discarded Claude + Codex minor no note
Em dashes vs AGENTS.md absolute no-em-dash rule Claude minor no note
Codex review

Fix Validations

No explicit bug-fix claim to validate. This is feature work, so findings below are new issues in the changed behavior.

Findings

4) Concurrency, Ordering & State Safety

  • Severity: major
  • Confidence: high
  • Evidence: src/core/usage-policy/matcher.js:42, src/core/usage-policy/matcher.js:50, src/core/usage-policy/matcher.js:51, hypaware-core/plugins-workspace/claude/src/projector.js:99, hypaware-core/plugins-workspace/codex/src/exchange-projector.js:45, src/core/cli/core_commands.js:3753, src/core/cli/core_commands.js:3797
  • Why it matters: A live daemon caches the policy result per cwd for the projector lifetime, so hyp ignore or hyp unignore can silently fail to affect future captures for a cwd already resolved before the file change.
  • Suggested fix: Add invalidation for mutable policy files, for example mtime/TTL checks, avoid caching full, or signal the daemon from the CLI; add a regression that resolves a cwd, creates/removes .hypignore, then resolves the same cwd again.

1) Behavioral Correctness

  • Severity: major
  • Confidence: high
  • Evidence: src/core/cli/core_commands.js:3737, src/core/cli/core_commands.js:3790, src/core/cli/core_commands.js:3826, src/core/cli/dispatch.js:310, src/core/cli/dispatch.js:322
  • Why it matters: Relative hyp ignore [path], hyp unignore [path], and hyp ignore --check [path] arguments resolve against the Node process cwd instead of the command context cwd, so injected/remote/test dispatch can write, remove, or check the wrong tree.
  • Suggested fix: Resolve as path.resolve(ctx.cwd ?? process.cwd(), parsed.path ?? '.'), or equivalent, in all three command paths.

1) Behavioral Correctness

  • Severity: major
  • Confidence: medium
  • Evidence: hypaware-core/plugins-workspace/claude/src/backfill.js:176, hypaware-core/plugins-workspace/claude/src/backfill.js:185, hypaware-core/plugins-workspace/claude/src/backfill.js:344, hypaware-core/plugins-workspace/claude/src/backfill.js:379
  • Why it matters: Claude backfill derives the policy cwd from windowed entries, so a --since/--until window can miss a cwd present elsewhere in the loaded session and import rows from an ignored session.
  • Suggested fix: Compute the policy cwd from record?.cwd ?? sessionEntries.find(...)?.cwd before applying the time window, and add a test where the cwd-bearing transcript line is outside the import window.

11) Debuggability & Operability

  • Severity: minor
  • Confidence: high
  • Evidence: src/core/usage-policy/format.js:39, src/core/usage-policy/matcher.js:66, src/core/usage-policy/matcher.js:67
  • Why it matters: Unknown or reserved .hypignore tokens are clamped to ignore, but the emitted warn is dropped by the resolver, making fail-safe drops hard to diagnose.
  • Suggested fix: Carry warn through ResolveResult or log once per governing file at the adapter/CLI boundary.

No Finding

  1. Contract & Interface Fidelity; 3) Change Impact / Blast Radius; 5) Error Handling & Resilience; 6) Security Surface; 7) Resource Lifecycle & Cleanup; 8) Release Safety; 9) Test Evidence Quality; 10) Architectural Consistency.

Evidence Bundle

  • Changed hot paths: src/core/usage-policy/matcher.js, src/core/usage-policy/format.js, src/core/cli/core_commands.js, Claude/Codex live projectors, Claude/Codex backfills, hypignore_capture_drop smoke.
  • Impacted callers: hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:503, hypaware-core/plugins-workspace/ai-gateway/src/source.js:117, src/core/cli/dispatch.js:322.
  • Impacted tests: test/core/usage-policy.test.js:132, test/core/ignore-command.test.js:110, test/core/ignore-command.test.js:165, test/plugins/claude-usage-policy-drop.test.js:32, test/plugins/claude-usage-policy-drop.test.js:103, test/plugins/codex-backfill.test.js:341, test/plugins/codex-exchange-projector.test.js:36.
  • Unresolved uncertainty: Whether process-lifetime policy caching is intentional despite the CLI contract; whether historical Claude transcripts can have cwd only outside a filtered backfill window; tests were not run.
Claude review

Claude review

Process-lifetime negative (full) policy cache makes hyp ignore silently ineffective on a running daemon

  • Severity: major
  • Confidence: 92
  • Evidence: src/core/usage-policy/matcher.js:48-55 (caches every result incl. {class:'full'} for the resolver/daemon lifetime); src/core/cli/core_commands.js:3764 (hyp ignore prints only wrote <file>)
  • Why it matters: A user who has already been working in a folder (its cwd resolved+cached as full by the live projector's long-lived resolver) then runs hyp ignore keeps being recorded for every later exchange in that folder until the daemon restarts — the exact "I flagged it, now stop" intent fails silently, the leak this control exists to prevent. It is faithfully implemented per LLP 0052 #matcher ("Cache is process-lifetime ... --check constructs a fresh resolver"), so it is a design gap rather than a code deviation, but R1 ("an exchange whose resolved cwd has an ancestor .hypignore MUST NOT be written") is violated for the window between file-write and daemon-restart, and nothing tells the user. This is the one judgment call a human should adjudicate (accept-as-documented vs harden).
  • Suggested fix: At minimum, have hyp ignore/hyp unignore print + document that a running daemon must be restarted for the change to affect already-active folders. For a real fix, bound the negative-result cache (short TTL or re-stat the nearest ancestor dirs on lookup) so a newly-written .hypignore is honored without restart while still satisfying R6 (note: simply not caching full reintroduces the per-exchange walk R6 forbids). Either path is a change to Accepted LLP 0052 and should be deliberated, not silently patched.

Fail-safe warn for unimplemented/unknown class tokens is produced then discarded

  • Severity: minor
  • Confidence: 85
  • Evidence: src/core/usage-policy/matcher.js:66-67 (walk() keeps class/declared, drops parsed.warn; ResolveResult has no warn field); src/core/usage-policy/format.js:36-40 (warn produced); adapter drop-logs log governed_by but not declared
  • Why it matters: R3 says an unimplemented class MUST resolve to ignore (satisfied — privacy-safe) and SHOULD warn; LLP 0052 #telemetry asks to warn "with the declared token and the governing path". As written a .hypignore containing local-only or a typo is clamped to ignore with no observable signal, so operators can't distinguish a misdeclared policy from an intentional ignore.
  • Suggested fix: Carry warn/declared through ResolveResult and emit a warn-level event on fail-safe drops in both adapters including the declared token.

Em dashes (U+2014) in new runtime string + comments violate the head AGENTS.md absolute rule

  • Severity: minor
  • Confidence: 95
  • Evidence: src/core/cli/core_commands.js:3680 (em dash baked into the HYPIGNORE_TEMPLATE # header written to every .hypignore), plus em dashes in new comments/JSDoc/@ref glosses at core_commands.js:3725,3739,3749,3775,3817,3855; usage-policy/repo_root.js:10,18,19; codex/src/exchange-projector.js:81
  • Why it matters: The PR head's AGENTS.md bans U+2014 "anywhere: code, comments, JSDoc, strings, or docs"; the template em dash is functionally harmless (it sits in a # comment line the matcher never tokenizes) but is still a written-to-disk violation, and two of the offenders are @ref LLP 0049#cli glosses.
  • Suggested fix: Replace each with -, a colon, or a sentence split.

(Cross-reviewer, FIXED) Relative path arg resolved against process cwd, not ctx.cwd

  • Severity: major
  • Confidence: 90
  • Evidence: src/core/cli/core_commands.js:3737,3790,3826 (was path.resolve(parsed.path ?? ctx.cwd))
  • Why it matters: Raised by Codex (cat 1). A relative path argument to hyp ignore/unignore/--check resolved against the Node process cwd instead of the command-context cwd, diverging from the sibling verbs (core_commands.js:1610,1708) and letting injected/remote/test dispatch write/remove/check the wrong tree — a privacy-relevant misfire.
  • Suggested fix: APPLIED — now path.resolve(ctx.cwd ?? process.cwd(), parsed.path ?? '.') at all three sites; no-path/absolute behavior unchanged, full CI mirror green, pushed as 77bfab9.

Notes verified clean (no finding)

  • R1/R4: all four drop sites (claude live projector.js:173, claude backfill.js:185, codex exchange-projector.js:83, codex backfill.js:196) use the single shared createUsagePolicyResolver and check the EXACT cwd each site stamps on the row (projector.js:298; exchange-projector.js:130/866-869; backfill.js:379; backfill.js:555). No leak.
  • R2: live drop returns undefined BEFORE building rows, after the response has streamed; dispatcher (message_projector.js:512-520) treats undefined as "no rows" without logging aigw.projector_invalid_output; the proxied response is never touched.
  • R3 fail-safe: format.js clamps unknown/local-only/empty to ignore; matcher safeRead clamps unreadable .hypignore to ignore. Always suppress-more.
  • #scope: ancestor walk to fs root, nearest wins, no-file => full, root terminates (no infinite loop), symlink handling is lexical and consistent with how cwd is stamped.
  • Contracts/imports/tests: dispatcher always passes {log} to project(); all CLI imports/ctx fields present; coverage exercises fail-safe, nearest-ancestor, all four drop sites (ignored-vs-clean), R5 idempotency, and --check LIKE-superset/exact-refine; smoke flow is filename-registered and asserts only-clean-row-lands + a usage_policy_drop event.

Reviewed from a clean detached worktree at the PR head. Reports: /Users/phil/workspace/hypaware/.git/worktrees/tmp.dBlx0CjJrd/dual-review/pr-211

@philcunliffe philcunliffe added the neutral:stuck neutral attempted this but cannot complete it autonomously — needs a human label Jun 30, 2026
@philcunliffe

Copy link
Copy Markdown
Contributor Author

🤖 neutral: built, green, reviewed — held on one design decision

The loop drove this end-to-end from request LLP 0049: design 0052 → plan 0053 → 5 implemented+verified tasks → green PR → dual-review. lint + typecheck + test green at head 77bfab9, mergeable, never merged.

Fixed during review (77bfab9, CI-green): a MAJOR — relative path args to hyp ignore/unignore/--check resolved against process.cwd() instead of ctx.cwd, diverging from the sibling verbs and mis-targeting under injected/remote dispatch.

Held for you — the lone request_changes driver (a design call, not a code bug)

The shared matcher's process-lifetime cache (src/core/usage-policy/matcher.js, faithful to LLP 0052 #matcher) means a cwd first resolved as full keeps being recorded after you later write a .hypignore / run hyp ignore, until the daemon restarts — a silent leak window against R1. Both reviewers raised it independently.

It's a documented design choice, so resolving it is yours:

  • (a) amend 0052 #matcher to a bounded staleness check — short TTL, or an mtime re-stat of the governing path — that still satisfies R6 (no unbounded hot-path fs work) — and update matcher.js; 0052 is neutral-minted and unmerged, so this is a clean follow-up commit on this branch; or
  • (b) accept it for V1 and print a "restart the daemon to apply" hint in hyp ignore output.

Not "don't cache full" — that reintroduces the per-exchange walk R6 forbids.

Minor (non-blocking, left as notes)

  • R3 SHOULD-warn: the fail-safe warn from format.js is dropped by matcher.js (a clamped local-only/typo is indistinguishable from an intended ignore in logs).
  • Em-dashes in the .hypignore template string + some new comments violate AGENTS.md's no-em-dash rule.

Full findings + blast-radius: the dual-review comment above. The privacy core (all four capture-seam drop sites use the shared resolver on the exact stamped cwd; R2 live-call-untouched; fail-safe/scope) was verified clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

neutral:stuck neutral attempted this but cannot complete it autonomously — needs a human

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant