Skip to content

feat(analyze): prime analyst engine — --analyzer prime posts the span projection to an OpenAI-compatible bridge - #81

Merged
drewstone merged 2 commits into
mainfrom
feat/prime-analyst-engine
Aug 8, 2026
Merged

feat(analyze): prime analyst engine — --analyzer prime posts the span projection to an OpenAI-compatible bridge#81
drewstone merged 2 commits into
mainfrom
feat/prime-analyst-engine

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

What

traces analyze --analyzer prime: any captured session can now be analyzed by the prime-RLM arm. Registered at the external-engine seam (ExternalAnalyzer), deliberately not the agentic dspy seam — prime here is a one-shot model call with no REPL/tools, and registering it as an agentic engine would misrepresent its execution model (seam map with file:line in the commit).

The engine posts the span projection to an OpenAI-compatible bridge (cli-bridge prime backend), using the proven protocol: short-strings finding contract, one bounded repair turn (malformed reply forwarded, trajectory never), per-attribute-cap re-render for oversized projections, findings decoded with exact-key + span-URI grounding against the artifact. Additive only — default analyze behavior byte-identical.

Includes a review fix: the per-finding span_ids cap is now enforced on the deduplicated list (was 2× the documented contract pre-dedup; found by the adversarial verifier).

Proof

  • 15/15 new engine tests (fake bridge; happy/null/repair/fail/HTTP/transport/oversized/rejected-rows paths).
  • Full suite 640/640 (baseline main was 625/625 — +15, zero regressions); typecheck + DTS build clean.
  • Docs: analyzer section extended with prerequisites (running cli-bridge prime backend), example command, and a cross-link: the scored prime-vs-dspy comparison lives in agent-eval's analyst benchmark.

Named follow-up

A registryBenchmarkRunner adapter wrapping this engine for scored benchmark runs belongs in agent-eval, not here.

… projection to an OpenAI-compatible bridge

primeAnalyzer joins the external-engine registry as a peer of haloAnalyzer
and hodoscopeAnalyzer: it inlines the full OpenInference span projection
into a one-shot prompt (per-attribute char cap on oversized traces, loud
failure when inline delivery is impossible), enforces a short-strings
fenced-JSON reply contract with one bounded stateless repair turn, maps
rows to grounded AnalystFindings validated against the artifact's span
URIs, and records bridge-reported usage in the result output.

Transport is injectable; the default is plain node:http(s) because
undici's fixed 300s headers timeout kills prime calls that legitimately
run longer. Config follows the env idiom: TRACES_PRIME_BRIDGE_URL,
TRACES_PRIME_MODEL, TRACES_PRIME_TIMEOUT_MS; --model and
--analyzer-prompt forward like every other engine. Default analyze
behavior is unchanged; the branch only fires on --analyzer prime.

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 4ac1ff1f

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-08-08T03:43:07Z

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ Value Audit — audit-incomplete

Verdict audit-incomplete
Concerns 1 (1 low)
Heuristic 0.0s
Duplication 0.1s
Interrogation 90.0s (2 bridge agents)
Total 90.1s

💰 Value — error

value agent produced no parseable value-audit JSON.

  • Model: opencode/deepseek/deepseek-v4-pro
  • Bridge attempts: 3
  • Bridge error: opencode/kimi-for-coding/k2p7: Bridge returned 503: {"error":{"message":"cli-bridge admission timed out after 30000ms","type":"admission_rejected","reason":"queue_timeout","admission":{"active":20,"queued":2,"maxActive":20,"maxQueue":48}}}; opencode/zai-coding-plan/glm-5.2: Bridge returned 503: {"error":{"message":"cli-bridge admission timed out after 30000ms","type":"admission_rejected","reason":

🎯 Usefulness — error

usefulness agent produced no parseable value-audit JSON.

  • Model: opencode/deepseek/deepseek-v4-pro
  • Bridge attempts: 3
  • Bridge error: opencode/zai-coding-plan/glm-5.2: Bridge returned 503: {"error":{"message":"cli-bridge admission timed out after 30000ms","type":"admission_rejected","reason":"queue_timeout","admission":{"active":20,"queued":2,"maxActive":20,"maxQueue":48}}}; opencode/kimi-for-coding/k2p7: Bridge returned 503: {"error":{"message":"cli-bridge admission timed out after 30000ms","type":"admission_rejected","reason":

🔎 Heuristic Signals

🟡 Cruft: console debug added examples/external-engines.ts

+console.log(


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260808T034628Z

@drewstone
drewstone merged commit aad428a into main Aug 8, 2026
1 check passed
@drewstone
drewstone deleted the feat/prime-analyst-engine branch August 8, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants