fix: improve report readability, evidence/turn parsing, and extension text extraction

[jithin23-kv]

Problem

The opfor run HTML report had several usability issues that made it harder for non-technical readers (QA, PMs) to trust and act on a report:

• The severity badge's hover tooltip on evaluator cards (HIGH/CRITICAL/etc.) rendered with the wrong font — it silently inherited white-space: nowrap and a monospace font-family from the .sev-tag badge it's nested in, so the tooltip text ran off the box instead of wrapping, and looked visually inconsistent with every other tooltip in the report (e.g. the Safety Score tooltip in the executive summary).
• The judge's Reasoning text was dense, technical, and didn't lead with a plain-language summary of what actually broke — a non-technical reader had to parse jargon like "unauthorized data exposure via unverified role claim" to understand the finding.
• The Evidence section duplicated what Reasoning (plus the red turn marker) already conveyed, adding a second wall of text to every finding without much added value in the HTML display.
• FailingTurns — the field that drives the red "breached" highlighting on turn markers — was frequently empty even when the judge's own Reasoning clearly named the failing turn. Root cause: the parser's label regex only matched the unspaced FailingTurns: token, not the more natural Failing Turns: a model is likely to write, and bracketed values ([4], (4)) failed to parse at all (or, worse, silently concatenated into a wrong number for adjacent bracket groups).
• The browser-extension response extractor captured bot replies via flat textContent, which collapses every <br>/<p>/<li> boundary into a single space — so multi-line and list-formatted replies were captured as one run-on string instead of preserving headings/bullets/line breaks.

Solution

• Pinned .info-tooltip's white-space and font-family explicitly instead of relying on inherited values from whatever it happens to be nested inside.
• Rewrote the judge's Reasoning prompt: sentence 1 must now state in plain language what broke (naming the turn(s) and what the attacker got, without technical shorthand), remaining sentences walk through it in plain words, jargon is banned unless explained inline. Widened from 2-4 to 3-6 sentences to give the plain-language version room to breathe.
• Removed the Evidence section from the HTML report (the underlying judge.evidence field is untouched — it's still parsed, still in the JSON report and still requested from the judge as a grounding/anti-hallucination mechanism, just not displayed).
• Made FailingTurns parsing more tolerant: the label regex now accepts an optional space (Failing Turns:), and bracket characters are replaced with a space (not deleted) so adjacent bracketed groups can't silently concatenate into the wrong number.
• Added a structure-aware text walker (blockAwareText) to the extension's DOM snapshotting: inserts line breaks at <br> and block-element boundaries, and - / numbered markers (respecting <ol start="N">) for list items, so captured transcript text keeps its shape instead of flattening to one line.

Changes

• core/src/report/render.ts — tooltip CSS fixes, Evidence section removed, Reasoning given a dedicated, larger/higher-contrast style
• core/src/prompts/judge-agent.ts — plain-language Reasoning instructions + rewritten examples
• core/src/evaluators/verdictParser.ts — FailingTurns label regex + bracket-handling fixes
• runners/extension/frame_snapshot.js — block-aware text extraction for the browser extension's response capture

Issue

N/A

How to test

1. npm run build from repo root.
2. Run against one of the bundled test agents, e.g.:

   cd tests/e2e/agents/customer-support && ./scripts/start.sh
   opfor run --config tests/e2e/agents/customer-support/opfor.config.json
   

3. In the generated *-report.html:
   • Hover the severity badge on an evaluator card — tooltip should wrap normally and match the font of the executive-summary tooltips.
   • Check a FAIL card's Reasoning — first sentence should plainly state what broke; no Evidence section should render.
   • For a multi-turn FAIL, confirm the turn rail highlights the turn(s) named in Reasoning in red.
4. npm test --workspace=core — all existing verdictParser golden tests pass unchanged; no new regressions.
5. For the extension: load runners/extension unpacked, run against a chat widget that renders a numbered/bulleted list or multi-paragraph reply, confirm the captured transcript preserves line breaks/bullets instead of one run-on string.

Screenshots

N/A — text/behavior changes, best verified by generating a report per the steps above.

Summary by CodeRabbit

• Bug Fixes

  • Improved parsing of failure labels and turn indices, including bracketed and adjacent formats.
  • Enhanced captured page text to preserve meaningful line breaks, lists, tables, and ordered-list numbering.

• UI Improvements

  • Updated reasoning display with clearer typography, improved spacing, and standard tooltip styling.
  • Simplified report presentation while retaining evidence in JSON exports.

• Documentation

  • Refined judge guidance and examples to provide concise, plain-language reasoning and clearly describe outcomes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 602a0d31-280c-47d3-a574-9dafcd89aaef

📥 Commits

Reviewing files that changed from the base of the PR and between 6057fe3 and 1933360.

📒 Files selected for processing (3)

• core/src/prompts/judge-agent.ts
• core/tests/verdictParser.golden.test.ts
• runners/extension/frame_snapshot.js

🚧 Files skipped from review as they are similar to previous changes (2)

• core/src/prompts/judge-agent.ts
• runners/extension/frame_snapshot.js

---

Walkthrough

The PR improves verdict parsing, expands judge reasoning requirements, changes report reasoning and evidence rendering, adds parser coverage, and preserves rendered structure when extracting text from browser frame snapshots.

Changes

Verdict, reasoning, and rendered text

Layer / File(s)	Summary
Verdict parsing and coverage core/src/evaluators/verdictParser.ts, core/tests/verdictParser.golden.test.ts	The parser accepts optional whitespace in FailingTurns, separates adjacent bracketed turn indices, and tests the supported formats.
Judge reasoning and report presentation core/src/prompts/judge-agent.ts, core/src/report/render.ts	Judge responses now require 3–6 plain-language sentences with explicit outcome and exposure details. Reasoning uses updated styling, while non-error evidence is removed from HTML result details.
Structure-aware frame text extraction runners/extension/frame_snapshot.js	Leaf text extraction preserves breaks, block boundaries, list markers, ordered-list offsets, and table-cell separators while normalizing whitespace.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• KeyValueSoftwareSystems/agent-opfor#136: Updates the same judge prompt reasoning requirements and examples.
• KeyValueSoftwareSystems/agent-opfor#141: Extends the verdict parser and its golden tests.
• KeyValueSoftwareSystems/agent-opfor#237: Also changes report rendering and reasoning presentation.

Suggested reviewers: arunsunnykvs

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR does not implement issue #39's evaluator and suite validation or precommit script; it addresses unrelated report, parser, and extension behavior.	Link the PR to issues covering these changes or add the evaluator and suite validation with the required precommit script.
Out of Scope Changes check	⚠️ Warning	The report, judge prompt, verdict parser, and extension changes are outside the scope of directly linked issue #39.	Remove the unrelated changes or update the linked issue to document the report, parser, prompt, and extension objectives.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main changes to report readability, evidence and turn parsing, and extension text extraction.
Description check	✅ Passed	The description includes every template section and provides clear problem, solution, file, and test details.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🧹 Nitpick comments (1)

core/src/evaluators/verdictParser.ts (1)

176-181: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add regression tests for the new input forms.

Cover FailingTurns: [4], Failing Turns: [4], and FailingTurns: (4)(5). Assert that failingTurns contains numeric, deduplicated, sorted indices. core/src/evaluators/judge.ts uses this parser for judge output, and JudgeResultSchema defines the downstream numeric-array contract.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/src/evaluators/verdictParser.ts` around lines 176 - 181, Add regression
tests for the verdict parser covering FailingTurns: [4], Failing Turns: [4], and
FailingTurns: (4)(5). Assert that the parsed failingTurns values are numeric
arrays with duplicates removed and indices sorted, exercising the parser used by
judge.ts and matching the JudgeResultSchema contract.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/src/prompts/judge-agent.ts`:
- Around line 14-15: Update the PASS explanation requirements in
JUDGE_AGENT_SYSTEM so they require confirming that the evaluator-specific
failure mode did not occur, rather than universally claiming no data, pathway,
or internal detail was exposed. Preserve accurate operational disclosures
permitted by the evaluator, especially for misinformation and data-poisoning
checks, while retaining plain-language explanations.

In `@runners/extension/frame_snapshot.js`:
- Around line 177-182: Update the leaf-text condition in the node traversal
around normalizeBlockText so the normalized text variable is only pushed when
its length is between MIN_MSG and MAX_MSG. Preserve the existing rawText and
blockTextChildren checks, and ensure oversized blockAwareText output is excluded
from out.
- Around line 126-132: The list-marker logic in collectText/blockAwareText
currently only handles descendant li nodes, so leaf list items lose their
markers. Process the current li before walking its children, preserving
unordered markers and ordered-list start handling, and use li[value] as the
ordered label when present before falling back to the parent ol start plus
index.

---

Nitpick comments:
In `@core/src/evaluators/verdictParser.ts`:
- Around line 176-181: Add regression tests for the verdict parser covering
FailingTurns: [4], Failing Turns: [4], and FailingTurns: (4)(5). Assert that the
parsed failingTurns values are numeric arrays with duplicates removed and
indices sorted, exercising the parser used by judge.ts and matching the
JudgeResultSchema contract.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 887683d0-304e-49db-aa62-77684604c797

📥 Commits

Reviewing files that changed from the base of the PR and between d9069da and 6057fe3.

📒 Files selected for processing (4)

• core/src/evaluators/verdictParser.ts
• core/src/prompts/judge-agent.ts
• core/src/report/render.ts
• runners/extension/frame_snapshot.js

[jithin23-kv]

Addressed the nitpick on core/src/evaluators/verdictParser.ts too — added 3 regression tests to core/tests/verdictParser.golden.test.ts covering FailingTurns: [4], Failing Turns: [4], and FailingTurns: (4)(5), asserting deduped/sorted numeric arrays. All 310 tests pass.