Skip to content

fix: enforce typed finding detail contracts - #439

Open
soyeon-oai wants to merge 28 commits into
mainfrom
soyeon/finding-detail-contracts
Open

fix: enforce typed finding detail contracts#439
soyeon-oai wants to merge 28 commits into
mainfrom
soyeon/finding-detail-contracts

Conversation

@soyeon-oai

@soyeon-oai soyeon-oai commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Define concrete contracts for finding validation and attack-path details, then keep the bundled runtime and finalizer aligned with those accepted inputs. This prevents compatible scan drafts from being rejected or losing detail when they are finalized and loaded by consumers.

Changes

  • Add typed validation and attack-path definitions to the scan-draft and canonical findings schemas while continuing to allow unknown extension fields.
  • Preserve compatible aliases and scalar/object forms when validating and projecting finding details.
  • Accept preserved headless scan context without a route-local length cap.
  • Derive sibling identities from both authored and generated identities, scoped by rule and anchor.
  • Report nullable evidence catalogs as contract errors and include canonical plus legacy evidence in SARIF with safe line bounds.
  • Regenerate the TypeScript models and bundled MCP runtime, with behavior-level regressions for the shipped plugin.

Testing

  • Focused bundled-plugin contract tests — passed (4 tests).
  • pnpm run types — passed.
  • pnpm run format — passed.
  • pnpm run build — passed.
  • bun test --timeout 30000 --randomize --seed 12345 ./tests-ts — 1,301 passed, 11 skipped; one macOS sandbox integration was blocked by an outer sandbox and passed when rerun outside that wrapper.
  • pnpm run test — same 1,301 passed and 11 skipped, with the same outer-sandbox-only failure.
  • Packed-package contract and installed-package smoke checks — passed (218 package entries and 106 bundled plugin files).

The local environment used Node.js 25.9.0, which emits the package's unsupported-engine warning; the declared supported versions are Node.js 22, 24, and 26.

Risk and rollout

Known nested fields reject incompatible shapes earlier, while supported aliases, legacy evidence, authored identities, and long preserved context remain compatible. Unknown extension fields remain allowed. No data migration or staged rollout is required.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@soyeon-oai soyeon-oai changed the title Enforce typed finding detail contracts fix: enforce typed finding detail contracts Aug 15, 2026
@github-actions github-actions Bot added the bug Something isn't working label Aug 15, 2026
kmbroai
kmbroai previously approved these changes Aug 15, 2026
@soyeon-oai
soyeon-oai marked this pull request as ready for review August 15, 2026 01:04

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6639161891

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/schemas/tools/scan-draft.schema.json

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b10dbd7de6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/schemas/tools/scan-draft.schema.json Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ee0380260

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/schemas/tools/scan-draft.schema.json

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95826e3b8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/models.ts
ianw-oai
ianw-oai previously approved these changes Aug 15, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cc462b796c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/schemas/tools/scan-draft.schema.json
Comment thread sdk/typescript/_bundled_plugin/schemas/tools/scan-draft.schema.json

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d5d4528b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/report_projection.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8ec9360ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/report_projection.py Outdated
Comment thread sdk/typescript/_bundled_plugin/scripts/report_projection.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0842ef020d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py
Comment thread sdk/typescript/_bundled_plugin/scripts/report_projection.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a358237aee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a80b11647

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff4de63a87

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py Outdated
Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7340e6cc59

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/schemas/tools/scan-draft.schema.json

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if evidence_id in evidence_ids:
raise ContractError(f"{evidence_context}.id: duplicate code-evidence id")

P2 Badge Reject legacy evidence duplicates before writing drafts

When a new draft repeats an ID within code_evidence or across codeEvidence and code_evidence, the shipped MCP semantic validator explicitly skips the duplicate legacy row and persists both catalogs, but this check rejects that same draft during finalization; workbench recovery consequently discards the entire finding and marks coverage partial. Fresh evidence beyond the earlier duplicate-ID comment is this remaining MCP/finalizer disagreement after the Python validator was fixed; reject or normalize duplicates before writing the draft.

AGENTS.md reference: sdk/typescript/AGENTS.md:L16-L16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if evidence_id in evidence_ids:
raise ContractError(f"{evidence_context}.id: duplicate code-evidence id")

P2 Badge Reject duplicate legacy evidence IDs when recording drafts

When a submitted draft contains duplicate IDs within code_evidence, or reuses an ID from codeEvidence, the bundled MCP's validateFindingSemantics skips the duplicate whenever it occurs in the legacy catalog, but completion reaches this global duplicate check and fails. Thus a draft acknowledged as written cannot be finalized; mirror this check in the MCP validator. Fresh evidence in the final tree is that the Python finalizer now rejects duplicates across both catalogs while the shipped draft validator still accepts them.

AGENTS.md reference: sdk/typescript/AGENTS.md:L16-L16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if evidence_id in evidence_ids:
raise ContractError(f"{evidence_context}.id: duplicate code-evidence id")

P2 Badge Reject duplicate legacy evidence IDs when recording drafts

When a draft has two code_evidence rows with the same ID, or a legacy row collides with codeEvidence, the bundled MCP's validateFindingSemantics explicitly skips the duplicate legacy row, so recording succeeds; completion then reaches this global duplicate check and rejects the same scan. Fresh evidence after the earlier cross-alias finding is this remaining disagreement between the shipped draft validator and finalizer; make both stages reject or normalize duplicates consistently.

AGENTS.md reference: sdk/typescript/AGENTS.md:L16-L16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if evidence_id in evidence_ids:
raise ContractError(f"{evidence_context}.id: duplicate code-evidence id")

P2 Badge Reject evidence ID conflicts before storing drafts

When a draft contains the same ID in codeEvidence and code_evidence, the bundled MCP's validateFindingSemantics accepts it because duplicates encountered in the legacy catalog are skipped, but this new global duplicate check later rejects the saved finding. Workbench completion can consequently discard the finding during malformed-finding recovery, while direct finalization errors; reject cross-catalog duplicates during the MCP write instead. Fresh evidence beyond the earlier cross-catalog comment is the remaining mismatch between the bundled MCP validator and this finalizer check.

AGENTS.md reference: sdk/typescript/AGENTS.md:L16-L16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/schemas/findings.schema.json

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fbcd699ea6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 54d27ea626

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 639d524dbc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/report_projection.py
Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e762b78ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py
@soyeon-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the exact current head, 43f73dfbac067a57aac53c8854ce97e99a24d83c. All prior review threads are resolved and the current-head CI suite is green.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 43f73dfbac

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@soyeon-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please run the regular code review on exact head 43f73dfbac067a57aac53c8854ce97e99a24d83c. The exact-head security review is already clean; the regular code review result has not appeared yet.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 43f73dfbac

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@soyeon-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 7271a5e435

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants