From cb7d76f89c631aa23dd7d70818957476528ec335 Mon Sep 17 00:00:00 2001 From: topcoder1 Date: Fri, 14 Aug 2026 14:46:46 -0700 Subject: [PATCH 1/4] fix(automerge): anchor the quiet period on the review TRIGGER, not only on elapsed silence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit wxa_vpn#1392, incident PR wxa_vpn#1512 (2026-08-14). The quiet gate held the arm until the conversation had been silent for findings_quiet_minutes, measured against three anchors — newest authored commit, newest inline comment, newest non-self [bot] comment. All three measure silence that has ALREADY ELAPSED, which is the right question only when the event that woke the workflow did not itself start a review. `ready_for_review` does exactly that. pr-review.yml, pr-codex-review.yml and this workflow share the same `types:` list, so readying a draft fires the review lanes and arms the gate in the same instant, while every anchor the gate can read still dates from before the PR was drafted. The longer a PR sits in draft, the more thoroughly the gate is satisfied that nobody is talking — at the exact moment three review lanes start talking. Measured on #1512, which was drafted at 03:21 and readied at 05:38:04Z: Quiet: newest commit/review activity is 7757s old (>= 1200s). whois-api-llc/wxa_vpn#1512 — no unaddressed findings (last commit 03:20:37Z) Enabling auto-merge (squash) [05:38:26Z] It never slept. The detector's "clean" was accurate and useless: it read a conversation the reviews had not joined yet. Codex posted VERDICT: REGRESSION at 05:39:01Z — 35s after the arm — and the PR merged at 05:42:51Z carrying it. The regression shipped and needed follow-up PR #1518 to fix. #1518 then cleared the same way at 3074s; its review came back clean, so that one was a near miss rather than a hit. A longer window is not the fix: no window helps when the clock starts pre-expired, and 7757s already exceeded the 20-minute default by 6.5x. The anchor has to include the moment the CURRENT round of review was triggered — so c4 is the newest `ready_for_review`/`reopened` event. This is not new policy. The push path already waits: wxa_vpn#1513, the same day, slept 1162s then 441s and merged clean. This only stops the ready path from skipping a wait the push path has always taken. Three properties the tests pin, because over-gating is how these controls get switched off: * the anchor is the event's TIMESTAMP, not a boolean "was readied", so a later `labeled` re-run sees it age out normally and the gate cannot be held shut by its own re-runs; * only the two review triggers anchor — `labeled` above all must not, since pr-classify labels every PR within seconds of opening and this workflow re-runs on `labeled`; * a PR with no such event keeps exactly the pre-fix anchor set. The read fails CLOSED like its three siblings: "could not read the timeline" and "the PR was never readied" are the same empty value downstream, and only the second justifies arming. Tests: selftest/test_automerge_quiet_anchor.sh (14 checks). Proven to bite by mutation — removing the anchor fails 5 checks with A1 reporting `sleeps=[] clear=1 reason=quiet+clean`, the #1512 log line exactly; making the read fail-open fails A5 alone; dropping the event filter fails S2 and all six A4 cases. test_automerge_findings_gate.sh's S4 loop grew the fourth endpoint (its claim is "every anchor read paginates", and a list enumerating three of four silently stops being that) and its gh stub grew a timeline arm. Co-Authored-By: Claude Opus 5 --- .github/workflows/claude-author-automerge.yml | 59 +++- selftest/test_automerge_findings_gate.sh | 16 +- selftest/test_automerge_quiet_anchor.sh | 320 ++++++++++++++++++ 3 files changed, 388 insertions(+), 7 deletions(-) create mode 100755 selftest/test_automerge_quiet_anchor.sh diff --git a/.github/workflows/claude-author-automerge.yml b/.github/workflows/claude-author-automerge.yml index 82b69d2..5a361ae 100644 --- a/.github/workflows/claude-author-automerge.yml +++ b/.github/workflows/claude-author-automerge.yml @@ -152,7 +152,7 @@ on: type: boolean default: true findings_quiet_minutes: - description: "Quiet period (minutes) the PR must be free of new commits and review-bot comments before auto-merge is armed, after which the unaddressed-findings check runs (wxa_vpn#1392: findings posted 3-16 min after the last push merged unanswered). Applies to EVERY arm path, bypass labels included — the escape for a wrong finding is the manual merge click, which this gate never blocks. Set 0 to disable the whole gate (no wait, no findings check)." + description: "Quiet period (minutes) the PR must be free of new commits, review-bot comments and review triggers (ready_for_review / reopened) before auto-merge is armed, after which the unaddressed-findings check runs (wxa_vpn#1392: findings posted 3-16 min after the last push merged unanswered; #1512: a draft readied after 2h09m idle cleared the window without waiting at all and armed 35s before a REGRESSION verdict). Applies to EVERY arm path, bypass labels included — the escape for a wrong finding is the manual merge click, which this gate never blocks. Set 0 to disable the whole gate (no wait, no findings check)." required: false type: number default: 20 @@ -1032,10 +1032,14 @@ jobs: # 1. QUIET — wait until >= findings_quiet_minutes have passed since # the newest of: the newest AUTHORED (non-merge) commit, the # newest inline review comment (any author — an inline comment - # is a deliberate code-level finding), and the newest top-level + # is a deliberate code-level finding), the newest top-level # [bot] comment that is not this workflow's own marker comment # or pr-classify chatter (a gate that counts its own sticky - # comment as review activity never opens). Sleeping here is + # comment as review activity never opens), and the newest + # `ready_for_review`/`reopened` event — the trigger that STARTS + # a round of review, without which the clock is pre-expired on + # exactly the event that begins one (wxa_vpn#1512; see c4 in + # read_anchor for the measurement). Sleeping here is # safe: callers run concurrency cancel-in-progress per PR, so a # new push replaces this run and restarts the clock, and the # arm below binds --match-head-commit so a moved head can never @@ -1098,8 +1102,49 @@ jobs: # a --paginate stream (one top-level array per page) aggregates # ACROSS pages, not per page — the multi-page defect pinned in # test_automerge_hold_gate.sh. + # + # c4 — THE REVIEW-TRIGGER ANCHOR, and the reason this gate armed + # over a live finding on wxa_vpn#1512 (2026-08-14). The other + # three anchors all measure silence that has ALREADY ELAPSED, + # which is the right question only when the event that woke this + # workflow did not itself start a review. `ready_for_review` and + # `reopened` do exactly that: pr-review.yml, pr-codex-review.yml + # and this workflow share the same `types:` list, so readying a + # draft fires the review lanes and arms the gate IN THE SAME + # INSTANT — while every anchor the gate could read still dates + # from before the PR was drafted. + # + # Measured on #1512: the PR sat as a draft for 2h09m, so at + # ready_for_review the step logged + # Quiet: newest commit/review activity is 7757s old (>= 1200s) + # cleared the 20-minute window without sleeping once, ran the + # detector against a conversation the reviews had not joined yet, + # got a legitimate "no unaddressed findings", and armed at + # 05:38:26Z. Codex posted VERDICT: REGRESSION at 05:39:01Z — 35 + # seconds later — and the PR merged at 05:42:51Z carrying it. + # #1518 cleared the same way at 3074s (its review happened to + # come back clean, so it was a near miss rather than a hit). + # + # The fix is not a longer window — no window helps when the clock + # starts pre-expired. It is to anchor on the moment the CURRENT + # round of review was triggered, so `ready_for_review` waits + # exactly as long as a push does. That is not new policy: the + # push path already waits (wxa_vpn#1513 slept 1162s + 441s and + # merged clean), and this only stops the ready path from skipping + # a wait the push path has always taken. + # + # These are POINT-IN-TIME event timestamps, not "now", so a later + # `labeled` run re-reads the same fixed value and sees it age out + # normally — the gate cannot be held shut by its own re-runs. A + # PR opened non-draft and never reopened has no such event, `max` + # yields empty, and the anchor set is exactly what it is today. + # + # Fail-closed like every sibling read: `issues/{n}/timeline` is + # already load-bearing in this job (the detector fetched below + # reads issues/{n}/comments fatally), so an unreadable timeline + # is an unreadable anchor, not an absent one. read_anchor() { - local commits inline issue c1 c2 c3 + local commits inline issue timeline c1 c2 c3 c4 commits=$(retry gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR}/commits?per_page=100" --paginate) || return 1 c1=$(printf '%s' "$commits" | jq -rs '[(add // [])[] | select((.parents | length) < 2) | .commit.committer.date] | max // empty') || return 1 inline=$(retry gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR}/comments?per_page=100" --paginate) || return 1 @@ -1109,7 +1154,11 @@ jobs: | select(((.user.login // "") | endswith("[bot]"))) | select(((.body // "") | test("