Skip to content

e2e/qa: retry seat withdraw while in-flight rejection persists instead of polling pending flag#4065

Closed
thijsvanemmerik wants to merge 5 commits into
mainfrom
e2e-qa-settlement-ack-slot
Closed

e2e/qa: retry seat withdraw while in-flight rejection persists instead of polling pending flag#4065
thijsvanemmerik wants to merge 5 commits into
mainfrom
e2e-qa-settlement-ack-slot

Conversation

@thijsvanemmerik

@thijsvanemmerik thijsvanemmerik commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

The multicast settlement QA test failed persistently on seat teardown. Two problems in the old flow:

  • wait_for_seat_allocation_acked polled the client seat's pending flag to detect the oracle ack. That flag cannot distinguish "the oracle has not acked yet" from "no allocation request was created" (a re-fund of an already-active seat creates none), so the wait could time out with saw_pending=false when there was simply nothing to ack.
  • withdraw_seat did a single-shot withdraw. The CLI preflight rejects with an in-flight error while it reads the allocation request PDA as still present, and that read can lag an already-closed request, so a single attempt fails transiently.

Changes:

  • Remove the pending-flag ack-wait and make the withdraw retry while the in-flight rejection persists (WithdrawSeatWaitingForAck). The retry absorbs both a slow oracle ack and a lagging RPC read; an in-flight withdraw bails atomically with no state change, so retrying is safe. Any non-in-flight error fails fast.
  • Log the seat's tenure and pending flag right after pay (diagnostic only), so a "no request created" run is visible in the run log.

Tradeoff

With the pending-flag ack-wait removed, a genuine oracle outage now surfaces as validate_tunnel_up timing out rather than a specific "not acked" error. The withdraw's timeout error calls out a persistent in-flight rejection (a stale RPC view) versus a slow ack.

Scope

Self-heal of a seat left active by a prior failed teardown is handled separately and is intentionally not part of this PR.

Testing Verification

Verify with a manual workflow_dispatch of qa.mainnet-beta with doublezero_branch set to this PR branch, confirming TestQA_MulticastSettlement passes end to end including withdraw_seat (baseline is green after the stuck seat was cleared).

@thijsvanemmerik
thijsvanemmerik marked this pull request as draft July 17, 2026 12:32
@thijsvanemmerik thijsvanemmerik changed the title e2e/qa: gate settlement ack-wait on seat subscription start slot e2e/qa: retry seat withdraw until oracle acks instead of polling pending flag Jul 17, 2026
@thijsvanemmerik

Copy link
Copy Markdown
Contributor Author

Closing unmerged. This was a QA-environment issue handled separately; no change to land here.

@thijsvanemmerik thijsvanemmerik changed the title e2e/qa: retry seat withdraw until oracle acks instead of polling pending flag e2e/qa: retry seat withdraw while in-flight rejection persists instead of polling pending flag Jul 17, 2026
ben-dz added a commit that referenced this pull request Jul 17, 2026
…persedes #4065)

Co-authored-by: Thijs van Emmerik <thijs@doublezero.xyz>
@ben-dz

ben-dz commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Superseded by #4066

@ben-dz ben-dz closed this Jul 17, 2026
ben-dz added a commit that referenced this pull request Jul 17, 2026
…persedes #4065)

Co-authored-by: Thijs van Emmerik <thijs@doublezero.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants