[Visual Test] Workflow fails at 'Ensure visual-test label exists' step; screenshot capture never runs

[MichaelFisher1997]

Summary

The visual-test scheduled workflow has been failing on every run since it was introduced in #977 (Aug 3 2026). The failure is not in the game/render code, screenshot pipeline, or PPM/PNG conversion. The "Run menu screenshot capture" step (and every downstream screenshot step) is skipped because an earlier CI housekeeping step exits with a non‑zero status, so build-output.log and screenshot.png are never produced. This is exactly the artifact gap the diagnosis prompt expects to read.

Observed failure

I queried the GitHub API for the last several runs of this workflow. Every run has the same shape:

#	Step	State
1–7	checkout, devenv, weston, opencode cache, prompt loads	success
8	Ensure visual-test label exists	failure (≈2s)
9	Setup Lavapipe Vulkan	skipped
10	Run menu screenshot capture	skipped
11–16	post-screenshot bookkeeping	success / skipped
17	Run opencode visual verification	skipped
18	Run opencode failure diagnosis	in‑progress (this agent)

Recent runs confirming this:

• Design unified water shader system for reuse #44 (2026‑08‑03) – failure
• Fix per-frame string allocation in uniform setting #45 (2026‑08‑04) – failure
• Branding: Rename project to ZigCraft #46 (2026‑08‑05) – failure
• Use compile-time strings for uniforms #47 (2026‑08‑06) – failure
• Debug code behind compile-time flag #48 (2026‑08‑11, this run) – in progress, step 8 already failed at 05:45:02 UTC

Because step 10 never runs, the workspace contains weston.log but no build-output.log and no screenshot.png, which is why the diagnosis prompt's expected artifact is missing in this checkout.

Root cause

File: .github/workflows/visual-test.yml, lines 55–69.

The workflow inlines gh label list / gh label create for two labels (visual-test and run-visual-test) using GH_TOKEN: ${{ secrets.OPENCODE_PAT }}. As written, any failure of either gh invocation (auth, jq, race where the label was just created by another run, etc.) makes the step exit non‑zero and cascade‑skip the entire screenshot pipeline.

Other opencode workflows in this repo already solved this by routing the same logic through a small helper that degrades gracefully:

• .github/workflows/opencode-audit.yml:51-54
• .github/workflows/opencode-test-writer.yml

Both invoke scripts/ensure_label.sh, which:

if ! gh label list --json name --jq '.[].name' | grep -qxF "$label"; then
    if ! gh label create "$label" --description "$description" --color "$color"; then
        printf '::warning::Failed to create label %s\n' "$label" >&2
    fi
fi

i.e. the create failure is downgraded to a GitHub Actions warning, so the workflow keeps going. visual-test.yml does not use this helper and does not downgrade the failure, which is the exact behavior the API results show.

I confirmed against the live repo that visual-test and run-visual-test already exist on OpenStaticFish/ZigCraft (color E06C75), so the create branch is at minimum racy — and the helper resolves the race by no‑op‑ing on "already exists" without a non‑zero exit.

Suggested fix

Replace the inlined block in .github/workflows/visual-test.yml with two calls to the existing helper, mirroring opencode-audit.yml:

- name: Ensure visual-test label exists
  run: |
    bash scripts/ensure_label.sh "visual-test" \
      "Issues from automated visual regression tests" "E06C75"
    bash scripts/ensure_label.sh "run-visual-test" \
      "Run deterministic visual regression workflow on a PR" "E06C75"
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

Optionally tighten the existing helper to use --force on the create (idempotent) or to swallow the "already exists" error explicitly, so the helper is safe to call from any future workflow.

Secondary observations (not the cause)

While reading the prompt/code pair, two documentation drifts are worth flagging in the same issue so future diagnoses aren't chasing ghosts:

1. .github/prompts/visual-test-diagnose.md still describes the workflow as running with -Dscreenshot-path=screenshot.ppm and "PPM is then converted to PNG". The current workflow runs -Dscreenshot-path=screenshot.png directly (.github/workflows/visual-test.yml:81), and modules/engine-graphics/src/vulkan/screenshot.zig only implements PNG / JPEG / GIF / WebP encoders (detectScreenshotFormat at screenshot.zig:262-268, writeImage at screenshot.zig:245-260). If anyone re‑adds a PPM path, they need an encoder first; otherwise they will hit screenshot: unsupported image path '…ppm' (use .png, .jpg, .jpeg, .gif, or .webp) from screenshot.zig:40 and exit ScreenshotCaptureFailed from src/game/app.zig:588.

2. The screenshot capture is gated on screenshot_settle_frames >= 30 (src/game/app.zig:556), independent of ZIGCRAFT_SMOKE_FRAMES. With ZIGCRAFT_SMOKE_FRAMES=5 the menu screenshot still requires ~30 consecutive settled frames before captureFrame fires — worth knowing if anyone tunes ZIGCRAFT_SMOKE_FRAMES and expects the capture to happen on frame 5.

Where to look

• .github/workflows/visual-test.yml – failing step at lines 55‑69
• scripts/ensure_label.sh – existing helper that should be reused
• .github/workflows/opencode-audit.yml:51-54 and opencode-test-writer.yml – reference patterns

Workflow run

https://github.com/OpenStaticFish/ZigCraft/actions/runs/31462404266

[github-actions]

Posted confirmation comment on issue #48 with:

• Bug classification (CI workflow failure, not render regression)
• Verified the root cause (inlined gh label logic at visual-test.yml:55-69 vs. the existing scripts/ensure_label.sh helper used by opencode-audit.yml and opencode-test-writer.yml)
• Drop-in patch mirroring opencode-audit.yml
• Confirmed both secondary observations (PPM/PNG doc drift in visual-test-diagnose.md, and the screenshot_settle_frames >= 30 gate being independent of ZIGCRAFT_SMOKE_FRAMES)
• Scope verdict: single small fix, no need to split into sub-issues

Comment: #48 (comment)

opencode session  |  github run