fix(ci): switch workflows to self-hosted runner

[hummbl-dev]

Summary

• Switches all workflows from ubuntu-latest to [self-hosted, Linux, X64, wsl, <repo>]
• Self-hosted runner registered on WSL Ubuntu-24.04, online and idle

Why

GitHub org account is locked due to a billing issue, blocking all ubuntu-latest (GitHub-hosted) runner jobs. Self-hosted runner is online and unaffected.

Test plan

• All workflow files updated to self-hosted runner labels
• CI passes on this PR using self-hosted runner

[coderabbitai]

Warning

Review limit reached

@hummbl-dev, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 12 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ca0e472a-5b94-456e-b10e-e32ccaaf2017

📥 Commits

Reviewing files that changed from the base of the PR and between d8c5c0e and 4bec410.

📒 Files selected for processing (6)

• .github/workflows/arbiter-quality-gate.yml
• .github/workflows/ci.yml
• .github/workflows/dependabot-automerge.yml
• .github/workflows/publish-dev.yml
• .github/workflows/publish.yml
• .github/workflows/weekly-leaderboard.yml

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/ci/self-hosted-runner

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[hummbl-dev]

Problem

The self-hosted-runner migration is currently a failing PR, not a verified CI restoration. The only connector-visible workflow run on the current head failed: one Python matrix job failed its test step, the remaining matrix jobs were cancelled, and the self-grade job failed while uploading its report artifact. The exact underlying test and artifact errors are not yet independently classified.

Evidence

• Current head: 4bec41031d61d23a73717ad65003388bd03cbdbf.
• Workflow run 29270241661 (CI) completed with conclusion failure.
• test (3.11) failed at PYTHONPATH=src python -m pytest tests/ -v --ignore=tests/test_api.py.
• test (3.12), test (3.13), and test (3.14) were cancelled.
• self-grade completed its analysis and report-generation steps but failed at Upload report artifact.
• The jobs did execute on runner Anvil, proving the new runner labels can acquire a runner; it does not prove the migration is correct or merge-ready.
• The existing automated review comment reports that independent bot review did not run because its review limit was reached.
• Duplicate-control search found no existing incident artifact for PR fix(ci): switch workflows to self-hosted runner #138 or run 29270241661.

Proposed Work

1. Classify the Python 3.11 test failure from the full job log and determine whether it reproduces on current main, only on the PR merge ref, or only on the self-hosted environment.
2. Classify the artifact-upload failure separately; do not conflate successful report generation with preserved report evidence.
3. Repair the branch and obtain terminal results for every Python matrix member.
4. Require a successful artifact upload or an explicitly approved evidence-preservation alternative.
5. Obtain non-author review after the repaired head is available.

Acceptance Criteria

• The exact failing test(s) and error output are recorded with a reproducible command or environment-specific explanation.
• Python 3.11, 3.12, 3.13, and 3.14 jobs all reach terminal non-cancelled conclusions.
• The self-grade report artifact is uploaded and connector-visible, or an approved equivalent receipt is linked.
• The current repaired head has a successful required CI run.
• Independent review evidence exists before merge.
• No merge/release/publication claim is based on the current failing run.

Agent Notes

ci_incident_card:
  severity: P1
  factory_line: arbiter
  failed_station: workflow
  current_state: failing
  owner_lane: ci-remediation
  repro_command: PYTHONPATH=src python -m pytest tests/ -v --ignore=tests/test_api.py
  expected_evidence: terminal green matrix plus preserved self-grade artifact on the repaired head
  close_condition: all required jobs pass without cancelled matrix members and report artifact is retrievable
  not_authorized:
    - workflow_dispatch
    - branch_rewrite
    - merge
    - release
    - package_publication
    - repository_settings_change

Keep the PR unmerged until the two failure classes are independently resolved. This audit did not dispatch or rerun any workflow.

Automation Receipt

receipt:
  schema: repo_hygiene_push.v0.2
  run_date: 2026-07-14
  finding_id: RH-20260714-02
  severity: P1
  repository: hummbl-dev/arbiter
  accountable_artifact: pull/138
  root_cause_class: self_hosted_migration_has_test_and_artifact_failures
  verified_evidence:
    head_sha: 4bec41031d61d23a73717ad65003388bd03cbdbf
    workflow_run: 29270241661
    workflow_conclusion: failure
    failed_jobs:
      - test (3.11)
      - self-grade
    cancelled_jobs:
      - test (3.12)
      - test (3.13)
      - test (3.14)
  hypotheses:
    - exact pytest root cause not yet classified
    - exact artifact service failure not yet classified
  authority: comment_existing_pr_only
  writes_used: 1
  not_authorized:
    - workflow_rerun
    - merge
    - release
    - publish

[hummbl-dev]

2026-07-14 CI triage — P1 incomplete self-hosted migration evidence

Live connector evidence on head 4bec41031d61d23a73717ad65003388bd03cbdbf shows CI run 29270241661 reached the self-hosted runner Anvil, so this is not a hosted-runner startup failure. The run nevertheless finished failure:

• test (3.11) failed at PYTHONPATH=src python -m pytest tests/ -v --ignore=tests/test_api.py;
• test (3.12), test (3.13), and test (3.14) were cancelled;
• self-grade completed its scoring/report steps successfully but failed at Upload report artifact.

The connector exposed the job and step states but truncated the terminal pytest and artifact-upload error text. Root cause therefore remains unknown. The branch workflow currently installs only pytest for the test matrix rather than installing the package/test extra; that is a review-worthy dependency-contract hypothesis, not a verified cause.

severity: P1
factory_line: arbiter
failed_station: CI / Python test matrix and report-artifact preservation
current_state: failing
owner_lane: ci_reliability
repro_command: >-
  On a clean self-hosted Linux runner at 4bec41031d61d23a73717ad65003388bd03cbdbf,
  run `python -m pip install pytest` followed by
  `PYTHONPATH=src python -m pytest tests/ -v --ignore=tests/test_api.py`,
  preserve the complete terminal failure, and separately exercise the
  actions/upload-artifact@v4 step with the generated report files.
expected_evidence: >-
  Complete non-truncated error evidence for the first failed step, followed by
  one exact-head CI run where Python 3.11-3.14, CI Incident Card validation,
  self-grade, and report artifact upload all reach terminal success.
close_condition: >-
  The verified root cause is recorded, the workflow installs every dependency
  required by tests and artifact production, all matrix jobs complete rather
  than being cancelled, and exact-head green evidence is connector-visible.
not_authorized:
  - workflow rerun or dispatch
  - PR merge
  - branch rewrite
  - runner or network configuration mutation
  - repository or organization settings change
  - secret or credential access

No workflow was rerun and no branch, runner, or settings mutation was performed by this triage run.

[hummbl-dev]

SECURITY HOLD — DO NOT MERGE AS WRITTEN

Governed by hummbl-dev/founder-mode#1534 and related runner-placement issue hummbl-dev/founder-mode#1452.

Critical finding

arbiter is a public repository. This PR changes the pull_request CI jobs to run on a persistent self-hosted WSL runner and checks out PR-controlled code before executing tests and package installation.

GitHub's current security guidance states that self-hosted runners do not guarantee clean ephemeral environments, can be persistently compromised by untrusted workflow code, and should almost never be used for public repositories because any user can open a pull request and compromise the environment:

https://docs.github.com/en/actions/reference/security/secure-use#hardening-for-self-hosted-runners

The risk is not limited to the individual CI run. This PR also routes reusable quality-gate, Dependabot auto-merge, scheduled leaderboard, development publishing, and release publishing jobs to the same runner label. A compromised PR job can leave persistent state that affects later privileged jobs, local credentials, the host, or network-accessible services.

Additional evidence gaps

• The PR is not a draft even though its test plan still says CI has not passed.
• CodeRabbit did not review the change because its review limit was reached.
• The PR has no requested independent reviewer.
• The patch performs a blanket runner substitution without event-specific trust classification, ephemeral cleanup, runner-group isolation, host inventory, or secret/network boundary evidence.

Required redesign

1. Keep public and fork-origin PR execution off persistent trusted runners.
2. Preserve GitHub-hosted runners for public/untrusted PRs when Actions service is available; during the billing outage, use local manual validation rather than exposing the runner.
3. Restrict any self-hosted lane to trusted events and trusted refs only, such as explicitly approved pushes, bounded workflow_dispatch, or release jobs with protected environments.
4. Use separate runner identities/groups for validation and publishing; do not share a persistent workspace or trust boundary.
5. Prefer ephemeral/JIT clean runners for any job that evaluates repository code.
6. Document host secrets, network reachability, workspace cleanup, concurrency, teardown, and compromise-recovery controls.
7. Split CI/test routing from publish/automerge routing so each trust class can be reviewed independently.
8. Obtain an independent security review before returning the PR to ready-for-review status.

Disposition

DO_NOT_MERGE_AS_WRITTEN

The billing outage is not sufficient justification to replace isolated hosted execution with a persistent public-PR execution surface.