Three skills for finding bugs in a system that already has real data and real users — and for making sure the ones you close are actually fixed.
npx skills add https://github.com/veronalabs/qa-loop-skillA test suite passes. Every feature was checked off. The product is still broken when a customer uses it.
That happens because tests ask whether a function returned the right value, and never whether the outcome reached a person. An email is sent — to a dead link. A rate is stored — and displayed as zero. A rule is enforced on one screen and not on the path everything else uses.
None of that is visible from a green test run. It is visible from asking the live data a question.
| Skill | Who runs it | What it does |
|---|---|---|
qa-hunt |
the finder | States rules, sweeps real data against them, verifies each candidate, files issues |
qa-fix |
the fixer | Takes one issue at a time, fixes the cause, proves it, closes with evidence |
qa-retest |
the finder, later | Re-tests what was closed against what is actually deployed, reopens what did not change |
They are meant to run in separate sessions. One side finds, the other side fixes, and a person decides between them. Collapsing them into one loses the thing that makes it work: nobody grades their own homework.
Sweep, do not click. State a rule the data must obey, run it over every row, read the count before the examples. Clicking finds the bug in front of you. A sweep finds all of them, and tells you how many people are affected — which is what decides whether anyone should care.
Rules that have found real bugs:
- no refund exceeds what was actually collected
- every field a form marks required is populated in the data that form will edit
- every address that takes a record id refuses somebody else's record
- nothing switched off is still being sent
- every button in an email opens the page its label promises
Verify before filing. Most of what looks wrong at first is not. Is it deliberate — is there a comment explaining the choice? Did you measure the right field? Does it reach a person, or only a column? Is it live code or inherited data? Is it your own test data? Candidates that do not survive these are worth more as a note than as an issue.
Record what is sound. "All 931 active subscriptions were compared against what the customer last paid; not one would be charged more" is worth more before a launch than a list of small bugs — and it stops the next person re-treading the ground.
Optional, and worth it after the first run. Add docs/agents/qa.md describing:
- how to reach real data read-only, and what must never be written to
- the invariants that matter in this system, in its own words
- what test data looks like, so sweeps do not report it
- where the deployed version is recorded, so
qa-retestcan compare it with the main branch
MIT