test: fix repo-wide pytest failure — reconcile validate-command fixtures with the no-outputs check (BE-4097)#576
Conversation
…ts server-parity check (BE-4097) Two test_validate_command.py fixtures (from BE-3359 / #553) predate the prompt_no_outputs check (BE-3357 / #551) landing on top of them and have been failing on main since 09de2e5: - test_api_format_unchanged: the bare EmptyLatentImage fixture has no output node, so the CLI now (correctly) exits 1. Replace it with the complete server-valid sd15 txt2img graph — carrying the SaveImage output node every real API-format export has — so the test still proves the 'validated directly, no UI conversion' path at exit 0. - test_empty_dict_payload_unchanged → renamed test_empty_dict_payload_not_converted_and_rejected: rejecting an empty prompt is the intended #551 behavior (the server refuses any zero-output prompt), so assert exit 1 + prompt_no_outputs while keeping the original point — no conversion, no converted_from_ui key. No production code changed.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 11 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Comment |
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @mattmillerai.
✅ No high-signal findings.
Panel: 6/8 reviewers contributed findings.
Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)
Unreviewed admin-merge — justificationThis was merged via admin override ( Justification: this is a no-blast-radius test change.
Because the blast radius is limited to the test suite and CI was fully green, the unreviewed admin merge carries no user-facing or runtime risk. Merge commit: |
ELI-5
Two recently merged PRs each work fine alone but broke the test suite together: #551 taught
comfy validateto reject workflows with no output node (matching what the ComfyUI server does), and #553 added tests whose sample workflows… have no output node. Every PR'sbuildCI has been red since. This PR fixes the two stale test fixtures so they match the intended behavior — no product code changes.What
Repo-wide pytest has been failing on
mainsince 09de2e5 (#553 / BE-3359) landed on top of 354113a (#551 / BE-3357): two fixtures intests/comfy_cli/command/test_validate_command.pytrip the newprompt_no_outputsserver-parity check.test_api_format_unchanged— fixture was a bareEmptyLatentImage(not an output node), so the CLI now correctly exits 1. Replaced with the complete, server-valid sd15 txt2img graph (same graphtest_engine.py::TestValidateServerParity._sd15_fullalready proves clean against the samesd15_object_info.jsoncatalog), which carries theSaveImageoutput node every real API-format export has. The test still proves exactly what it did: an API-format file is validated directly, exit 0, noconverted_from_uikey.test_empty_dict_payload_unchanged→test_empty_dict_payload_not_converted_and_rejected— fixture is{}; rejecting a zero-output (including node-less) prompt is the intended behavior of fix(validate): presence-check required inputs, no-outputs check, hard-error range violations (BE-3357) #551, per the check's own code comment incomfy_cli/cql/engine.py(mirroring ComfyUI serverexecution.py:1155-1162). Updated to assert exit 1 +prompt_no_outputs, while keeping the test's original point: the empty dict is not treated as UI format (no conversion, noconverted_from_uikey). Renamed since "unchanged" no longer describes it.Why this isn't a new dead-end
The flipped assertion (
exit_code == 1for{}) does not introduce a capability denial — it aligns a stale test with behavior already merged and intentionally designed in #551, asserted identically by the engine's own unit test (tests/comfy_cli/cql/test_engine.py::test_empty_workflow_is_no_outputs) and grounded in real server behavior (ComfyUIexecution.pyrejects zero-output prompts withprompt_no_outputs). The ticket calls this reconciliation "mechanical, no design call."Verification
uv run --extra dev pytest .— 2640 passed, 37 skipped, 0 failed (main currently fails 2)ruff check+ruff format --checkwith the CI-pinnedruff==0.15.15— cleanJudgment calls
_sd15_fullfromtest_engine.py— cross-test-module imports are worse coupling than one duplicated literal fixture, and this file already inlines all its fixtures.Note: the "Windows Specific Commands" smoke workflow is red on all PRs from an unrelated pre-existing issue (unpinned mixpanel) — not addressed here, out of scope.
Overlap flag
Open PR #573 (BE-3358, feature work) incidentally bundles fixes to these same two tests. This PR is the standalone, minimal unbreaker so
main's CI goes green without waiting on #573's feature review. Whichever lands second takes a small conflict intest_validate_command.py— after this merges, #573 should rebase and keep this branch's version of the two tests (they're equivalent in intent; this one also renames the empty-dict test).