Skip to content

feat(generate): derive partner model enums from the active openapi spec (BE-3392)#575

Merged
mattmillerai merged 3 commits into
mainfrom
matt/be-3392-spec-derived-model-enums
Jul 23, 2026
Merged

feat(generate): derive partner model enums from the active openapi spec (BE-3392)#575
mattmillerai merged 3 commits into
mainfrom
matt/be-3392-spec-derived-model-enums

Conversation

@mattmillerai

Copy link
Copy Markdown
Collaborator

ELI-5

When ByteDance ships a new Seedance model, users couldn't pick it with comfy generate seedance --model … until someone edited a hardcoded list in the CLI and cut a release. But the openapi spec the CLI already ships (and refreshes from api.comfy.org) lists the valid models as data. Now the CLI reads the model list straight out of the active spec, so a spec refresh alone surfaces new models — no code change, no release.

What changed

  • spec.model_enum(endpoint_id, field="model") — new helper that walks the endpoint's (already $ref-resolved) request schema for properties[field] and returns its string enum, handling enum directly or nested under items / anyOf / oneOf / allOf. Returns None (never raises) when the spec carries no enum there, for an unknown endpoint, or an unknown field.
  • adapters.get() now refreshes the --model flag's enum + default from spec.model_enum lazily at lookup time. Every consumer routes through it (schema.flags_for → help text/comfy generate schema output, parse_args validation errors, resolve_path URL substitution, build_body), so they all see the spec-derived list.
  • Fallback preserved: SEEDANCE_MODELS / GEMINI_IMAGE_MODELS tuples are kept and take effect whenever the spec has no enum. Gemini's model is a URL path param (no request-body enum in the spec today), so its tuple remains the effective list — verified against the vendored spec.
  • Default model: the pinned default (seedance-1-0-pro-250528) survives while the derived enum still lists it; if a future spec drops it, the first enum entry takes over — in the flag default, and in _seedance_build_body's no---model fallback.

Tests

  • spec.model_enum derives from the vendored spec; returns None for enum-less / unknown endpoint / unknown field; _extract_enum shape coverage (direct / items / anyOf / oneOf / non-string enums).
  • Simulated refresh via spec.write_cache fixture: seedance schema lists the new (seedance-2-0-*) models; pinned default kept while present; default falls back to first entry when dropped; hardcoded fallback when the spec's model property has no enum; validation error message names the derived models.
  • Gemini keeps its hardcoded tuple + default (path-param case).

tests/comfy_cli/command/generate/: 185 passed. Full tests/comfy_cli: 2544 passed; the 2 failures in test_validate_command.py (test_api_format_unchanged, test_empty_dict_payload_unchanged) are pre-existing — they fail identically on a clean origin/main tree in this environment. Ruff 0.15.15 (CI's pin) check + format clean.

Judgment calls / out of scope

  • emit.py's MODEL_NODE_MAP fixed={"model": …} values are left hardcoded on purpose: they parameterize the partner node workflow (--emit-workflow), whose valid values are governed by the ComfyUI node's own input enum, not the proxy spec — a different contract from the --model proxy flag this ticket targets.
  • New endpoints/partners (e.g. Seedance 2.0's separate byteplus-seedance2/... polling path family) still require allowlist/adapter code — accepted residual per the ticket (BE-3352).
  • With a refreshed spec, an explicitly passed --model not in the spec's enum is now rejected by validation (previously the stale hardcoded tuple governed). That's the intent: the spec is authoritative when it carries an enum.

…ec (BE-3392)

New spec.model_enum(endpoint_id, field) reads the model-variant enum out of
the active spec's request schema (direct, items, or anyOf/oneOf/allOf).
adapters.get() now refreshes the --model flag's enum/default from it lazily,
so a spec refresh surfaces new Seedance releases with zero code changes; the
hardcoded tuples remain as fallback when the spec carries no enum (Gemini's
model is a path param, so its tuple stays effective). The pinned default
survives while the spec still lists it, else the first enum entry takes over.
@mattmillerai mattmillerai added the agent-coded PR authored by the agent-work loop label Jul 22, 2026
@mattmillerai
mattmillerai marked this pull request as ready for review July 22, 2026 19:41
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 21 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 108af60d-0681-4629-b6a8-791d527b02bc

📥 Commits

Reviewing files that changed from the base of the PR and between 18314a2 and f5934bc.

📒 Files selected for processing (4)
  • comfy_cli/command/generate/adapters.py
  • comfy_cli/command/generate/spec.py
  • tests/comfy_cli/command/generate/test_adapters.py
  • tests/comfy_cli/command/generate/test_spec.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch matt/be-3392-spec-derived-model-enums
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch matt/be-3392-spec-derived-model-enums

Comment @coderabbitai help to get the list of available commands.

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Jul 22, 2026
@mattmillerai mattmillerai added the cursor-review Request Cursor bot review label Jul 22, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 4 finding(s).

Severity Count
🟡 Medium 2
🟢 Low 1
⚪ Nit 1

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)

Comment thread comfy_cli/command/generate/spec.py Outdated
Comment thread comfy_cli/command/generate/adapters.py
Comment thread comfy_cli/command/generate/spec.py Outdated
Comment thread comfy_cli/command/generate/spec.py Outdated
- _extract_enum: union anyOf/oneOf branches instead of first-found,
  intersect allOf branches, and coerce numeric enum members to strings
  so unquoted YAML values aren't silently dropped
- model_enum: descend top-level allOf/anyOf/oneOf composition when the
  request body carries no direct properties
- resolve_path: percent-encode the substituted path param and reject
  dot segments so a tampered spec enum can't redirect the proxied
  request via path traversal

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mattmillerai

Copy link
Copy Markdown
Collaborator Author

Review panel findings addressed in 89acf28 (anyOf/oneOf union + allOf intersection, top-level composition descent, path-param encoding, numeric enum coercion); all threads resolved. Note: the red build check is the 2 pre-existing test_validate_command failures on main (BE-3357×BE-3359) — the fix rides on #573, not this PR. Full suite green locally with those 2 deselected (2649 passed).

@bigcat88 bigcat88 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and verified live against production api.comfy.org. Approving:

  • The headline claim works exactly as designed: with the vendored tuple the CLI knows 4 seedance models; after comfy generate refresh, generate schema seedance derives 8 from the spec's request-body enum — including seedance-1-5-pro-251215 and the three dreamina-seedance-2-0-* releases — with the pinned default seedance-1-0-pro-250528 kept because it's still listed. No code change, no release.
  • Gemini verified live as the documented path-param fallback: still its 3-model hardcoded tuple.
  • The resolve_path hardening (percent-encode the substituted segment, reject ./..) is the right complement to the refreshable-cache trust model from #560 — a tampered spec can't steer the proxied request via path traversal.
  • _extract_enum's union-for-anyOf/oneOf vs intersect-for-allOf semantics are correct, and the bool-excluding numeric coercion covers unquoted YAML values.
  • Confirmed every consumer routes through adapters.get() (client, schema, app ×2) — nothing reads _ADAPTERS directly.
  • Full unit suite on this branch merged with current main: 2554 passed. Mutation check: hard-disabling the spec derivation in _spec_model_flags fails 6 tests — the refresh behavior is pinned, not decorative.

One out-of-scope observation (pre-existing on main, no action needed here): comfy --json generate schema … prints the pretty text rather than an envelope — might deserve its own ticket for the JSON contract.

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Jul 23, 2026
@bigcat88

Copy link
Copy Markdown
Contributor

The red build check is the repo-wide test_validate_command.py failure #551 introduced and #576 fixed on main this morning — this branch's CI ran last night, in between. Update from main and it should go green (full suite passes locally on this branch merged with current main).

@mattmillerai
mattmillerai merged commit 090dc1b into main Jul 23, 2026
16 checks passed
@mattmillerai
mattmillerai deleted the matt/be-3392-spec-derived-model-enums branch July 23, 2026 19:36
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 23, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

agent-coded PR authored by the agent-work loop cursor-review Request Cursor bot review enhancement New feature or request lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants