Skip to content

build: bound ruff to >=0.16,<0.17 and make lint config explicit - #69

Merged
abrichr merged 1 commit into
mainfrom
build/bound-ruff-0-16
Jul 28, 2026
Merged

build: bound ruff to >=0.16,<0.17 and make lint config explicit#69
abrichr merged 1 commit into
mainfrom
build/bound-ruff-0-16

Conversation

@abrichr

@abrichr abrichr commented Jul 28, 2026

Copy link
Copy Markdown
Member

Why

openadapt-ml declared ruff>=0.1.0 in dev deps and had no [tool.ruff] section at all — no line-length, no target-version, no select, no include. Every lint decision was inherited from whatever ruff version a runner happened to resolve.

ruff 0.16.0 (published 2026-07-23) then changed both halves of that implicit contract at once:

0.14.9 0.16.0
default rules 59 413
ruff check openadapt_ml/ 0 findings 1035 findings
default include no *.md *.md added
ruff format --check openadapt_ml/ 109 files clean 3 .md files + 1 .py file would reformat

Neither change needed a commit in this repository to take effect. main is green today only because uv.lock happens to pin ruff 0.14.9 — I confirmed against the most recent main run (30316173438, job log shows + ruff==0.14.9) that CI would otherwise already be red.

This follows the same fix already merged in openadapt-consilium (#11, 0→54) and OmniMCP (#34, 0→375).

The rule-set decision

An explicit bounded select, not ruff's defaults. 1035 findings in openadapt_ml/ is not a set anyone can review honestly in one change, and it is overwhelmingly not defect-finding:

family count why not now
UP 581 Optional[X]X | None, Listlist. Pure typing churn.
BLE001 156 Blind except Exception on best-effort dashboard/VM-status probes that are deliberately blind.
PLW1510 73 subprocess.run without check=. Every fix is a behaviour change needing per-call judgement.
I001 82 Real — adopted.

Adopting 413 rules by accident is the opposite of the predictability this PR is for. Each of those families can be adopted later, deliberately, on its own merits.

Chosen: select = ["B", "E", "F", "I", "W"], ignore = ["E501"].

  • E,F,I,W is the OpenAdapt house selection (openadapt-capture, -privacy, -telemetry, -crier, -herald, -desktop, OpenAdapt). Note this is wider than what this repo actually enforced before — ruff's old default was effectively E4,E7,E9,F, so I and the rest of E/W are net new coverage, not a retreat.
  • B (flake8-bugbear) is added on top because it is the one extra family that pays for itself here: 36 findings repo-wide, all real defect shapes rather than style. It guards code paths that pair model outputs against ground truth, where a silent length mismatch is exactly the class of quiet wrong answer this project cares about.

[tool.ruff] is now explicit: line-length = 88 (what the tree already uses — the house value of 100 would rewrap 70 of 112 files for no behavioural gain and bury this change), target-version = "py310", and include pinned to 0.16's default list minus *.md, so a future addition to either default list is inert until someone edits this file.

Reasons are written inline in pyproject.toml, not just here.

Defects fixed

Real behavioural defects:

  • scripts/generate_vm_screenshots_simple.py:84,88 — two bare except: around PIL font loading. These also caught KeyboardInterrupt and SystemExit, so pressing Ctrl-C while a font was loading was silently swallowed and fell through to the fallback font instead of exiting. Narrowed to except OSError (what ImageFont.truetype actually raises).
  • openadapt_ml/training/grpo/config.py:81,94task_dir declared twice in the same dataclass, the documented declaration shadowed by a later bare one. Both defaulted to None, so behaviour is unchanged today, but editing the documented default would silently have had no effect. This is the same shape as the config constant shadowing found in the consilium work.
  • scripts/p0_validate_demo_persistence.py:95,124,184 — three "Episode" annotations naming a symbol the module never imports. Honest caveat: from __future__ import annotations is in effect, so these were not runtime NameErrors — but they were unresolvable to typing.get_type_hints() or any runtime introspection. Now imported under TYPE_CHECKING.
  • 10 × B904 — exception chaining restored (from e), so a friendly "install pyarrow" / "install sentence-transformers" ImportError no longer hides the underlying cause. A broken transitive install surfacing as an ImportError deep inside a package is usually the actually-useful part of the message.

B905 — 16 × zip() now declares its intent, with no behaviour change:

  • strict=True only where equality is already guaranteed by an adjacent len() guard or by construction, so it can never fire.
  • strict=False with an inline reason where truncation is genuinely tolerated (frame_describer._load_recording has four backends and none guarantees one action per frame; grpo_e2e_report renders arbitrary on-disk JSON and must not crash on it; CanonicalEpisode.source_recordings / source_episode_ids are documented as related but not declared parallel — flagged as a latent smell, not changed).
  • The one substantive case is representation_shootout/evaluator.py:527, where a short model_outputs could silently shrink the sample set the aggregated metrics are computed over. Its docstring already declared the correspondence as a precondition; that precondition is now enforced.

Mechanical (cosmetic, no behaviour change): 130 × I001 import sorting, 54 × F541 f-strings without placeholders, 37 × F401 unused imports, 5 × F811 redundant import os shadowing the module-level import, 4 × F841 dead bindings, 3 × E712 == True/False in assertions, 10 × B007 unused loop variables.

No import removal touched openadapt_ml/. I diffed the imported-name set of every changed file: all 21 removals are in scripts/, tests/ and examples/. The __init__.py changes are pure reordering — nothing was dropped, and re-exports including the TextEmbedder/TFIDFEmbedder backward-compat alias were verified importable.

Rules deliberately disabled

Exactly one, at config level:

  • E501 (line too long) — line length is left to the formatter. ruff format already wraps what it can; the residue is long string literals, URLs and comments that E501 would only push into worse line breaks. Every OpenAdapt repository ignores it for this reason. Reason is inline in pyproject.toml.

Two narrow # noqa: F401, each with an inline reason, on optional-dependency availability probes (tests/test_api_adapter.py, tests/test_trl_trainer.py) where the import is the test. importlib.util.find_spec — the rule's own suggestion — only proves a module is locatable, not that it imports, which is exactly what these guards need on a partial --all-extras install. Preferred over a whole-file per-file-ignore.

Reviewed and deliberately not changed

  • 33 × S110/S112 (try/except/pass and /continue). I read every one. All are best-effort dashboard, telemetry, VM-status and cache-read probes that degrade to a documented default. None swallows a credential or decision-path failure — i.e. nothing of the shape found in the consilium work.
  • 7 × RUF012 mutable class defaults — all read-only class-level registries (SUPPORTED_MODELS, DEFAULT_TUNNELS, RELEVANT_EVENT_TYPES). No mutation bug.
  • B006 mutable argument defaults — none exist in this tree.
  • 3 × PLW0603 global statements — benign lazy singletons.

CI gate

ruff check widened from openadapt_ml/ to the whole repository. Every genuine defect above was in scripts/ or tests/ — outside the old gate — so fixing them without widening would let them regress immediately. Ruff runs in well under a second; this costs no measurable CI time.

ruff format --check deliberately stays scoped to openadapt_ml/: 37 files under scripts/, tests/ and examples/ have never been formatted, and normalising them is a separate, purely cosmetic change.

Lockfile

uv.lock relocked so CI genuinely exercises the new bound (a bound CI never runs proves nothing). The relock also repairs a lockfile that predated the build and openadapt-types dependencies. Despite the line count, the semantic diff is small and safe:

  • version changes: ruff 0.14.9 → 0.16.0, and this package's own stale self-version 0.9.0 → 0.16.1
  • 21 additions (the previously-unlocked build, openadapt-types, and openadapt-capture[share]'s chain)
  • zero removals, zero downgrades

The remaining churn is a lockfile format bump (revision = 3) adding upload-time metadata.

Release

build:, not fix:patch_tags = ["fix", "perf"], so this deliberately does not trigger a release. Nothing in the shipped openadapt_ml package changes behaviour for a user: the bare-except and undefined-name defects are both in scripts/, which is not packaged, and the duplicate dataclass field had identical defaults on both declarations. Burning a release and a 4-leg matrix on that is not warranted; the B904 chaining will ride along with the next real release.

Verification

Run locally against ruff 0.16.0 installed from the updated lock, using CI's exact commands:

uv sync --all-extras                        # + ruff==0.16.0, uv.lock unchanged
uv run ruff check .                          # All checks passed!
uv run ruff format --check openadapt_ml/     # 109 files already formatted
uv run pytest tests/ -v --ignore=tests/integration
                                             # 454 passed, 2 skipped

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

`ruff>=0.1.0` with no `[tool.ruff]` section at all meant every lint
decision in this repository was inherited from whatever ruff version a
runner happened to resolve. ruff 0.16.0 (2026-07-23) then changed both
halves of that implicit contract at once:

  * the default rule set grew from 59 rules to 413, taking
    `ruff check openadapt_ml/` from 0 findings to 1035; and
  * the default `include` gained `*.md`, so `ruff format` began rewriting
    Python code blocks inside README/USAGE documentation (3 files).

Neither needed a commit here to land. `main` is still green only because
uv.lock happens to pin ruff 0.14.9.

Changes:

  * dev dependency bounded to `ruff>=0.16,<0.17`, and uv.lock relocked so
    CI actually runs 0.16.0 against the new gate. The relock also repairs
    a lockfile that predated the `build` and `openadapt-types`
    dependencies; the only version changes are ruff and this package's own
    stale self-version, with no downgrades and no removals.
  * explicit `[tool.ruff]`: `line-length = 88` (what the tree already
    uses), `target-version = "py310"`, and `include` pinned to 0.16's
    default list minus `*.md`, so a future addition to either default is
    inert until someone edits this file.
  * explicit `select = ["B", "E", "F", "I", "W"]` with `ignore = ["E501"]`.
    This is the OpenAdapt house set plus flake8-bugbear, and is wider than
    what this repository actually enforced before (ruff's old default was
    effectively `E4,E7,E9,F`). Full 0.16 defaults were rejected: 1035
    findings is not honestly reviewable, and ~90% of it is `UP` typing
    churn, `BLE001` on deliberately blind status probes, and `PLW1510`,
    whose fixes are all behaviour changes. Reasons are recorded inline.
  * `ruff check` in CI widened from `openadapt_ml/` to the whole
    repository, because every genuine defect the audit surfaced was in
    scripts/ or tests/, outside the old gate.

Defects fixed:

  * scripts/generate_vm_screenshots_simple.py: two bare `except:` around
    PIL font loading also swallowed KeyboardInterrupt and SystemExit, so
    Ctrl-C during font load silently fell through to the fallback font
    instead of exiting. Narrowed to `except OSError`.
  * openadapt_ml/training/grpo/config.py: `task_dir` was declared twice in
    the same dataclass, with the documented declaration shadowed by a
    later bare one. Both defaulted to None so behaviour was unchanged, but
    editing the documented default would silently have had no effect.
  * scripts/p0_validate_demo_persistence.py: three `"Episode"` annotations
    referenced a name the module never imports. `from __future__ import
    annotations` keeps this from raising at import time, but the
    annotations were unresolvable to any runtime introspection. Imported
    under TYPE_CHECKING.
  * 10 x B904: exception chaining restored (`from e`) so a friendly
    "install X" ImportError no longer hides the underlying cause.
  * 16 x B905: every `zip()` now declares its intent. `strict=True` only
    where equality is already guaranteed by an adjacent guard or by
    construction, so it can never fire; `strict=False` with an inline
    reason where truncation is genuinely tolerated. The one substantive
    case is representation_shootout/evaluator.py, where a short
    `model_outputs` could silently shrink the sample set the aggregated
    metrics are computed over; its docstring already declared the
    correspondence, which is now enforced.
  * 5 x F811 redundant `import os` shadowing the module-level import,
    4 x F841 dead bindings, 3 x E712 `== True/False` in assertions.

Reviewed and deliberately not changed: 33 S110/S112 sites are all
best-effort dashboard, telemetry and VM-status probes that degrade to a
default; none swallows a credential or decision-path failure. 7 RUF012
sites are read-only class-level registries. No B006 mutable default
arguments exist in this tree.

`build:` rather than `fix:`: nothing in the shipped `openadapt_ml`
package changes behaviour for a user. The bare-except and undefined-name
defects are both in `scripts/`, which is not packaged, and the duplicate
dataclass field had identical defaults. This should not burn a release.

Verified locally with ruff 0.16.0: `ruff check .` clean repository-wide,
`ruff format --check openadapt_ml/` clean, 454 passed / 2 skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
@abrichr
abrichr merged commit ab43a9d into main Jul 28, 2026
4 checks passed
@abrichr
abrichr deleted the build/bound-ruff-0-16 branch July 28, 2026 02:33
abrichr added a commit that referenced this pull request Jul 28, 2026
#70)

A failure rendered as a successful empty result is the defect class this
project exists to catch in other people's software. This repository produces
GRPO rewards, grounding metrics and exported artifacts, so here the same shape
does not raise -- it yields a wrong number.

Each site below could not tell "I looked and the answer is empty/zero" from
"I could not look", and returned the first for the second. Each fix makes the
distinction representable rather than logged.

GRPO milestone reward (openadapt_ml/training/grpo/reward.py) -- highest blast
radius, it is the training signal:
  * openadapt-evals not installed -> logged a warning and returned reward 0.0.
  * VLM judge raised -> caught per milestone; the milestone stayed in the
    denominator, so a 429 depressed the reward exactly like an unmet milestone.
  * Milestone with no description -> `continue`d past but still counted, so the
    task could never score 1.0 and nothing said why.
  * No milestones / none of type "screenshot" -> returned 0.0.
All now raise MilestoneEvaluationError. GRPOTrainer catches it at the one place
that can decide, and returns None ("no measurement, keep the existing reward"),
which the caller already handles -- rather than feeding an invented 0.0 into
compute_group_advantages and taking a gradient step on it. GRPOTrainer's unused
`_compute_milestone_reward` raised the same 0.0 for an unknown task_id; it now
raises KeyError, matching its sibling.

Grounding (openadapt_ml/grounding/): `GeminiGrounder.ground` printed the error
and returned []; `_parse_bbox_response` swallowed JSONDecodeError/KeyError/
TypeError; `extract_ui_elements` did both. [] is the legitimate answer for "I
looked and matched nothing", and evaluate_grounder scores it as best_iou 0.0 /
centroid_hit False -- so a rate limit or a bad key was recorded as a grounding
miss the model never made. These now raise GroundingError; the honest empty
result still returns [].

Grounding metrics (openadapt_ml/evals/grounding.py): a step whose screenshot
could not be opened was `continue`d past. Every GroundingMetrics property
divides by len(results), so the sample left the denominator and the reported
hit rate stayed high over a silently smaller sample. Now raises.

Parquet summary (openadapt_ml/export/parquet.py): `except ImportError: return`
made `include_summary=True` a no-op that returned None exactly like success.

Azure login probe (scripts/setup_azure.py): an unrecognised az error returned
True -- "I could not tell, so assume you are logged in". Now re-raises.

Also: PR #69 fixed `task_dir` being declared twice in GRPOConfig (PIE794) and
kept the correct, documented declaration. Nothing in the configured rule set
`B,E,F,I,W` catches a duplicated class field, so the defect could return the
moment the audit stopped. PIE794 is adopted as a single rule (repository count:
0) and tests/test_failure_is_not_success.py adds an AST guard over every
dataclass in the package.

Every test in tests/test_failure_is_not_success.py is mutation-checked: with
the production file reverted to its pre-fix form the matching test fails with
"DID NOT RAISE" (behaviour, not a missing import -- the new exception types are
resolved dynamically for exactly that reason).


Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant