Fix mt.ora selecting the bottom instead of the top n_up features - #347
Merged
Conversation
`sts.rankdata` returns ascending ranks, so the top `n_up` features are the ones with the largest ranks. `row > n_up` therefore kept everything but the bottom `n_up` features, i.e. 95% of them under the default `n_up`. Compare against `nvar - n_up` instead, as decoupler<2 did via `n_up_msk`. As a side effect of selecting ~all features as observed, `a + c` could exceed `n_bg`, making the contingency table invalid and crashing `_runora` inside numba with an opaque SystemError. Validate `n_up`/`n_bm`/`n_bg` up front so that misconfigurations report what is actually wrong, and do the same in `query_set`, which shares the `d = n_bg - a - b - c` background subtraction. Also fix `n_bm` selecting the bottom `n_bm - 1` features instead of `n_bm`. The existing tests reimplemented the buggy `row > n_up` convention in their reference implementation, which is why they passed; they now use the correct one, plus new tests pinning the exact contingency table for top/bottom selection and covering the reported crash. Fixes #346 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #347 +/- ##
=======================================
Coverage 94.00% 94.00%
=======================================
Files 79 79
Lines 4119 4123 +4
=======================================
+ Hits 3872 3876 +4
Misses 247 247
🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #346.
The bug
src/decoupler/mt/_ora.py:Ranks are ascending, so the top features hold the largest ranks.
row > n_uptherefore keptnvar - n_upfeatures — everything but the bottomn_up— rather than the topn_up. With the defaultn_up(top 5%), 95% of the features were selected as observed.decoupler<2compared againstlen(c) - n_up; the subtraction was lost in the rewrite.The reported crash follows from that: selecting ~all features makes
a + cexceedn_bg, soab + ac > abcd + atrips theinvalid contingency tableguard in_mlnTest2t, which surfaces fromprangeas an opaqueSystemError. Reproduced withnvar=1000, n_bg=100; the reporter's numbers (ac=23750 = 25000 - 1250,abcd=20000 = n_bg) match exactly.Changes
nvar - n_upso the topn_upfeatures are selected.n_bmselected the bottomn_bm - 1features (row < n_bm); nowrow <= n_bm. Harmless at the defaultn_bm=0, wrong for any explicit value.decoupler<2usedn_bottom + 1as the threshold.decoupler<2had, and rejectn_bg < n_up + n_bmup front. That condition is exact:a + calways equalsn_up + n_bm, so a smallern_bgguarantees a negativedregardless of the feature set. It would have caught the report with a message naming the actual problem.mt.query_setshares thed = n_bg - a - b - csubtraction and dies with scipy'sValueError: All values in 'table' must be nonnegativewhen the query set is larger thann_bg. It now asserts with a message that names the source and the count. This is a validation gap rather than a correctness bug — the input is user-supplied there — but it is the same sharp edge.n_updocstring: the default is the top 5% of all features, not of positive ones.mt.aucellis not affected — it ranks-row, so itsx <= n_upcorrectly takes the topn_up.Why CI did not catch this
tests/mt/test_ora.pyreimplemented the buggy convention in its own reference implementation (row > n_up), so it agreed with the code under test. Those reference implementations now use the correct convention, plus:test_func_ora_selection— pins the exact contingency table against a hand-built matrix with known ordering, parametrized overn_up,n_bm,n_up == nvarand a floatn_up. This distinguishes the correct table from the buggy one rather than just checking the sign.test_func_ora_validate— the two new guards.test_ora_wide— the reported crash (nvar - n_up > n_bg).test_query_set_n_bg— thequery_setguard.All 8 of these fail on the released code and pass here. Full suite: 242 passed (excluding
tests/opandtests/ds, which need network access).Release
Adds a
## 2.2.1CHANGELOG section and bumpsversioninpyproject.tomlto match.🤖 Generated with Claude Code