Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
## 2026-08-04 — E-EXACT-FIT-IS-WHERE-ABSOLUTE-ZERO-GUARDS-BREAK-1 — a misfit guard written as `x < 0.0` rejects the PERFECT case, and the doc example is what caught it

**Status:** FINDING (two measured defects, both fixed in the same commit). **Confidence:** High — reproduced, fixed, and each now carries a regression test. Code: `crates/jc/src/stats.rs` (D-KIA-C1b).

**The defect.** `omega_total` guards against a **Heywood case** — estimated common variance exceeding an item's total variance, `ψ_i = σ_ii − λ_i² < 0`, which means the single-factor model does not fit. Written as `if psi < 0.0 { return None }`, that guard **rejects the best-fitting input there is**: items that are exact multiples of one factor have `ψ = 0` in real arithmetic and land a few ulps *either side* of zero in f64. The perfect model was reported as misfit. Fix: a tolerance **relative to the item's own variance** (`1e-9 · max(|σ_ii|, 1)`), then clamp — only a violation larger than rounding is a real one.

**The generalisation, which is the reusable part.** A guard testing a quantity that is **exactly zero at the ideal** must never use an absolute `< 0` (or `== 0`) comparison. The failure is invisible on ordinary noisy inputs — where `ψ` is comfortably positive — and appears **only on the cleanest possible data**, which is exactly the case a doc example or a smoke test reaches for. Sibling instances to check for: any `variance < 0`, `residual < 0`, `determinant == 0`, or `SS_within <= 0` test in a fitting path.

**The second defect, same commit, different lesson.** The `multiple_r_squared` doc example used predictors `a` and `a + 1` and asserted `R² = 1`. It returned `None` — **correctly**: the design carries an intercept, so `[1, a, a+1]` is rank-deficient. Here the *example* was wrong and the code was right. A duplicated column is the collinearity everyone remembers; **collinearity with the intercept is the one that gets written by accident**, because the two predictor columns are visibly different.

**Why this is filed at all: doc examples are falsifiers, and these two were the only ones that fired.** 31 hand-written unit tests — including hand-computed reference values and five cross-identity checks against independently-computed quantities — passed while both defects were live. The doctests failed. The reason is structural rather than lucky: unit tests here were written around *hand-computable* fixtures (deliberately non-degenerate, so the arithmetic could be checked), while a doc example is written to be *readable*, which selects for the clean, exact, degenerate case — precisely the input class the unit tests avoided. **A test suite optimised for verifiable references systematically under-samples the perfect case.** Consequence for new numeric work: include at least one exact/zero-residual fixture deliberately, and treat a failing doc example as a finding rather than as a doc typo until the code is cleared.

**Paired with the falsifiability rule:** relaxing a guard is the mirror image of adding one, so the same twin applies — `omega_heywood_case_still_returns_none` proves the relaxed guard **still bites** on a real violation (hand-built fixture: `λ₁² = 70/9 ≈ 7.778 > σ₁₁ = 20/3 ≈ 6.667`). Without it, the tolerance fix would have silently disabled the guard, which is the strictly worse defect.

## 2026-08-04 — E-THE-HYGIENE-RULE-RECURSED-1 — a rule that requires an entry per merged PR generates an infinite chain unless hygiene-only PRs are exempted

**Status:** FINDING + rule amendment (termination clause added to `CLAUDE.md` § Mandatory Board-Hygiene Rule). **Confidence:** High — the chain is three links of observed fact, not a projection. Documentation only.
Expand Down
12 changes: 12 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
## 2026-08-04 — branch `claude/x265-x266-plans-review-h9osnl` (PR #887) — D-KIA-C1b SHIPPED: the additive `jc` statistics battery

`crates/jc/src/stats.rs` — **κ** (`cohen_kappa`, the gap that blocked **D3's fusion falsifier**, now closed), **ω** (`omega_total`), **φ** (delegating to `pearson`, `&[bool]` input), **R / R²** (`multiple_r_squared`, OLS with intercept), **η²** (`eta_squared`), and the significance companions (`t_test_one_sample` / `_paired` / `_welch` / `_student`, `anova_one_way`) over one shared regularised-incomplete-beta core. **107 lib + 11 doctests green, clippy-clean.**

**The effect-size family is r** (φ, R, R², η²); **Cohen's d is out by construction** — the t-tests report t/df/p as η²/R²'s significance companion, not as a d-family route.

**Additive constraint held tighter than permitted:** the entire existing-file diff is `fn` → `pub(crate) fn` on `mean` and `all_finite`, plus doc comments and one `pub mod` line. `average_ranks` / `pop_var` were NOT widened (permitted but unused); the new `sample_var` / `sample_cov` use the unbiased `n−1` divisor — a different estimator from `pop_var`'s `n`, not a duplicate.

**Validation is by cross-identity, not self-assertion:** φ vs `pearson`; R² vs `pearson²`; η² vs R² on a dummy; η² vs `t²/(t²+df)` and `F = t²`; ω vs α (equal under tau-equivalence, ω = 0.9473684 > α = 0.8684211 on the hand-computed congeneric fixture).

**Two defects, both surfaced by doc examples** (`E-EXACT-FIT-IS-WHERE-ABSOLUTE-ZERO-GUARDS-BREAK-1`): a Heywood guard written `psi < 0.0` rejected the *perfect* zero-residual model (fixed with a variance-relative tolerance + a can-it-fire test); and a doc example whose two predictors were collinear with the intercept (the example was wrong, the code right).

## 2026-08-04 — branch `claude/x265-x266-plans-review-h9osnl` (PR #884 MERGED `1e90cef`) — D-KIA-C1b scoped: the r-family, additive-only

Board/plan prose only; **no code, no runtime behaviour**. Carried the post-merge arc entries for #881/#882/#883 (see the entry below) and re-scoped the statistics work removed from #883 as its own deliverable, **D-KIA-C1b** (`Queued` — scope, not code).
Expand Down
14 changes: 14 additions & 0 deletions .claude/board/PR_ARC_INVENTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,20 @@
> - **Docs** — knowledge files produced (immutable)
> - **Confidence (YYYY-MM-DD):** — the ONLY mutable field

## 2026-08-04 — lance-graph #887 — D-KIA-C1b: the additive `jc` statistics battery (κ, ω, R/η², t-tests, φ)

**Head:** `<this branch>` (entry written in the same commit as the change, per the hygiene rule's "SAME commit" wording; merge SHA follows on merge). 5 files: 1 new module + 2 visibility lines + 3 board/plan.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Correct the arc file-count inventory.

Line 38 says 5 files, but the supplied stack lists three Rust files and four board/plan files. Update the count and breakdown to list the actual files. Do not count visibility lines as files.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/board/PR_ARC_INVENTORY.md at line 38, Update the file count and
breakdown in the PR inventory entry to accurately reflect the actual files in
the stack: the current count of 5 files is incorrect, and the breakdown should
list the actual composition of three Rust files and four board/plan files
instead of the current enumeration that includes visibility lines. Ensure
visibility lines are not counted as part of the total file count, only actual
files.


- **Added — `crates/jc/src/stats.rs`.** `cohen_kappa`, `omega_total`, `phi`, `multiple_r` / `multiple_r_squared`, `eta_squared`, `t_test_one_sample` / `_paired` / `_welch` / `_student`, `anova_one_way`, plus `TTest` / `Anova` result structs. Every p-value comes from one shared regularised-incomplete-beta core (Lanczos `ln Γ` + modified-Lentz continued fraction), pinned against textbook critical values. **31 new tests: 107 lib + 11 doctests green; `stats.rs` clippy-clean.**
- **Locked — κ was the real gap and it is now closed.** `cohen_kappa` is a *different estimator* from ICC, not a rename: ICC decomposes variance for interval ratings, κ corrects counts for marginal-expected agreement. **This unblocks D3's fusion falsifier.**
- **Locked — the effect-size family is r.** φ, R, R², η². **Cohen's d stays out by construction**; the t-tests are the significance companions to η²/R² (they report t/df/p), not a d-family back door.
- **Locked — φ is not re-implemented.** It delegates to `reliability::pearson` on a 0/1 coding and takes `&[bool]`, so the binary precondition is unforgeable. The marginal-capped ceiling is documented at the function: φ without its marginals is not interpretable.
- **Locked — the additive constraint held, TIGHTER than permitted.** The whole diff to existing files is `fn` → `pub(crate) fn` on **`mean` and `all_finite` only**, plus doc comments and the `pub mod` line. `average_ranks` and `pop_var` were *not* widened — allowed, but unused here; and `sample_var`/`sample_cov` in the new module use the unbiased `n−1` divisor, a different estimator from `pop_var`'s `n`, not a duplicate. No existing statistic's arithmetic, signature or semantics changed.
- **Locked — every new estimator is cross-checked against an independently-computed quantity**, not only against itself: φ vs `pearson`; R² vs `pearson²` at one predictor; η² vs R² on a 0/1 dummy; η² vs `t²/(t²+df)` and `F = t²` from the pooled t; ω vs α (equal under tau-equivalence, strictly greater on the hand-computed congeneric fixture: ω = 0.9473684 vs α = 0.8684211). The one-sample p is pinned to a closed-form incomplete-beta evaluation, never to this code's own output.
- **Corrected — two defects found by the doc examples** (`E-EXACT-FIT-IS-WHERE-ABSOLUTE-ZERO-GUARDS-BREAK-1`): (1) a real bug — the Heywood guard `psi < 0.0` rejected zero-residual models, since ψ = 0 exactly in real arithmetic lands ±ulps in f64; now a variance-relative tolerance, **paired with a can-it-fire test proving the guard still bites**. (2) a wrong example — predictors `a` and `a+1` are collinear *with the intercept*, so `None` was correct.
- **Deferred.** ω²/ε² (bias-corrected η²); weighted κ; multi-rater Fleiss κ; d-family. None blocks D3.
- **Confidence (2026-08-04):** working — 107 lib + 11 doctests green on the pinned toolchain.

## 2026-08-04 — lance-graph #886 — the board-hygiene rule gets a termination clause (it was recursing)

**Head:** `<this branch>` (entry written in the same commit as the change, per the rule's own "SAME commit" wording; merge SHA follows on merge). 3 files, doc/board prose only — **no code, no runtime behaviour.**
Expand Down
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Plan: `.claude/plans/kanban-64k-inverted-awareness-v1.md` (operator anchors a/b;
|---|---|---|---|---|
| D-KIA-0 | jc capability map + dichotomous-statistics decision note (phi/KR-20/kappa naming; Spearman dropped at view 2) | lance-graph | Queued | plan W0 |
| D-KIA-A1 | ⊘ RESCOPED 2026-08-04 (E-ACTOR-IS-NOT-THE-PHASE-PATH-1): #879 is the complete phase-progression path; KanbanActor has no assigned architectural responsibility (legacy compatibility code). SHIPPED: held-owner reschedule/wake. OPEN: run_cycle drained-writer retry guard; missing-owner counter in cognitive_pass | lance-graph | Queued | plan W1 |
| D-KIA-C1b | jc additive-only extension: kappa + McDonald's omega + r-family effect size (R/R-squared, eta-squared = explained variance) + t-test (t/df/p) + a named phi wrapper. Cohen's d explicitly OUT — calculated separately if ever wanted. HARD CONSTRAINT: additive only — pearson/spearman/cronbach_alpha/icc keep their arithmetic, signature and semantics; any diff changing an existing jc statistic is an automatic reject. ONE sanctioned edit: widening reliability.rs private helpers (mean/all_finite/average_ranks/pop_var) to pub(crate) for reuse, visibility only, no body change. C1 audit found phi = pearson-on-binaries (already present in substance) and KR-20 = alpha-on-dichotomous (naming only); kappa absent = the real gap. Blocks D3's fusion falsifier | lance-graph | Queued | plan W0/C1b |
| D-KIA-C1b | jc additive-only extension: kappa + McDonald's omega + r-family effect size (R/R-squared, eta-squared = explained variance) + t-test (t/df/p) + a named phi wrapper. Cohen's d explicitly OUT — calculated separately if ever wanted. HARD CONSTRAINT: additive only — pearson/spearman/cronbach_alpha/icc keep their arithmetic, signature and semantics; any diff changing an existing jc statistic is an automatic reject. ONE sanctioned edit: widening reliability.rs private helpers (mean/all_finite/average_ranks/pop_var) to pub(crate) for reuse, visibility only, no body change. C1 audit found phi = pearson-on-binaries (already present in substance) and KR-20 = alpha-on-dichotomous (naming only); kappa absent = the real gap. SHIPPED as crates/jc/src/stats.rs: cohen_kappa, omega_total, phi, multiple_r/multiple_r_squared, eta_squared, t_test_one_sample/paired/welch/student, anova_one_way; 31 new tests (107 lib + 11 doctests green), clippy-clean. Existing-file diff is visibility-only (mean/all_finite -> pub(crate); average_ranks/pop_var NOT widened, unused). Unblocks D3's fusion falsifier | lance-graph | In PR | plan W0/C1b |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Preserve the append-only board history.

This change rewrites the existing D-KIA-C1b row. It changes the deliverable text and evidence, not only the status or confidence field. Restore the historical row and limit the in-place edit to the allowed status/confidence field. Keep the new shipment details in a new prepended entry.

As per coding guidelines, “governance entries are append-only, with only status/confidence lines updated in place.” Based on learnings, merged historical entries must remain unchanged except for Status and Confidence updates.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/board/STATUS_BOARD.md at line 9, Restore the original D-KIA-C1b row
in STATUS_BOARD.md, changing only its permitted status or confidence field in
place. Move the updated deliverable, evidence, and shipment details into a new
prepended entry while preserving all other historical content unchanged.

Sources: Coding guidelines, Learnings

| D-KIA-A2 | parallelism falsifier (protocol pre-registered: median-of-5, >=2x at >=4k owners, +/-10% stay-silent; kill = regrade claim (a)) | lance-graph | Queued | plan W2 |
| D-KIA-B1 | catalog binary-range criterion contract type + generalized catalog-mirror drift guard | lance-graph | Queued | plan W3 |
| D-KIA-C5 | cohort-statistic witness type under the ELEVATED carve-out + held-out anti-circularity gate | lance-graph | Queued | plan W4 |
Expand Down
26 changes: 26 additions & 0 deletions .claude/plans/kanban-64k-inverted-awareness-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,32 @@ signature, or semantics is an automatic reject, independent of merit.**

Blocks D3.

> **✅ C1b SHIPPED (2026-08-04) — `crates/jc/src/stats.rs`.** All six rows
> delivered: `cohen_kappa`, `omega_total`, `phi`, `multiple_r` /
> `multiple_r_squared`, `eta_squared`, and `t_test_one_sample` / `_paired` /
> `_welch` / `_student` + `anova_one_way` (each returning `t`/`F`, df and p via
> a shared regularised-incomplete-beta core). 31 new tests — **107 lib + 11
> doctests green**, `stats.rs` clippy-clean.
>
> **The additive constraint held, tighter than permitted:** the entire diff to
> existing files is `fn` → `pub(crate) fn` on **`mean` and `all_finite` only**
> (plus doc comments and the `pub mod` line). `average_ranks` and `pop_var`
> were NOT widened — the carve-out allowed it, but this module does not consume
> them, and `sample_var`/`sample_cov` here use the unbiased `n−1` divisor,
> which is a different estimator from `pop_var`'s `n`, not a duplicate of it.
>
> **Every new estimator is cross-checked against independently-computed
> quantities** rather than only against itself: φ vs `pearson` on the 0/1
> coding; R² vs `pearson²` at one predictor; η² vs `R²` on a 0/1 dummy; η² vs
> `t²/(t²+df)` and `F = t²` from the pooled t; ω vs α (equal under
> tau-equivalence, strictly greater under the hand-computed congeneric fixture,
> ω = 0.9473684 vs α = 0.8684211). The t/F tails are pinned against textbook
> critical values, and the one-sample p against a closed-form incomplete-beta
> evaluation — not against this code's own output.
>
> **Two defects found by the doc examples, both fixed** — see
> `E-EXACT-FIT-IS-WHERE-ABSOLUTE-ZERO-GUARDS-BREAK-1`.

**C2 — name the dichotomous statistics correctly.** Over binary catalog
criteria: Pearson→**φ** (report the marginal-capped ceiling), Cronbach's
α→**KR-20**; **κ is a SEPARATE estimator, not a renamed ICC** — where a
Expand Down
10 changes: 10 additions & 0 deletions crates/jc/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,16 @@ pub mod weyl;
// (I-NOISE-FLOOR-JIRAK). See `src/reliability.rs`.
pub mod reliability;

// Second statistics battery (D-KIA-C1b) — chance-corrected agreement (Cohen's
// κ, the estimator `reliability` lacked and D3's fusion falsifier needs),
// congeneric reliability (McDonald's ω), the r-family effect sizes (φ, R, R²,
// η²) and their significance companions (t-tests, one-way ANOVA). Like
// `reliability` it is NOT a pillar. Strictly ADDITIVE: it borrows two private
// helpers from `reliability` (`mean`, `all_finite`) and re-uses `pearson` for
// φ, but changes no existing estimator's arithmetic, signature or semantics.
// Cohen's d is deliberately out of scope — the effect-size family here is r.
pub mod stats;

// PROBE-SIG-CHECKSUM — depth-2 truncated signature as a replayable
// trajectory digest (H.268 probe wave, grades E-WH-TWO-SIDES-SIG-CHECKSUM-1
// leg 2). A probe, not a 12th pillar: intentionally NOT added to the
Expand Down
12 changes: 10 additions & 2 deletions crates/jc/src/reliability.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,12 @@
//! or returning `NaN` — the caller decides how to treat an undefined estimate.

/// Arithmetic mean of a slice, or `None` if empty.
///
/// `pub(crate)` for reuse by [`crate::stats`] (visibility only — the behaviour
/// is unchanged); a second `mean` in that module would be a second source of
/// truth.
#[inline]
fn mean(xs: &[f64]) -> Option<f64> {
pub(crate) fn mean(xs: &[f64]) -> Option<f64> {
if xs.is_empty() {
return None;
}
Expand All @@ -67,8 +71,12 @@ fn mean(xs: &[f64]) -> Option<f64> {
/// as "equal" in the rank step and would otherwise receive an ordinary rank,
/// silently producing a finite-but-garbage Spearman ρ (and `Some(NaN)` for the
/// other three). Every public metric guards on this before computing.
///
/// `pub(crate)` for reuse by [`crate::stats`] (visibility only — the behaviour
/// is unchanged); the no-NaN contract must be enforced identically in both
/// modules, which a copy would not guarantee.
#[inline]
fn all_finite(xs: &[f64]) -> bool {
pub(crate) fn all_finite(xs: &[f64]) -> bool {
xs.iter().all(|v| v.is_finite())
}

Expand Down
Loading
Loading