Skip to content

Track 3: ZeroToNaNTransform heuristics are inert when applied to full multi-month trajectories #89

Description

@NarayanSchuetz

From a full-repo review (line refs from ecb7fe0).

src/data/transforms/nan_transforms.py:237-324 rules are parameterized for 1440-min single days (all-zero channel = "not worn"; short_sleep_threshold=180 min/day), but forecasting_evaluation/data/online_dataset.py:315/484 applies the transform once to whole multi-month (19, T_total) trajectories:

  • the all-zero rule fires only if a channel is zero for the entire study;
  • total sleep summed across months is always ≥ 180 min, so the short-sleep rule never fires.

Watch-off-overnight zeros therefore remain as ground truth in the binary sleep channels that AUROC/AUPRC score. Applied identically to all models (so relative comparisons survive) and partially mitigated by the day_remain_mask wear filter — but GT semantics silently diverge from the imputation track and from the transform's documented intent, and wear-pattern differences across demographics can bleed into subgroup metrics.

Suggested fix: apply the transform per-day (reshape to days before the rules), or re-parameterize the rules for trajectory length; add a test with a synthetic watch-off night asserting the zeros become NaN.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmedium priorityMedium priority

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions