validation-gate-stack

An open-source, IP-clean implementation of the statistical validation gates used in institutional systematic-strategy research.

Every gate is a pure function on a returns / P&L series, named so a research pipeline can read like a senior-report checklist.

One-line metric	Real-edge composite PASS ✓ · Placebo composite FAIL ✓ on the same marginals. The asymmetry is the deliverable.
Run the demo	python3 run_demo.py (numpy + matplotlib; no scipy)
Run the tests	python3 -m pytest tests/ -q
Use as a library	from gate_stack import g7_sharpe, g8_block_bootstrap_ci, g21_deflated_sharpe, …
Author / contact	@christianmacion26

---

What's here

validation-gate-stack/
├── README.md            ← you are here
├── memo.md              ← design rationale + honest-scope notes
├── gate_stack/
│   ├── __init__.py      ← public API
│   └── gates.py         ← the gate functions (24 gates, 6 levels)
├── run_demo.py          ← end-to-end demo: real edge vs placebo
├── plot_results.py      ← renders a heatmap of the gate report
├── tests/
│   └── test_gates.py    ← pytest suite for every gate
├── results/             ← populated by run_demo.py + plot_results.py
└── requirements.txt

---

Why this exists

A single statistical test (Sharpe, t-stat, bootstrap CI, …) is rarely enough. Each one answers a different question, and any one can be fooled by a specific failure mode — small-sample noise, autocorrelation, multiple-testing inflation, regime-dependence, parameter fragility.

The point of stacking is:

1. Each gate catches a distinct pathology.
2. A k-of-N composite gives a single "go / no-go" with auditable reasons.
3. The asymmetry between a real edge and a placebo — one passes, the other doesn't — is what makes the filter worth shipping.

This is the methodology core of how senior researchers think about a candidate: structured, multi-axis, auditable. The library implements the SEMANTICS; the thresholds are tunable.

---

The gate catalog (24 gates, 6 levels)

Level 1 — Data sanity & preflight

Gate	Tests	Default threshold
g1_no_nan_or_extreme	no NaN / inf, no > 8σ outliers	0 outliers
g2_sample_size	n ≥ N_min (small-sample stats are noise)	n ≥ 60
g3_distribution_reasonableness	skewness and kurtosis plausible	|skew| ≤ 3, kurt ≤ 12

Level 2 — Statistical significance

Gate	Tests	Default threshold
g4_tstat	per-period t-statistic vs zero	t ≥ 2.0
g5_monotone_cumsum	cumulative P&L mostly monotonic (sanity)	≥ 85% up-steps
g6_stationarity	simplified ADF t-stat on cumulative P&L	t ≤ -2.86 (1%)
g7_sharpe	annualized Sharpe ≥ min	≥ 0.5
g8_block_bootstrap_ci	block-bootstrap CI on Sharpe	lower bound > 0
g9_random_timing_null	actual SR > p95 of scrambled SRs	> p95(null)

Level 3 — Robustness

Gate	Tests	Default threshold
g10_hold_period_robustness	Sharpe at multiple hold periods	≥ 60% pass
g11_leverage_robustness	Sharpe scaled by leverage stays positive	≥ 50% pass
g12_net_of_cost	Sharpe net of per-period cost	≥ 0.2
g13_regime_robustness	positive Sharpe across vol regimes	≥ 50% pass
g14_is_oos_ratio	OOS Sharpe ≥ min AND IS/OOS ratio reasonable	OOS ≥ 0.2, IS/OOS ≤ 2.0
g15_turnover_adjusted	Sharpe net of turnover cost ≈ gross	net/gross ≥ 50%
g16_lo_adjusted_sharpe	Lo (2002) autocorrelation-corrected Sharpe	≥ 0.5

Level 4 — Multiple-testing & search bias

Gate	Tests	Default threshold
g17_bonferroni	Bonferroni-adjusted p-value for m trials	p < α/m
g18_param_sensitivity	robustness to ±10% parameter perturbations	≥ 70% retained
g19_regime_diversity	at least N distinct regimes produce positive Sharpe	≥ 3 distinct
g20_minbtl	Bailey-López-de-Prado minimum backtest length	n ≥ MinBTL

Level 5 — Composite / portfolio

Gate	Tests	Default threshold
g21_deflated_sharpe	DSR for m trials (Bailey & López de Prado 2014)	DSR > 0.5
g22_pbo	Probability of Backtest Overfitting via CSCV	PBO ≤ 0.5
g24_white_reality_check	sign-consistency of sub-period means	p < 0.05

Level 6 — Runners

Runner	What it does
run_required_gates	runs the "every required gate must pass" suite
k_of_n_summary	aggregates a list of gate results into a single verdict

---

Quick start

git clone https://github.com/christianmacion26/validation-gate-stack
cd validation-gate-stack
python3 -m pip install -r requirements.txt   # numpy + matplotlib only
python3 run_demo.py                          # end-to-end demo

The demo:

1. Synthesizes a 5-year daily returns series with a regime-conditional edge (annualized Sharpe ≈ 1.4).
2. Synthesizes a placebo with the same marginals and zero edge.
3. Runs the full required-gate suite + Level-5 composites on each.
4. Prints a side-by-side report and asserts the real-edge composite passes while the placebo composite fails.

FINAL — Real edge composite: PASS ✓
FINAL — Placebo composite:   FAIL ✓
        (the gate stack earns its keep when placebo → FAIL)

Use as a library

import numpy as np
from gate_stack import (
    g4_tstat, g7_sharpe, g8_block_bootstrap_ci,
    g21_deflated_sharpe, run_required_gates, k_of_n_summary,
)

returns = np.loadtxt("my_backtest_returns.csv")
results = run_required_gates(returns, n_trials_m=12)
dsr = g21_deflated_sharpe(returns, n_trials_m=12)

for r in results + [dsr]:
    print(f"  {r.name:<28} {'PASS' if r.passed else 'FAIL':<6} "
          f"{r.value:>10}  {r.detail}")

ok, n_pass, total = k_of_n_summary(results + [dsr], fraction=0.5)
print(f"\nComposite: {n_pass}/{total} gates pass · {'PASS' if ok else 'FAIL'}")

Every gate is a pure function. Each returns a GateResult dataclass:

@dataclass
class GateResult:
    name: str
    passed: bool
    value: float
    threshold: float
    detail: str
    extras: dict = field(default_factory=dict)

---

Honest scope

This is a small, IP-clean re-derivation on synthetic data — no proprietary signals, no firm-specific thresholds. It does not claim to implement every nuance of Bailey & López de Prado (e.g., full PBO requires a strategy ENSEMBLE; we synthesize one). It does claim:

Given a returns series, this library produces a structured, auditable, multi-gate report — the SEMANTICS of how senior researchers think about a candidate.

Thresholds are tunable. Gates can be added or removed. The gate_stack package has zero runtime dependencies beyond NumPy (and matplotlib only for the demo's optional heatmap).

---

Tests

python3 -m pytest tests/ -q

Every gate has at least one positive-test and one negative-test case.

---

Related repos in this portfolio

• multiple-testing-deflated-sharpe — the headline application of G21 / G29 to BTC data.
• bias-audit — the look-ahead bias shift test, the concrete use-case for G18.
• rag-recall — the AI-side sibling: same engineering signature (define quality numerically, measure honestly, let the number gate).

Built June 2026 by Christian Macion.