TRADINGSIGNALS / DEDICATED RESEARCH MODULE

Strategy Validation Lab

Compare research-sample and forward holdout assumptions with cost sensitivity and leakage checks.

PRO LAB 03 / OUT-OF-SAMPLE DISCIPLINE

Performance that survives a new sample?

Inspect the difference between a research-period average and a chronologically separated holdout average after additional modeled costs. A positive number is not statistical verification. Selection, leakage and repeated testing must be documented independently.

Validation statusUser-entered summaries only. No strategy code, immutable revision history or broker reconciliation is connected.

Research and holdout results

Research-integrity checks (self-reported only)
FRAGILITY CHECK / NOT A CERTIFICATION

What the holdout really says

Calculate to inspect the gap and cost sensitivity.

Research avg R—
Holdout avg R—
Cost-stressed holdout—
Holdout / research ratio—
This is descriptive arithmetic, not a p-value, a confidence interval, a proof of replicability or a trading recommendation. Multiple testing, intratrade dependence and changing risk scales can invalidate simple inferences.

Why professionals guard the holdout

Repeatedly consulting the holdout while tuning indicators turns a supposed out-of-sample segment into another training set. Record an initial research question, code/data versions, optimization search space, transaction assumptions and a frozen decision rule before opening the holdout. A subsequent strategy change needs another independent test rather than a rewritten history.

Even a chronological split is not always independent: overlapping trades, serial correlation, related instruments and liquidity conditions can make apparent sample size misleading. Evaluate execution realism and data provenance before claiming anything about performance.

Case study: a fragile positive result

A fictional strategy averages +0.28R across 130 research trades but only +0.07R across 45 holdout trades. The held-out mean is just 25% of the research-period mean. Adding an omitted 0.06R cost leaves only +0.01R. Doubling that extra cost turns the result into −0.05R. The apparent benefit is not robust to a modest change in assumptions.

Professional exercise: freeze a rule before testing a new period, estimate implementation costs, and document which observations would invalidate the result. If you have tried twelve parameter variants, the strongest result requires especially skeptical interpretation.

NEXT RESEARCH STEP

Keep the evidence connected.

Each tool is independent. Copy or export your findings, then continue on the relevant dedicated page. An all-in-one legacy view remains available for users who depend on same-page module handoffs.

Previous combined workspace (compatibility)

Risk disclosureTools are for educational research, do not rely on verified live quotes and never submit or execute orders.