Why can a good backtest fail in live trading?
Check information boundaries, execution assumptions and independent validation before trusting a curve.
A backtest calculates a historical outcome under assumptions. It does not establish future profitability. Investigate whether the information was available at the time, whether orders could fill as assumed, and whether repeated parameter selection has contaminated the test period.
What does a backtest establish?
It records the outcome for specified data, rules, execution assumptions and costs. Save those inputs with the result so someone can reproduce the calculation. A simulation cannot establish that future market structure or actual fills will match.
Was the information available at the decision time?
Record availability timestamps. Fitting preprocessing on the full dataset or selecting features after inspecting the test set can leak information. The scikit-learn documentation recommends keeping training and test processing isolated.
Could the order really fill at that price?
A signal depending on a closing price needs an explicit execution assumption. Check submission time, order type, slippage, fees and unfilled orders. A touched limit price alone does not demonstrate that your order fully executed.
Is the out-of-sample period still independent?
Reserve the test period before developing rules. Repeatedly changing a strategy in response to that period makes it part of the research process. Retain the unsuccessful attempts as well as the best curve.
What records should survive the research?
Keep data and code versions, the strategy specification, order logs, cost assumptions and failure cases. Examine nearby parameters, cost stress and separate time periods before deciding on further validation. These checks cannot guarantee returns.
Questions
Does a strong backtest prove a strategy works?
It establishes a historical calculation under assumptions, which still needs independent validation.
Does weaker out-of-sample performance always invalidate a strategy?
A single metric is insufficient. First check comparability of samples, costs and market conditions, and document the next test.
Sources and scope
External sources support the definitions and method discussion; they do not validate WeiMind’s research results.
For education and research only. No investment advice or promised returns. Backtests do not predict future performance. Any live demonstrations are historical records delayed by at least one trading day.