Backtesting Without Cheating - Overfitting and Lookahead

Introduction

Backtesting is like a microscope - it can show the truth or a great lie, depending on how you use it. 90% of the equity curves you see online are the result of lookahead bias, survivorship bias, ignoring costs, or parametric overfitting. The goal of this lesson is not to teach you how to draw beautiful curves that only go up. The goal is to teach you how to destroy your own system before the market does it for you.

In systematic trading you are not looking for confirmation that the system works. You are looking for evidence that it does not. Only if a system survives every attempt at falsification does it deserve a forward test.

The three cardinal sins of a fake backtest

1. Lookahead Bias - you saw the future

You use data that was not available at the moment of the decision.

Lookahead fix:
Signal on bar N calculated on close[N]
Entry at open[N+1] or close[N+1] with slippage

In Pine: process_orders_on_close=false (default)
In Python: shift(1) on signals

2. Survivorship Bias - you only test the winners

You test the S&P 500 as it is today = 500 companies that survived. You do not test the 1000 that went bankrupt or were delisted. Your “buy the dips” system looks brilliant because the winners came back. The losers missing from the database would have ruined the result.

The fix: use point-in-time databases, for crypto test the top 100 of 2020 that partly no longer exist, for stocks use survivorship-free data (Norgate, CRSP). If you do not have it, at least test on an index rather than a basket of winners.

3. Costs and slippage that do not appear in Excel

Real Equity = Backtest Equity - Commissions - Slippage - Swap - Financing - Spread widened on news

For M5 scalping: 1-2 ticks of slippage = 0.02-0.05% per trade
For 1000 trades a year that is 20-50% of the return eaten

The rule: if you have not subtracted 30% of gross profit for costs on M5-M15, you do not have a result.

Overfitting - why the curve only rises in the past

Overfitting is fitting the system to historical noise instead of to signal. Symptoms: 10 parameters, a perfect curve, a Sharpe of 4.0 in the backtest, collapse in the forward test.

Overfitting metric - Parameter Stability:

Test: you change every parameter by +-20%
If EV drops by <25% -> the parameter is stable
If EV drops by >50% or changes sign -> overfitting

The 1:5 rule:
For every 5 trades in the OOS sample, a maximum of 1 optimised parameter.
Have 100 OOS trades = max 20 combinations? No, max 4 parameters.
Have 100 trades = max 1-2 parameters. The rest must come from market logic, not optimisation.

The honest backtest protocol - the KursTradera checklist

Before every backtest, go through 7 steps:

Step 1 - Definition: Write the hypothesis down in 1 sentence: “I buy a pullback in an uptrend defined as … because I assume continuation of momentum due to …”

Step 2 - Data: Is the data survivorship-free? Do you have OHLCV + volume? Does it cover at least 1 bear market and 1 bull market?

Step 3 - Costs: Set the commission, 1-2 ticks of slippage, the spread. For crypto 0.05% taker, for futures 1 tick of slippage.

Step 4 - Logic without the future: Every entry at the next bar open. Every indicator calculated on close[1] or with shift(1).

Step 5 - IS/OOS split: 70% of the data In-Sample for building, 30% Out-of-Sample for validation. Never optimise on the OOS.

Step 6 - Metrics: Not the equity curve. PF, EV in R, MaxDD, SQN, number of trades, stability. Equity is the result, not the proof.

Step 7 - Destruction: Run randomisation (Monte Carlo), a test that delays the signal by 1 bar, a test on random markets. If the system dies - it is not traded.

The tests that kill the illusion

The random entry test

Replace your entries with random ones using the same SL/TP. If the random ones have a similar PF, your edge is in position management, not in the entry. That is valuable information, but not what you thought it was.

The slippage test

Add +0.1% to every loss and -0.1% to every profit. If the EV changes sign - the system is too fragile.

Python pseudo:
df['pnl_real'] = df['pnl'] - abs(df['pnl'])*0.001 - commission

The market swap test

Test a system built on BTC H1 on ETH H1 and SOL H1 with no re-optimisation. If the PF is <1.0 everywhere outside BTC, the edge is specific and probably over-optimised to BTC.

When the plan says do not trade

1. When the backtest has no defined slippage and commission: There is no result. Costs first, euphoria later.
2. When the IS and the OOS cover the same date range as the optimisation: That is not OOS, that is a split IS. The OOS must come after the IS in time (walk-forward) or be completely separated by markets.
3. When the system has 6 parameters and 120 trades: You have 20 trades per parameter. That is guaranteed overfitting. Reduce the parameters or increase the sample 10x.
4. When you test only on one bull market, 2020-2021: You have not checked a bear market. A long-only system always works in a bull market.
5. When, after adding 1 bar of delay, the system changes EV from +0.3R to -0.1R: You have lookahead. Fix the logic, do not trade.

The mistakes that cost the most

1. Brute-force optimisation in TradingView. Scanning 5000 SMA combinations and picking the best one is a recipe for bankruptcy. Use the OOS, not the top 1 from the IS.
2. Ignoring survivorship bias on stocks. Buying companies from the WIG20 (the Polish blue-chip index) today that were at rock bottom in 2020 makes for an easy backtest. The delisted companies disappeared from the database.
3. Backtesting on M1 without volume or slippage. M1 data from Binance has gaps and 10% wicks. Your 0.2% SL is hit 80% of the time in reality, and 30% of the time in the backtest.
4. Confusing a backtest with a forward test. “I tested it on the last 3 months” is not a backtest, it is a forward test on a small sample. Tell them apart.
5. No decision journal. If you do not write the hypothesis down BEFORE the backtest, your brain will add it AFTER, to fit the result. Keep a pre-registration of ideas.

Measurable exercise

Goal: Destroy one of your systems with anti-cheating tests.

90-minute task:

1. Take any strategy from Lesson 2.
2. Apply 3 tests:

Test A - 1-bar delay:
Shift all signals 1 bar forward (entry at the open of the next bar +1)
Calculate EV in R before and after. Write down the % drop.

Test B - Costs x2:
Double the commission and the slippage. If EV < 0 -> the system is not fit for live trading.

Test C - Random Entry:
Replace the signal with a random one (50% long). Keep the same SL/TP.
Calculate the random PF vs your PF. If the difference is <0.2 -> the edge is not in the entry.

3. Fill in the table:

If the system does not pass 2 out of 3, it goes back to the laboratory, not to the market.

Test yourself

1. Explain the difference between lookahead and survivorship bias and give one example of each.
2. You have 200 IS trades and you optimise 4 parameters. Is that overfitting? Apply the trades/parameter rule.
3. Why is entering at the close of the bar on which the signal was generated lookahead at most real brokers?
4. How does IS differ from OOS and why must the OOS come after the IS in time in a walk-forward?
5. A system has a PF of 2.0 without costs and a PF of 1.05 with costs x2. Is it a system ready for trading? Justify.

Common questions

How many trades does a backtest need to be credible? A minimum of 100, better 300+ to calculate the SQN and the stability. For H1 systems that is 1-2 years of data. For D1 it is 3-5 years (covering different regimes). Fewer than 100 is a game, not a measurement.

Can I use Binance data from 2017 for crypto? You can, but remember the regime change: in 2017 liquidity was 1% of today’s, commissions were different, there were gaps. Split the data into regimes, 2017-2020 vs 2020-2024, and test the stability.

Is overfitting always bad? A small amount of overfitting is unavoidable when you optimise. The problem starts when the IS and OOS results differ by >40% or when a small change in a parameter kills the edge. That is destructive overfitting.

How do I recognise that I have lookahead in Python? If adding .shift(1) to the signal makes the result drop by >30%, you had lookahead. Any signal calculated on close and entering at close is lookahead. Always shift.

Can I backtest on TradingView and trust the result? Yes, if: commissions + slippage are set, there is no repaint, entry is at the next bar, the OOS test is done manually (change the dates), and you export and calculate the metrics in Excel. TradingView is a prototype, not a certification.

---