Python for the Trader - From Backtest to Report

Introduction

Python is not magic that earns money for you. Python is a scalable Excel. If your lab from lesson 4 has 5,000 trades, 5 markets and 3 regimes, Excel starts to choke. Python lets you: pull data, compute 10 systems at once, run Walk-Forward, 10k Monte Carlo simulations and generate a PDF report without clicking.

In this lesson you are not learning Python from scratch like a programmer. You are learning the minimum stack of a systematic trader: pandas, vectorised backtest, R metrics, a report you can reproduce a year from now.

Minimum stack and project structure

You need only: Python 3.10+, pandas, numpy, matplotlib. You do not need 15 libraries.

trading_lab/
  data/ BTCUSDT_H1.csv (OHLCV)
  notebooks/ 01_lab.ipynb
  src/ backtest_engine.py
  reports/ BTC_H1_trend_R.pdf

The rule: notebook for exploration, .py file for a repeatable engine. Do not write everything in the notebook, because in a month you will not reproduce the result.

Installation:
pip install pandas numpy matplotlib yfinance

Pulling data and the first vectorised backtest

Do not start with the backtesting.py library that hides the logic. Write a vectorised engine you can see.

import pandas as pd
import numpy as np

# Load H1 data - columns: timestamp, open, high, low, close, volume
df = pd.read_csv("BTCUSDT_H1.csv", parse_dates=["timestamp"])
df = df.sort_values("timestamp")

# Indicators - pandas only, no magic
df["sma200"] = df["close"].rolling(200).mean()
df["sma20"] = df["close"].rolling(20).mean()
df["atr14"] = (df["high"]-df["low"]).rolling(14).mean()
df["rsi"] = df["close"].rolling(14).apply(lambda x: 100 - (100/(1+x.pct_change().mean())))

# Hypothesis: a pullback inside an uptrend
df["trend"] = df["close"] > df["sma200"]
df["pullback"] = df["close"] < df["sma20"]
df["signal"] = (df["trend"] & df["pullback"]).astype(int)

# NO LOOKAHEAD: shift the signal by 1 bar
df["signal_shifted"] = df["signal"].shift(1).fillna(0)

# R risk parameters
atr_mult_sl = 1.5
atr_mult_tp = 3.0
df["sl_dist"] = df["atr14"] * atr_mult_sl
df["tp_dist"] = df["atr14"] * atr_mult_tp

The logic continues in the position loop:

position = 0
entry_price = 0
sl = 0
tp = 0
trades = []

for i in range(1, len(df)):
    row = df.iloc[i]
    if position == 0 and row["signal_shifted"] == 1:
        position = 1
        entry_price = row["open"]
        sl = entry_price - row["sl_dist"]
        tp = entry_price + row["tp_dist"]
        entry_time = row["timestamp"]
        risk = entry_price - sl
    elif position == 1:
        low = row["low"]
        high = row["high"]
        exit_price = None
        exit_reason = None
        if low ≤ sl:
            exit_price = sl
            exit_reason = "SL"
        elif high >= tp:
            exit_price = tp
            exit_reason = "TP"
        if exit_price is not None:
            pnl = exit_price - entry_price
            r_multiple = pnl / risk if risk !=0 else 0
            trades.append((
                "entry_time": entry_time,
                "exit_time": row["timestamp"],
                "entry": entry_price,
                "exit": exit_price,
                "pnl": pnl,
                "R": r_multiple,
                "reason": exit_reason
            ))
            position = 0

trades_df = pd.DataFrame(trades)

This is your engine. Every line visible, every decision with shift(1).

System metrics in pandas - the same as in Excel but automatic

def system_metrics(trades_df, costs_R=0.05):
    trades_df["R_net"] = trades_df["R"] - costs_R
    N = len(trades_df)
    win_rate = (trades_df["R_net"] > 0).mean()
    avg_win = trades_df.loc[trades_df["R_net"]>0, "R_net"].mean()
    avg_loss = trades_df.loc[trades_df["R_net"]<0, "R_net"].mean()
    ev = trades_df["R_net"].mean()
    pf = trades_df.loc[trades_df["R_net"]>0, "R_net"].sum() / abs(trades_df.loc[trades_df["R_net"]<0, "R_net"].sum())
    sqn = (N**0.5) * (trades_df["R_net"].mean() / trades_df["R_net"].std()) if N>5 else 0

    trades_df["equity_R"] = trades_df["R_net"].cumsum()
    trades_df["peak_R"] = trades_df["equity_R"].cummax()
    trades_df["dd_R"] = trades_df["equity_R"] - trades_df["peak_R"]
    max_dd = trades_df["dd_R"].min()

    return (
        "N": N,
        "WinRate": win_rate,
        "AvgWin_R": avg_win,
        "AvgLoss_R": avg_loss,
        "EV_R": ev,
        "PF": pf,
        "SQN": sqn,
        "MaxDD_R": max_dd
    )

If you cannot compute this in pandas, go back to lesson 4. Python is only meant to speed things up, not replace understanding.

A visualisation that gives a decision, not just a pretty chart

import matplotlib.pyplot as plt

trades_df["equity_R"].plot(title="Equity curve in R - costs included")
plt.axhline(0, color="black", linewidth=0.5)
plt.ylabel("Cumulative R")
plt.show()

trades_df["R_net"].hist(bins=30)
plt.title("R-multiple distribution - do you have a profit tail?")
plt.show()

You are looking for: does the R distribution have a long right tail (a few big winners) or is it symmetric? A trending system must have a right tail. If it does not - filter or TP too close.

The automatic report - this is the product of this lesson

Instead of screenshots, you generate an md report:

metrics = system_metrics(trades_df)

report = f"""
# System Report - BTC H1 Pullback Trend
N: {metrics['N']}
WinRate: {metrics['WinRate']}
EV: {metrics['EV_R']} R
PF: {metrics['PF']}
SQN: {metrics['SQN']}
MaxDD: {metrics['MaxDD_R']} R
"""

with open("reports/report.md","w") as f:
    f.write(report)

In 6 months you will open the report and in 30 seconds you know whether the system has degraded.

When the plan says do not trade

1. When you have no shift(1) on the signal: Your backtest has lookahead. In Python that is one line that destroys the illusion.
2. When you use functions that compute on future data without a shift: Every indicator computed on a bar that contains future data in live trading.
3. When you test on only one CSV file from a bull market: Split into IS/OOS by time, as in lesson 3.
4. When you have no costs_R in the metrics: Commission and slippage must be inside R_net. Without that the report is fiction.
5. When your engine opens 3 positions at once but you compute MaxDD per trade: You have to simulate a portfolio position, otherwise MaxDD is understated.

The mistakes that cost the most

1. Looping over a DataFrame without vectorisation for 1M rows. For learning a loop is fine (readable). For production use vectorisation or numba. But do not optimise prematurely.
2. Pulling data from yfinance without checking for gaps. yfinance has gaps, split adjustment, missing delisted tickers. Validate isna() and volume.
3. Ignoring open vs close. Entering at close in the code but at the next open in reality = a different result. Always entry = open[i+1].
4. No seed for Monte Carlo and no data version. In a month you will not reproduce the result because the data changed or the draw was different. Save the data file hash and the seed.
5. Trusting a backtest library that hides execution. Libraries such as backtrader have cheat_on_close on by default. You have to know what the engine does.

A measurable exercise

Goal: Build a minimal backtest engine and report for 1 market.

Task 2h:

1. Download BTCUSDT H1 from Binance (CSV) - at least 2 years.
2. Copy the engine code from the lesson, run it.
3. Export trades_df to CSV and compare the metrics with Excel from lesson 4 - they must match +/- 5%.
4. Add 2 cost levels: costs_R = 0.05 and costs_R = 0.10. Calculate how PF and EV change.
5. Generate 3 charts: equity in R, R distribution, drawdown in R. Save them to reports/.

Pass criteria:
N > 100
EV_R after costs 0.05 > 0
MaxDD_R computed and saved
Code has shift(1) and entry at open

Check yourself

1. Why shift(1) on the signal in Python and what happens if you skip it?
2. Why is entry at open rather than close safer against lookahead?
3. Write the Expectancy formula in pandas: how do you compute EV in R from trades_df["R"]?
4. How does R differ from R_net and where in the code do you subtract costs?
5. You have 1M H1 rows and you loop for i in range(len(df)) - why can that be slow and what is the solution?

Frequently asked questions

Do I have to know programming to be a systematic trader? No, but you have to know the tools. Excel is enough to start. Python is needed when Excel is too slow or you want to test 10 markets. If you do not want to code, stay with Pine + Excel, but understand the limitations.

Which backtest library is the best? For learning: your own engine (as in the lesson) + pandas. For production: vectorbt or backtesting.py, but only after you understand lookahead. Avoid black boxes at the start.

Can I backtest in Jupyter or do I need an IDE? Jupyter is ideal for this lesson. But move the final engine into a .py file so it can be versioned in git. Jupyter has a tendency to hide state.

How do I pull data without survivorship bias? For crypto: pull the top 50 from a given year, not today. For stocks: buy a Norgate database or use an ETF instead of individual stocks. To start, test an index, not a basket of companies.

Do I have to use AI/ML in Python? No. 90% of durable systems are simple rules like in this lesson. ML adds enormous overfitting if you do not have Walk-Forward and 10k+ trades. R and PF first, then possibly ML.

---