6.3 — Structural Arbitrage I: Cointegration and Pairs Trading

Chapter 6 · Time Series Models for Trading and Risk

Prof. Xuhu Wan

Section 6.3 · Chapter 6 · Learning Statistics with Python

Structural Arbitrage I: Cointegration and Pairs Trading

Time Series Models for Trading and Risk

Prof. Xuhu Wan

ISOM, HKUST Business School · 2026 Edition

Structural Arbitrage I: Cointegration and Pairs Trading

Two drunks leave the pub and wander. Each path is a random walk. But if they are tied together by a rope, the distance between them is stationary. That rope is cointegration — and a stationary spread is the one thing in this chapter that a time-series model can trade directly. You will see two pairs with no rope, one with a rope, and learn why the rope is always tighter in-sample.

First, the trap: a regression of two strangers

Regress Apple’s log price (2015–2024) on the Dow’s log price from 1985–1995, lined up row by row. The two series have nothing to do with each other. Predict the R².

Apple 2015–24 on Dow 1985–95, 2 516 rows each. What will R² and the t-statistic look like?

R² ≈ 0, t ≈ 0 — unrelated series
R² ≈ 0.05, t ≈ 2 — mild chance correlation
R² ≈ 0.3, t ≈ 5
R² ≈ 0.8, t > 100 — a “highly significant” fiction

R² 0.804, t 101.7 — and an ADF p of 0.2138 on the residual: the “relationship” wanders. Rule: a regression between two unit-root series is only meaningful if the residual is stationary. That is the cointegration test.

The test: Engle–Granger in one call

Each series has a unit root (ADF fails).
Regress \(\log P^A_t = \alpha + \beta \log P^B_t + u_t\).
Test \(u_t\) for a unit root — with harsher critical values, because \(\hat\beta\) was chosen to make \(u_t\) look as stationary as possible.

statsmodels.tsa.stattools.coint(a, b) does steps 2–3 and returns (t_stat, p_value, critical_values).

You run a plain ADF on the OLS residual and get p = 0.024, but coint() reports p = 0.080. Which is right?

ADF — coint() is just ADF with more decimals
coint() — the residual’s β was estimated, so the critical values must be harsher
Neither — average them
ADF, because the sample is large

NVDA and SPY: tied by a rope?

Both levels keep their unit roots (NVDA p 0.23, SPY p 0.79). β = 4.38; the naive ADF on the residual says p 0.024, but Engle–Granger says p 0.080. Not cointegrated at 5 %, borderline at 10 %: the rope is loose. The residual plot shows the drunks drifting apart for months at a time.

Formation then trade: the discipline, on a pair with no rope

Estimate the rope on 2023 (formation), then watch the spread in 2024 (trading) — never the other way round.

The 2023 residual already fails (ADF p 0.32), and in 2024 the spread’s mean sits at -0.227 — 1.5 formation-sigmas below zero, and it never came back. A z-score would have fired 17 “entry” days on a spread with no anchor. No rope, no pairs trade. (AAPL vs the S&P over 2015–2024 fails too: coint p = 0.25.)

A rope that holds: Alphabet and the S&P 500, 2015–2020

Five years of daily closes, 1 259 days. Predict: will coint reject at 5 %?

Both have unit roots (p 0.85, 0.80). β = 1.271, Engle–Granger t = -3.800 against a 5 % critical value of -3.341: p = 0.0136. Cointegrated at 5 %. A deviation halves in 24.3 days — slow, but it comes back. This is the rope.

Why does in-sample cointegration overstate tradability?

The full-sample test rejects at p = 0.014. Split into formation (first 3 years) and trading (last 2). What do you expect on the formation window alone?

The same p ≈ 0.01 — cointegration is a property of the stocks
A smaller p — fewer points means less noise
A larger p, possibly above 0.05 — the full-sample β used the trading window to fit itself
p is undefined without both windows

Formation to Oct 2017: p 0.239; to Oct 2018: p 0.113. Neither rejects at 5 %, and the trading-window spread drifts to a mean of 0.071 (1.8 formation-sigmas) in the first case. The rope you can see in 2020 is not the rope you could have tied in 2017. So: re-estimate it as you go.

Walk-forward hedge ratio and the z-score

Every 63 trading days, refit \((\alpha, \beta)\) on the trailing 252 days and hold them for the next 63. The spread uses only ropes that were known at the time; the z-score uses a 40-day rolling mean and std.

16 refits, 1 007 trading days from October 2016. The hedge ratio moves from 0.569 to 1.601 — the rope is not a constant. 110 days have |z| > 2: those are the candidates. Now the rule.

The rule: enter at |z| > 2, exit at |z| < 0.5

Short the spread (short GOOG, long β·SPX) when \(z > 2\); long it when \(z < -2\); close when \(|z|\) falls below 0.5. Yesterday’s position earns today’s spread change.

In a trade 35 % of days, 52 entries and exits. Pairs: cumulative 30.5 %, Sharpe 0.75, max drawdown -11.6 %. Holding GOOG: 68.6 %, Sharpe 0.63, drawdown -36.8 %. Less money, better money — and market-neutral: the book made it through March 2020 with a third of the drawdown. Before costs.

Your turn: a looser trigger

Change entry to 1.5 (keep exit at 0.5) and read off sh_rule. More trades — but is it a better book, or just a busier one?

Entry 1.5: in a trade 47 % of days, 80 round trips, cumulative 53.0 %, Sharpe 1.09, drawdown -11.6 %. Better on paper — with 80 crossings the transaction costs you ignored are now 50 % larger. One tuned threshold on one pair is a hypothesis, not a book.

What you discovered

Regressing two random walks gives R² 0.80 and t 102 from nothing. Only a stationary residual — coint, with its harsher critical values — makes a level regression meaningful.

NVDA/SPY: p 0.080, and the 2024 spread never returned to its 2023 anchor. AAPL/S&P: p 0.25. No rope, no pairs trade.

GOOG/S&P 500 2015–2020: t −3.80, p 0.014, half-life 24 days — cointegrated. But on the formation windows alone p is 0.24 and 0.11: the in-sample rope is always tighter than the one you could have tied.

Walk-forward: refit the hedge ratio (0.57 → 1.60), z-score on 40 days, enter at 2, exit at 0.5: Sharpe 0.75, drawdown −11.6 % vs 0.63 and −36.8 % for GOOG alone.

Next: §6.4 — the hedge ratio as a state that moves every day (Kalman), and the market’s regime as a hidden state (Markov switching).