Chapter 7 · Modern Statistical Learning in Practice
Section 7.1 · Chapter 7 · Learning Statistics with Python
Modern Statistical Learning in Practice
Prof. Xuhu Wan
ISOM, HKUST Business School · 2026 Edition
In §4.5 you trained a boosting model to minimise squared error on next month’s return and then built a top-20 / bottom-20 portfolio from it. The portfolio never used the level of the prediction — only its order. You will show that a monotone transform of the prediction leaves the portfolio untouched while multiplying the MSE a hundredfold, then train three models that target the order directly and put all of them through the §4.5 walk-forward.
Standardise per month exactly as in §4.5 (median / std within the month, clipped at ±3). New this time: the target’s within-month percentile rank, centred at zero.
ret_next has mean 0.013 and std 0.082 with a range from −0.735 to +0.805 — a few meme months dominate any squared error. y_rank runs from −0.495 to +0.5 with std 0.289 in every month: the rank target has thrown away the level and kept the order.
Take a prediction \(\hat y\) and replace it by \(3\tanh(50\hat y)\) — strictly increasing, very different values. What happens to the top-20 / bottom-20 portfolio and to the MSE?
Same 20 names long, same 20 short, same spread — and an MSE 110 times larger. A squared-error model spends its capacity on the level of returns, which is almost all noise (§6.2); a ranking model spends it on the order, which is all the portfolio uses.
The scoreboard of §4.5 was the rank IC — Spearman correlation between prediction and realised return across the month’s stocks. Before running: is it, like the portfolio, blind to the monotone transform?
spearmanr(p1, y)[0] and spearmanr(p2, y)[0] for October 2022, to 4 dp — the same number or different?
-0.3316 -0.3316
Spearman −0.3316 for both; Pearson moves from −0.338 to −0.333. October 2022 was a month in which momentum lost — the high-momentum names fell hardest — and the rank IC records that with one number that no monotone transform can touch. It is the score that matches the trade, and the first column on every slide that follows.
Same walk-forward as §4.5: test months from 2019-01, refit every 12 months on all earlier rows, two scores per month.
71 test months, 6 refits, 10 212 training rows for the first fit. Rank IC is the Spearman correlation between prediction and realised return across the 213 stocks in a month — a score that, like the portfolio, sees only the order. The spread’s Sharpe is \(\bar s / \sigma_s \cdot \sqrt{12}\), before costs.
A linear model with the MSE loss — the loss of every regression in Chapter 4.
Mean IC +0.001 (t = 0.06): the linear MSE model ranks no better than a coin, although its extremes earn a spread of +8.5 % a year (Sharpe 0.34). A one-sd move in momentum is worth +0.26 % a month, in volatility +0.26 %, in price-to-high −0.22 % — and last-month return gets nothing (−0.004): the loss, dominated by the ±80 % months, sees no reversal at all.
Replacing ret_next by its within-month percentile changes…
IC −0.004 (t = −0.17), spread +5.0 % a year, Sharpe 0.24. Not better — with five price-based characteristics on 213 survivors there is little order to learn, whichever target you pick. Note the coefficient signs: last-month return negative (short-term reversal), 12-1 momentum positive — the two best-known cross-sectional anomalies, present but faint.
Ranking is a statement about pairs: within month \(t\), stock \(i\) should score above stock \(j\) when \(r_{i} > r_{j}\). RankNet (Burges et al., 2005) models
\[P(i \succ j) = \sigma\big(f(x_i) - f(x_j)\big), \qquad \sigma(u) = \frac{1}{1 + e^{-u}},\]
and maximises the log-likelihood of the observed orderings. With a linear scorer \(f(x) = w^\top x\) the difference is \(w^\top (x_i - x_j)\): a logistic regression, without intercept, on pair differences, label \(\mathbb 1[r_i > r_j]\).
IC −0.005 (t = −0.21), spread −1.8 % a year, Sharpe −0.09. The weights again say reversal (−0.031) and momentum (+0.026), and the model is about as good as the other two — that is, indistinguishable from zero. The pairwise loss is the right loss for a ranking problem; it cannot manufacture order that five characteristics on 213 stocks do not contain.
Same specification as §4.5 (max_depth=2, learning_rate=0.05, max_iter=200, min_samples_leaf=50); the only change is the target column.
The §4.5 number returns: IC −0.015 (t = −1.04) with the MSE loss. On the rank target the same trees give IC +0.017 (t = +0.98) and the best spread of the five, +9.3 % a year, Sharpe 0.64. The sign of the IC flipped with the target — which tells you the difference between the two is noise, not that ranks are magic.
Every mean IC lies within one standard error (0.02) of zero and the five cumulative spreads fan out from the same 2020 whipsaw. The ranking losses did what they promise — they changed which order the model learns: the MSE trees’ monthly ICs correlate only 0.31 with OLS and 0.20 with the pairwise model, while the three rank-based models move together (0.87–0.92) — but 71 months of a survivorship-biased file cannot tell 0.6 from 0. Report the protocol, not the best line.
The pairwise loss you just fitted is a preference model: the data are statements “\(i\) was better than \(j\) this month” and the score \(f(x)\) is a latent utility that rationalises them. Push the idea further and it becomes a research programme:
Note
Nothing in this slide changes the verdict of the previous one: on five characteristics and 213 survivors, no loss finds a reliable order. The loss decides what you estimate; the data decide whether there is anything to find.
evaluate builds a top-K / bottom-K spread with K=20. Re-score the rank-target boosting predictions with K=10 (more concentrated) or K=40 (more diversified) and compare the Sharpe ratio with the K = 20 value of 0.64. Which direction improves the Sharpe, and is the change larger than the noise you saw across models?
K = 10: spread +20.7 % a year, Sharpe 1.10; K = 40: +8.9 %, Sharpe 0.75. A Sharpe of 1.1 from the ten most extreme names looks like a discovery — hold that thought for §7.6, where you will learn what the best of three values of K is worth. Over 71 months none of the three is separable from the others, or from zero.
Next: §7.2 — stop predicting a number; predict an interval, and check that it keeps its promise.
Prof. Xuhu Wan · HKUST ISOM · Learning Statistics with Python