Chapter 3 · Reshaping Statistics
Section 3.2 · Chapter 3 · Learning Statistics with Python
Reshaping Statistics
Prof. Xuhu Wan
ISOM, HKUST Business School · 2026 Edition
You have one sample. The bootstrap resamples it to reveal how much any statistic would wobble — no formula needed. Then the classical tests: was a year’s mean return zero, was one year different from the next, do two stocks share a distribution — and a two-sample test recomputed every day that becomes a regime detector.
Each section runs standalone. Reload Apple 2005–2022, studentise, refit the \(t\).
You want the sampling distribution of some statistic but cannot draw new samples. Classic statistics needs a formula for the standard error. The bootstrap (Efron, 1979) needs none — and you will see why by reinventing it.
You have a sample \(X_1,\dots,X_n\) and want to know how much \(\hat\theta\) would wobble across new samples — but you cannot draw new samples. Predict the bootstrap’s move.
With no access to the population, how do we mimic drawing a fresh sample?
The ECDF \(\hat F_n\) stands in for the unknown \(F\). Resampling with replacement is sampling from \(\hat F_n\). That single substitution is the whole idea — and Glivenko–Cantelli is why it works.
The recipe in code: build a \(B \times n\) matrix of resample indices, compute the statistic on each row, take the spread. A small skewed sample — 120 lognormal draws — and a statistic with no textbook SE: the median.
Bootstrap SE of the median, 120 lognormal draws (seed 11), \(B = 4000\). Order of magnitude first: 0.02, 0.2, or 2?
0.198
The trick you found: the B x n index matrix replaces every loop and every SE formula. There is no closed-form SE for a median — yet you just measured it: 0.198.
For the mean there is a formula: \(\mathrm{SE} = s/\sqrt n\). If the bootstrap is legitimate, its SE must match. Confirm it on Apple’s 251 returns of 2017.
Bootstrap the mean of r17 (\(B = 4000\), seed 5) into se_boot, and compute the textbook se_formula. They should agree within 10 %.
Both come out at about 0.0007 — so the bootstrap is not magic; it reproduces the answer you already trusted, then extends to medians, ratios and kurtosis, where no formula exists.
The simplest CI: take the 2.5 % and 97.5 % quantiles of \(\hat\theta^*\) directly. Predict whether the interval for the 2017 mean daily return contains zero.
The 95 % percentile CI for the 2017 mean daily return…
\([0.00014,\ 0.00294]\): zero is outside, barely. Clean and correct when the bootstrap distribution is symmetric and unbiased. Predict the next slide: when is that assumption dangerous?
For which statistic is the plain percentile CI least trustworthy?
BCa (bias-corrected and accelerated) fixes two blind spots: a bias correction \(z_0\) (how often \(\hat\theta^* < \hat\theta\) — should be 50 %) and an acceleration \(a\) from the jackknife (handles skew). Same cost as percentile.
Excess kurtosis is the statistic §3.1 warned you about — dominated by a handful of days. Bootstrap it (B = 2000) and compare the two intervals.
Resampling with replacement drops some of the extreme days, so most bootstrap kurtoses fall below the sample value. Is \(z_0 = \Phi^{-1}(\text{share below})\) positive or negative?
True
Kurtosis 4.58 with a bootstrap SE of 1.55 — a third of its own size. 59.4 % of resamples fall below it, so \(z_0 = +0.24\), and BCa pushes the interval from \([1.47, 7.31]\) up to \([2.23, 9.15]\). Rule discovered: for medians, quantiles, ratios and kurtosis the bias and skew are real, and BCa moves the endpoints to where they belong — for free.
A confidence interval says how much a number wobbles; a test asks whether a specific claim survives — the null / statistic / p-value logic of §3.1, now applied to a mean. Three claims: 2017’s mean return was zero, 2017 and 2018 share a mean, MSFT and AAPL share a distribution — then one test recomputed every day.
\(H_0: \mu = 0\). Predict the statistic \(\hat t = \bar x / (s/\sqrt n)\) for 2017.
With \(\bar x \approx 0.00151\), \(s \approx 0.0111\) and \(n = 251\), what is \(\hat t\) to two decimals?
2.16
The cell printed a one-sided \(p\) (Normal approximation) and ttest_1samp’s two-sided \(p\). The business question — “did Apple earn a positive return?” — is directional.
For a positive \(\hat t\), the one-sided \(p\) compared with the two-sided \(p\) is…
Both reject at 5 % — the bootstrap CI excluded zero for the same reason. The interval, the \(t\)-statistic and the \(p\)-value are three views of one fact: \(\bar x\) is 2.16 standard errors from zero.
The \(t\)-test assumed a Normal sampling distribution for \(\bar x\). For a sharp null — “the two groups have identical distributions” — you need no such assumption: build the null distribution by shuffling labels.
Under \(H_0: F_A = F_B\), why can we randomly relabel the pooled data?
No Normality, no equal-variance assumption, no large-\(n\) asymptotics. The price is \(B\) shuffles — trivial on a laptop.
Apple’s mean daily return was +0.151 % in 2017 and −0.028 % in 2018. Label the 502 returns by year, shuffle the labels 5 000 times, and ask how often a gap that large appears by chance.
The two-sided permutation \(p\) for the 2017 − 2018 gap will be roughly…
\(p = 0.18\) from shuffles, 0.18 from Welch’s \(t\) — the classical test was fine here because \(n\) is large. The permutation test’s advantage is that it would still be right for 20 days of fat-tailed returns, where the \(t\) is not. Its Monte-Carlo error shrinks like \(1/\sqrt B\), independent of \(n\).
The permutation test compared two means. ks_2samp compares two whole empirical CDFs and needs no reference distribution at all. The notebook’s MSFT-vs-AAPL check, on the catalog files’ common dates.
Null: the two samples come from the same distribution. On 778 common days \(D = 0.039\), \(p = 0.61\) — no evidence that MSFT and AAPL daily returns are distributed differently. Keep the p-value in mind — the same two-sample idea, run on a rolling window, becomes a regime detector on the next slide.
Compare the last 10 days’ returns with the 10 days ending 20 days earlier: \(\hat t = \dfrac{\bar r_{\text{now}} - \bar r_{\text{lag}}}{\sqrt{(s^2_{\text{now}} + s^2_{\text{lag}})/10}}\). Days with \(|\hat t| > 2\) are flagged.
Twenty of 249 days are flagged; the red dots cluster at the September 2012 peak and the April 2013 trough — the points where the return distribution shifted. A test statistic, recomputed every day, is a trading signal.
For each 20-day window, store the p-value of stats.ks_2samp(w_now, w_lag) in ks_p (the test compares the current 20 returns with the 20 returns lagged by 20 days). Then count windows with p < 0.2.
ks_2samp cannot tell MSFT from AAPL (\(D = 0.039\), \(p = 0.61\)). A rolling two-sample \(t\) turns “same distribution?” into a regime signal: 20 flagged days around the 2012 peak and 2013 trough.Next: §3.3 — tests you design: how many users, when to stop, and when not to test at all.
Prof. Xuhu Wan · HKUST ISOM · Learning Statistics with Python