3.1 — Population vs Sample and Distributions in Practice

Chapter 3 · Reshaping Statistics

Prof. Xuhu Wan

Section 3.1 · Chapter 3 · Learning Statistics with Python

Population vs Sample and Distributions in Practice

Reshaping Statistics

Prof. Xuhu Wan

ISOM, HKUST Business School · 2026 Edition

Population vs Sample and Distributions in Practice

A population parameter is a fact; a sample statistic is an estimate with a distribution of its own. You will build the empirical CDF from five numbers, read Apple’s fat tails off it, estimate a density two ways (bandwidth vs a formula), learn to see a Student-t in a Q-Q plot, and then let Shapiro–Wilk and Kolmogorov–Smirnov put numbers on what the eye saw.

Population vs sample: ddof=0 or 1?

The notebook’s 63 numbers are the whole population. We draw 10 without replacement (seed 1).

Why does pandas compute std() with ddof=1 by default?

  • Because \(n - 1\) is always closer to \(N\)
  • Because dividing by \(n\) underestimates the population variance from a sample
  • Because populations are never observed
  • It doesn’t — the default is ddof=0

Univariate EDA: Apple 2005–2022

Log return \(r_t = \ln S_t - \ln S_{t-1}\). Before running: is the excess kurtosis of daily returns positive or negative?

The excess kurtosis of Apple’s daily returns will be…

  • Large and positive — fat tails
  • About 0 — close to normal
  • Negative — thin tails
  • Undefined for returns

Excess kurtosis 5.99 and skew −0.28. Half the days sit between the two red lines — a band about 2 % wide — yet the axis runs from roughly −20 % to +14 %. That gap between the middle and the extremes is what the next slides quantify.

From theoretical to empirical

Textbooks hand you a density. Real work hands you a sample. The bridge between them is one line of arithmetic — you will build it from five numbers and then read Apple’s tails off it.

Build an ECDF from five numbers

You have the sample [3, 1, 4, 1, 5]. Define \(\hat F(x)\) = (fraction of the sample \(\leq x\)). Before any code, predict two values:

What fraction of [3, 1, 4, 1, 5] is \(\leq 3\)?

sum(v <= 3 for v in [3,1,4,1,5]) / 5

0.6

And \(\leq 1\)?

sum(v <= 1 for v in [3,1,4,1,5]) / 5

0.4

You just defined the ECDF

The object you built has a name. Given an i.i.d. sample \(X_1,\dots,X_n\), the empirical CDF is \[\hat F_n(x) = \frac{1}{n}\sum_{i=1}^{n}\mathbf 1\{X_i \le x\}.\] A step function. No assumptions about shape. It is the non-parametric maximum-likelihood estimate of the true CDF — and you computed it by hand.

Read the ECDF off Apple’s returns

A sorted sample’s \(i\)-th value sits at height \(i/n\). Studentise the 4 309 returns (\(z = (r - \bar r)/s\)) and sort them.

The ECDF height at the 2 155th of 4 309 sorted points — what is \(2155/4309\) to three decimals?

print(round(2155/4309, 3))

0.5

The median studentised return is −0.008, essentially zero. But 76.9 % of days lie within one standard deviation (a Normal says 68.3 %) and only 98.5 % within three (a Normal says 99.7 %): a taller peak and fatter tails. That is excess kurtosis 5.99, read off a staircase.

Why doesn’t a histogram do the job?

The notebook’s mixture sample (seed 7) is two Normals stuck together, one at −10 and one at +5. Predict what its ECDF looks like before you draw it.

The data is bimodal (two clusters). What will the ECDF curve look like?

  • A single smooth S-curve — bimodality is invisible in a CDF
  • A straight diagonal line
  • Two steep stretches separated by a flatter stretch
  • A bell shape

Run it and check your call

The curve climbs steeply through −10, then flattens between about −5 and 0 (the first quartile is −10.0, the median −5.2, the third quartile 5.0), then climbs again around +5. The rule you discovered: the ECDF has no tuning knob, so it cannot lie about shape. The flat stretch is the bimodality.

How fast does it converge?

The ECDF is built from \(n\) random points, so it is itself random. Will it land on the truth as \(n\) grows? Predict the worst-case gap for Apple’s \(n = 4\,309\).

The DKW bound on \(\Pr(\sup_x|\hat F_n - F| > 0.02)\) is \(2e^{-2n(0.02)^2}\). Evaluate it at \(n = 4309\), rounded to 3 decimals.

import math
print(round(2*math.exp(-2*4309*0.02**2), 3))

0.064

Glivenko–Cantelli (1933): \(\sup_x|\hat F_n(x) - F(x)| \xrightarrow{a.s.} 0\) — the ECDF converges uniformly to the truth. The DKW inequality is the finite-\(n\) version: with 4 309 days, the whole Apple staircase is within 0.02 of the true CDF with probability at least 93.6 %. This theorem is what the bootstrap (§3.2), the KS test and the KDE all lean on.

Kernel density estimation

The ECDF is a staircase. To get a smooth density, smear each point into a small bump and add them up. The width of the smear, \(h\), is the only choice that matters — and you will discover how much it matters.

Estimating a density without a formula: KDE

A kernel density estimate puts a small bump on every observation and adds them up. The bandwidth \(h\) is the only knob. Predict its effect on the notebook’s two-component mixture.

For a two-mode sample, what does a very small bandwidth do?

  • Produces the smoothest curve
  • Hides both modes
  • Produces a spiky curve that chases sampling noise
  • Has no effect: KDE is bandwidth-free

Estimating a density with a formula: fit a Student-t

Studentise the returns, then let scipy.stats.t.fit choose the degrees of freedom. Predict the meaning of the number before you see it.

If the fitted degrees of freedom come out near 3, what does that say?

  • The returns are close to normal
  • Tails so heavy that the fourth moment is not finite
  • The sample has only 3 observations
  • The mean is 3 standard deviations from zero

Your turn: degrees of freedom for Tesla

The notebook fits the same \(t\) to TSLA, CVX and BAC (df between 3 and 6 for every large stock). Do it for Tesla in 2020.

Studentised 2020 Tesla returns are in z_tsla. Set nu_tsla to the degrees of freedom returned by stats.t.fit (its first element).

Predict the Q-Q plot

A Q-Q plot puts sample quantiles against a reference distribution’s quantiles. A perfect match traces the diagonal. Apple’s returns have df ≈ 3.3 tails; predict what that does to the ends against a Normal reference.

The sample is heavier-tailed than Normal. The Q-Q plot will…

  • Lie perfectly on the diagonal
  • Curl away from the diagonal at both extremes (S-shape)
  • Be a horizontal line
  • Curve only on the right end

Testing for normality: look first (Q-Q)

A Q-Q plot compares sample quantiles with a reference distribution’s quantiles. Straight line = same shape.

Against the normal the tails peel away from the line on both sides — the classic S. Against the fitted \(t\) the points hug the line until the very last few, where the right tail is shorter than a \(t_{3.3}\) predicts.

Testing for normality: Shapiro–Wilk and Kolmogorov–Smirnov

The Q-Q plot showed where the departure is; a test puts a number on it. The notebook runs kstest(returns, stats.norm.cdf) and gets \(D \approx 0.47\). Predict why it is that large.

Why is the KS statistic 0.47 when raw returns are compared with stats.norm.cdf?

  • Because the returns are extremely non-normal in shape
  • Because the reference is N(0, 1) and the returns have sd ≈ 0.02 — a scale mismatch, not a shape test
  • Because KS cannot handle 4 000 observations
  • Because the returns contain NaN

Reading the tests

Null hypothesis: the population is normal. A tiny p-value rejects it: Shapiro–Wilk \(p \approx 10^{-38}\); KS on the studentised returns \(D = 0.075\), \(p \approx 10^{-21}\). Against the fitted \(t_{3.3}\) the KS test does not reject (\(D = 0.015\), \(p = 0.27\)) — that is what “the t fits” means in numbers. Shapiro–Wilk is the more powerful normality test; KS is the more general one — any CDF, or another sample (§3.2 uses the two-sample form).

A cheap bimodality detector

Before fitting anything, the bimodality coefficient \(b = (g_1^2 + 1)/(g_2 + 3)\) flags trouble when \(b > 5/9 \approx 0.555\). Predict the verdict on the mixture, then on Apple.

For the seed-7 mixture, \(g_1 = 0.381\) and \(g_2 = -1.214\). Is \(b\) above 0.555?

print((0.381**2 + 1) / (-1.214 + 3) > 0.555)

True

Mixture: \(b = 0.641\), flagged — negative kurtosis (−1.21) is the fingerprint of two separated humps. Apple: \(b = 0.120\), not flagged — one peak, fat tails. The detector tells you which problem you have before you choose a density.

What you discovered

  • A sample statistic estimates a population parameter; ddof=1 corrects the downward bias of the sample variance.
  • The ECDF is “fraction \(\le x\)” — built from five numbers, it converges uniformly (Glivenko–Cantelli; DKW: within 0.02 with probability ≥ 93.6 % at \(n = 4309\)). It showed Apple’s 76.9 % within ±1 sd against the Normal’s 68.3 %.
  • KDE trades bias for variance through the bandwidth; Silverman’s \(h \approx 0.2\) sd is a start, not an answer. A parametric fit trades flexibility for two numbers you can quote: Student-t, df ≈ 3.3.
  • The Q-Q plot’s S-curve is the eye’s kurtosis test. The formal tests agree: Shapiro–Wilk rejects the Normal at \(p \approx 10^{-38}\); KS against norm on raw returns tests scale, not shape — studentise first (\(D = 0.075\)); the fitted \(t_{3.3}\) is not rejected (\(D = 0.015\), \(p = 0.27\)).
  • The bimodality coefficient (0.64 for the mixture, 0.12 for Apple) tells a two-hump problem from a fat-tail problem before you choose a density.

Next: §3.2 — how sure are you of any of those numbers? Resample, and test.