Chapter 5 · Learning Statistics with Python
Priors, likelihoods and posteriors for risk and reward, credible intervals, Bayesian regression with predictive checks, robust regression with fat tails, dynamic updating.
Prof. Xuhu Wan
ISOM, HKUST Business School · 2026 Edition
Why this matters
Bayesian models can produce forecasts, share information across stocks and support investment decisions. Posterior probabilities require a defined quantity, explicit model assumptions and computational checks. Walk-forward evaluates the resulting research process.
Follow the numbered parts in order within each section. Each part states a question and a result. Read the concept, predict the example, run it, and explain the output before attempting the check.
The 3.5-hour class selects examples, group discussion and notebook work. The complete Slides and Book support self-study.
Worked examples run automatically when the page opens or refreshes. Each cell includes required earlier imports, data and definitions under Previously introduced setup. Run Code includes that setup; Copy complete code copies the whole cell. Practice and repair cells run when you choose. Make input changes in the current cell before the operation you are testing.
| Section | Concept | Tool |
|---|---|---|
| 5.1 | Prior × likelihood → posterior; Sharpe ratio of a Student-t return model; credible intervals | grid approximation, scipy.stats.t.logpdf, Metropolis in numpy |
| 5.2 | Regression as a normal distribution; prior predictive and posterior predictive checks | conjugate normal-linear posterior, rng.multivariate_normal |
| 5.3 | Fat tails break OLS; Student-t likelihood repairs it | scipy.optimize.minimize, robust beta of GOOG on the S&P 500 |
| 5.4 | Sequential learning and adaptive risk with explicit clocks | normal updates, local-level filter |
| 5.5 | Signals, hierarchical regression and partial pooling | normal posterior, prior sensitivity, predictive uncertainty |
| 5.6 | Model uncertainty, portfolio decisions and forward evidence | integrated evidence, predictive mixture, net accounts |
| Part | Question |
|---|---|
| 5.1.1 Prior, likelihood and posterior | How do assumptions and data determine a belief? |
| 5.1.2 Posterior calculation on a grid | How is a posterior normalised numerically? |
| 5.1.3 Sampling and uncertainty in a risk statistic | Can posterior draws support a defensible risk summary? |
Question: How do assumptions and data determine a belief?
Result: Write a return model with an explicit likelihood and prior.
Begin with the definitions and assumptions; then trace the worked example.
\[\underbrace{p(\theta \mid X)}_{\text{posterior}} \;\propto\; \underbrace{p(X \mid \theta)}_{\text{likelihood}}\;\underbrace{p(\theta)}_{\text{prior}}\]
Classical statistics fixes \(\theta\) and asks how the data would vary. Bayes fixes the data and asks how \(\theta\) varies.
After seeing 1 258 daily returns, which object lets you say “P(reward-to-t-scale > 1) = 0.99”?
GOOG daily returns, October 2015 to October 2020. For this exercise the risk-free return is zero and square-root annualisation assumes no serial dependence. The annualised Sharpe ratio is \(\sqrt{252}\,\bar r / s\).
Mean daily return ≈ 0.00080, daily sd ≈ 0.01644. What does the classical Sharpe ratio print (approximately)?
0.771
Sharpe 0.771, excess kurtosis 6.72. A normal has kurtosis 0 — these returns have far fatter tails. The model must say so.
\[r_t \sim \text{StudentT}(\nu,\ \mu,\ \sigma)\]
Three parameters: location \(\mu\), scale \(\sigma\), and degrees of freedom \(\nu\) that set the tail weight.
Which choice of \(\nu\) produces the fattest tails?
On a log scale the normal’s tail is a parabola falling off a cliff; the t(3) tail is almost a straight line. A four-scale-unit observation: 6.3e-05 under the normal, 0.028 under t(3) — about 440 times more likely.
The notebook’s choices, one per parameter:
| Parameter | Prior | Why |
|---|---|---|
| \(\mu\) | \(N(\bar r,\ s)\) | centred on the sample mean, very wide (sd = one daily sd) |
| \(\sigma\) | \(\text{Uniform}(s/1000,\ 1000\,s)\) | positive, essentially flat |
| \(\nu\) | \(2 + \text{Exponential}(\lambda = 1/29)\) | mean 31, but mass near 2 keeps fat tails possible; \(\nu > 2\) guarantees a finite variance |
What does the prior \(\nu = 2 + \text{Exp}(1/29)\) force?
Check it: expon.cdf(8, scale=29) = 0.241 — the prior gives only 24 % to \(\nu < 10\). It allows fat tails without insisting on them.
In Colab
PyMC is not available in the browser. This is the notebook’s model; the next slides reproduce its posterior with numpy and scipy.
with pm.Model() as sr_model:
mean = pm.Normal("mean", mu=rmean, sigma=rstd)
std = pm.Uniform("std", lower=rstd/1000, upper=rstd*1000)
df = pm.Exponential("df", 1/29, initval=5) + 2.0
returns = pm.StudentT("returns", nu=df, mu=mean, sigma=std,
observed=google["return"])
pm.Deterministic("sharpe", np.sqrt(252) * mean / std)
trace = pm.sample(tune=1500, draws=1000, chains=2, random_seed=8888)Four lines of priors, one likelihood, one derived quantity. Everything else is the sampler — which we now build by hand.
Question: How is a posterior normalised numerically?
Result: Compute one- and two-parameter grids and explain their scaling limit.
Builds on Prior, likelihood and posterior.
With one or two parameters you do not need a sampler at all: evaluate prior × likelihood on a grid and normalise. This is Bayes’ rule made literal.
Same five years, S&P 500. Treat \(\sigma\) as known (the sample sd). Prior \(\mu \sim N(0,\ 0.001^2)\). For a normal likelihood the posterior has a closed form — predict what the grid should reproduce.
Posterior precision = prior precision + \(n/\sigma^2\). With \(n = 1258\), \(\sigma = 0.01186\), prior sd 0.001, what is the posterior sd (approximately)?
0.000317
Grid 0.000497 0.000317, closed form 0.000497 0.000317 — identical to six decimals.
log_post - log_post.max() before exp is not cosmetic: 1 258 log-densities sum to about 3 800, and exp of that overflows.0.000497 sits between the prior mean 0 and the sample mean 0.00055: a precision-weighted average. The data’s precision (\(n/\sigma^2 \approx 8.9\) million) dwarfs the prior’s (1 million), so the data win — mostly.Two parameters? A 20 × 20 grid is still only 400 points. Let’s do \(\mu\) and \(\sigma\) for GOOG under a Student-t likelihood with \(\nu = 4\).
Posterior mean σ = 0.0108 — well below the sample sd 0.0164. The t-scale is not the standard deviation; fat tails inflate the sd. And reward-to-t-scale 2.035 with P(reward-to-t-scale > 1) = 0.972. Hold that thought.
A 400-point grid per parameter and a 3-parameter model (\(\mu, \sigma, \nu\)). How many likelihood evaluations?
The curse of dimensionality is why we need Markov chain Monte Carlo: instead of visiting every point, wander through parameter space so that time spent in a region is proportional to its posterior probability.
Question: Can posterior draws support a defensible risk summary?
Result: Inspect sampling diagnostics and distinguish t-scale from standard-deviation Sharpe.
Builds on Posterior calculation on a grid.
Propose a random step, accept it with probability \(\min(1, p^\star / p_t)\), otherwise stay. That is the whole algorithm — and it never needs the normalising constant.
At \(\theta_t\) the posterior density is \(p_t\); you propose \(\theta^\star\) with density \(p^\star\).
\(p^\star = 0.3\,p_t\). What does Metropolis do?
In code: if log(u) < log_post(prop) - log_post(current). Work in logs; the difference of two logs is a ratio of densities, so the unknown normaliser \(p(X)\) cancels.
Sample \(\theta = (\mu, \log\sigma, \log(\nu - 2))\) so the chain can never leave the allowed region. The extra ls and le terms are the Jacobians of those transforms.
3438.4 at the sample sd, 3443.2 at half of it. The likelihood already prefers a scale smaller than the sample sd — the tails are being handled by \(\nu\), not by \(\sigma\).
Acceptance 0.47 describes this proposal setting; it does not establish convergence. \(\nu \approx 2.7\): GOOG’s tails are heavier than a t(3) (the notebook’s PyMC run found 2.79). \(\sigma = 0.0097\), six-tenths of the sample sd.
What does a healthy trace look like?
Posterior mean 2.38, 95 % credible interval [1.30, 3.59], P(Sharpe > 1) = 0.99. The notebook’s PyMC run gave 2.35 and [1.31, 3.42] — a similar numerical estimate from a short sampler, without proving equal Monte Carlo precision. The classical 0.77 lies outside this interval for a differently defined quantity. This is a reward-to-t-scale ratio. The next comparison converts it to a standard-deviation Sharpe.
The posterior mean reward-to-t-scale is 2.38 against a conventional Sharpe estimate of 0.77. The main reason?
Be honest about what σ means
The t-distribution’s standard deviation is \(\sigma\sqrt{\nu/(\nu-2)}\), which for \(\nu \approx 2.7\) is almost twice \(\sigma\). Re-computing Sharpe on that basis gives a posterior mean of about 1.18. Reward-to-scale and conventional Sharpe are distinct estimands. Changing the denominator does not prove that classical estimates are biased downward.
For Student-t returns with \(\nu>2\), the return sd is \(\sigma\sqrt{\nu/(\nu-2)}\). The ratio \(\sqrt{252}\mu/\sigma\) is a reward-to-scale ratio; it is not the conventional Sharpe ratio.
The mean is 1.178, credible interval [0.575, 1.852], and probability above one about 0.70. The difference from the sample ratio does not prove that classical statistics systematically understates performance. Risk-free return is set to zero; annualisation assumes independent, stable daily returns.
A visually stable trace and an acceptance rate of 0.47 do not prove that all relevant posterior regions were explored. Run dispersed chains, inspect rank-normalised R-hat, bulk/tail effective sample size, Monte Carlo standard errors and sampler warnings. Check the derived investment quantity as well as individual parameters.
The single-chain browser sampler teaches Metropolis mechanics. It is not a validated computational pipeline for allocating capital. Normal conjugate calculations in §§5.5–5.6 avoid Monte Carlo error under their stated assumptions.
Under i.i.d. normal returns, the usual approximate standard error of an annualised Sharpe is \(\sqrt{252\,(1 + \tfrac12 \widehat{SR}_d^2)/n}\).
For a standard-deviation Sharpe, which interval supports a 95% posterior probability statement conditional on its model and prior?
[-0.11, 1.65][0.58, 1.85]The conventional interval includes zero under its assumptions; this concerns the population mean, not whether the observed investment made money. The t-scale posterior says P(reward-to-t-scale > 0.5) = 0.999, but uses a different denominator. Compare the sd-basis posterior for the same estimand.
The notebook’s practice. The starter runs the sampler on AAPL (2 515 returns). Switch r_p to NVDA from nv (2023–2024, 500 returns) and compare posterior mean and classical Sharpe for each.
Next: §5.2 — regression as a normal distribution, and how to check a model before fitting it.
| Part | Question |
|---|---|
| 5.2.1 Regression as a generative model | What observations could the assumed regression generate? |
| 5.2.2 Prior predictive checks | Do the priors imply plausible observations? |
| 5.2.3 Posterior coefficients | How do prior and data precision combine? |
| 5.2.4 Posterior predictive checks | Does the fitted model reproduce relevant data features? |
Question: What observations could the assumed regression generate?
Result: Separate coefficients, residual noise and generated observations.
Begin with the definitions and assumptions; then trace the worked example.
\[y_i \sim N(\mu_i,\ \sigma^2), \qquad \mu_i = \beta_0 + \beta_1 x_i\]
The notebook’s artificial data: \(\beta_0 = 1\), \(\beta_1 = 2\), \(\sigma = 0.5\).
In the Bayesian regression, which quantities get a prior?
y mean 2.041, sd 0.759. The sd of \(y\) (0.76) exceeds \(\sigma\) (0.5) because part of the spread is the line itself. Keep that distinction: spread of y ≠ noise.
In Colab
The notebook fits this with NUTS. We will get the same posterior in closed form on the next slides.
with pm.Model() as linear_regression1:
std = pm.HalfCauchy("sigma", beta=10, initval=1)
beta0 = pm.Normal("intercept", 0, sigma=20)
beta1 = pm.Normal("slope", 0, sigma=20)
likelihood = pm.Normal("y", mu=beta0 + beta1 * data["x"], sigma=std,
observed=data["y"])
trace1 = pm.sample(tune=6000, draws=2500, chains=2)The posterior draws come back as a table: one row per draw, one column per parameter. Every plot in this section is built from that table.
Question: Do the priors imply plausible observations?
Result: Compare simulated lines with the problem’s measurement scale.
Builds on Regression as a generative model.
Before fitting, draw parameters from the prior and plot the regression lines they imply. If the lines are absurd, the prior is absurd — and you found out for free.
\(x \in [-10, 10]\), \(y = 0.5 + 3x + N(0, 6^2)\), 200 points.
\(x\) has sd 5.8 and the slope is 3, noise sd 6. Roughly what is the sd of \(y\)? (\(\sqrt{3^2 \cdot 5.8^2 + 6^2}\))
18.4
Draw 100 lines with \(\beta_0, \beta_1 \sim N(0, 10)\). At \(x = 10\), what range covers 95 % of them?
Prior I: [-173.8, 209.7]. Prior II: [-20.8, 25.2]. The data at \(x = 10\) sit near 30. Prior II covers plausible slopes without covering nonsense — but it puts the true slope 3 at 2.5 prior sd. With 200 points the data will drag it there anyway; with 10 points they might not.
Question: How do prior and data precision combine?
Result: A coefficient posterior with its covariance retained.
Builds on Prior predictive checks.
Normal likelihood, normal prior on \(\beta\): the posterior is normal and you can write it down. The formula is the same precision-weighting you saw on the grid, now in matrix form.
\[\Sigma_n = \left(\Sigma_0^{-1} + \tfrac{1}{\sigma^2} X^\top X\right)^{-1}, \qquad \mu_n = \Sigma_n\left(\Sigma_0^{-1}\mu_0 + \tfrac{1}{\sigma^2} X^\top y\right)\]
OLS on (xt, yt) gives slope 3.011. Prior II is \(N(0, 1.2^2)\) on the slope, and the data precision on the slope is about \(\sum x^2/\sigma^2 \approx 6700/32.5 \approx 206\). Will the posterior mean slope be nearer 3.011 or 0?
≈ 3.00 — data precision 206 vs prior precision 0.69
OLS [0.385, 3.011], posterior mean [0.331, 3.000], posterior sd [0.374, 0.069]. The intercept is shrunk a little toward 0 (its prior sd 1 competes with a data sd 0.4); the slope barely moves. With \(n = 200\) the prior is a rounding error.
The 50 red lines are 50 rows of post_df. Slope credible interval [2.861, 3.139] — the same information a classical CI gives, but readable as a probability.
Question: Does the fitted model reproduce relevant data features?
Result: A predictive check including both parameter and observation uncertainty.
Builds on Posterior coefficients.
Now go one step further: for each posterior draw, simulate a whole replicated data set. If the replicas do not look like the data, the model is missing something.
For posterior draw \(s\): \(\mu_i^{(s)} = b_0^{(s)} + b_1^{(s)}x_i\), then \(y_i^{\text{rep},(s)} \sim N(\mu_i^{(s)}, \sigma)\) for every \(i\).
yrep has shape (1000, 200). What is yrep[7]?
Thirty cyan replicas and the black data share one shape. If the data had a bump the replicas never produce, the normal likelihood would be wrong — this is how the §5.3 outliers will announce themselves.
A Bayesian p-value: the fraction of replicas whose statistic exceeds the observed statistic. Near 0.5 is good; near 0 or 1 means the model cannot produce what you saw.
p_mean is done. Fix p_max so it compares the maximum of each replica row with the observed maximum yt.max(). Expect about 0.13 — the normal model slightly under-produces the largest value.
p_max = 0.13 is the first hint that normal tails are thin.Next: §5.3 — put three outliers in and watch the normal likelihood surrender.
| Part | Question |
|---|---|
| 5.3.1 Outlier sensitivity and likelihood choice | Why can a few observations dominate a Gaussian fit? |
| 5.3.2 Robust posterior uncertainty | How uncertain is the robust slope? |
| 5.3.3 Market beta and rolling robustness | When do ordinary and robust market associations differ? |
Question: Why can a few observations dominate a Gaussian fit?
Result: Compare squared-error sensitivity with a Student-t likelihood.
Begin with the definitions and assumptions; then trace the worked example.
\(y = 0.5 + 5x + N(0, 0.5^2)\) on 100 points, plus three points at \((0.1, 8), (0.15, 6), (0.2, 9)\) — far above the line on the left.
How will OLS respond to three high points at small \(x\)?
OLS [1.033, 4.403] — intercept doubled, slope off by 0.6. Under a normal likelihood a residual of 7 costs \(7^2/2 = 24.5\) log-units; the fit bends to reduce that at everyone else’s expense.
In Colab
with pm.Model() as model:
beta0 = pm.Normal("b0", 0.0, 1.0); beta1 = pm.Normal("b1", 0, 1.2)
std = pm.Exponential("sd", 1.0)
output = pm.Normal("y", mu=beta0 + beta1 * data2["x"], sigma=std, observed=data2["y"])
# robust version: replace the last line by
likelihood = pm.StudentT("y", mu=mu, sigma=std, nu=3, observed=data2["y"].values)Same priors, one word changed: Normal → StudentT. Let’s see what that word does.
No sampler needed to see the effect: minimise the negative t log-likelihood with scipy.optimize.minimize, \(\nu = 3\).
Robust [0.492, 5.167], scale 0.457 — the truth is 0.5, 5, 0.5. The green dashed line goes through the cloud; the red one is tugged up-left by three points.
The t log-density’s derivative gives each point an implicit weight \(w_i = \dfrac{\nu + 1}{\nu + r_i^2}\), where \(r_i\) is the standardised residual.
The outlier at \((0.2, 9)\) has standardised residual ≈ 16.5 under the robust fit. With \(\nu = 3\), what is its weight (approximately)?
0.015
Outliers get weights 0.017, 0.036, 0.015; a typical point gets 1.13. The t-likelihood is a self-weighting least squares — it down-weights exactly the points that do not fit. Nobody chose which points to drop.
Question: How uncertain is the robust slope?
Result: Posterior slope draws with the likelihood and prior stated.
Builds on Outlier sensitivity and likelihood choice.
Same sampler as §5.1, new log-posterior: Prior II from §5.2, \(\sigma \sim \text{Exponential}(1)\), t(3) likelihood.
Slope posterior mean 5.06, credible interval [4.64, 5.41]. OLS’s 4.40 lies outside it. The model is telling you the outliers are not from the same process.
Question: When do ordinary and robust market associations differ?
Result: Compare fitted betas and a rolling extension on matched returns.
Builds on Robust posterior uncertainty.
Daily returns 2015-10 to 2020-10, 1 258 days including the March 2020 crash. Two likelihoods, one question: what is \(\beta\)?
OLS gives \(\beta \approx 1.06\). What will a Student-t likelihood give?
OLS β 1.064, robust β 1.094, \(\nu = 3.35\) — close to the 2.7 the return model found in §5.1. The point estimates differ by 0.03; the difference in inference is larger: a normal model with kurtosis-11 residuals reports standard errors that are fiction.
Twelve days with \(|r_{SP}| > 5\%\), nine of them in March 2020. Those twelve points — 1 % of the sample — carry 37 % of \(\sum x^2\), the leverage that drives OLS. The t model weights each by its surprise, not by its size.
250-day windows, stepped every 25 days. The OLS beta is done; replace the placeholder so b_rob is the t(3) slope from minimize (hint: mimic nll_t with xx, yy). Then compare the two paths.
Normal → StudentT) is the whole robust model; minimize or Metropolis fits it in under a second.Next: §5.4 — stop refitting from scratch; let yesterday’s posterior become today’s prior.
| Part | Question |
|---|---|
| 5.4.1 Updating a fixed unknown quantity | How does genuinely new evidence update a belief? |
| 5.4.2 Allowing a latent quantity to change | What if the underlying quantity changes over time? |
| 5.4.3 Chronological adaptive-risk evaluation | Was the risk estimate available before the decision? |
Question: How does genuinely new evidence update a belief?
Result: A sequential posterior that counts each observation once.
Begin with the definitions and assumptions; then trace the worked example.
Normal mean, known \(\sigma\). One observation \(r_t\) updates \((m, v)\) — the posterior mean and variance of \(\mu\):
\[\frac{1}{v_t} = \frac{1}{v_{t-1}} + \frac{1}{\sigma^2}, \qquad m_t = v_t\left(\frac{m_{t-1}}{v_{t-1}} + \frac{r_t}{\sigma^2}\right)\]
After 253 daily updates with \(\sigma = 0.02\) and prior sd 0.002, what is the posterior sd of \(\mu\)?
253 days, mean 0.00277, sd 0.02939. We fix \(\sigma = 0.02\) (known variance is the price of a closed form) and start from a prior \(\mu \sim N(0, 0.002^2)\).
Day 1 sd 0.00199, day 253 mean 0.00199, sd 0.00106 — and the batch formula gives exactly 0.00199, 0.00106. Sequential = batch. The posterior is a sufficient summary of the past.
During the March 2020 crash the posterior sd of μ…
Question: What if the underlying quantity changes over time?
Result: Explain process noise, observation noise and a local-level update.
Builds on Updating a fixed unknown quantity.
The fix is one line: before each update, admit that \(\mu\) may have moved overnight.
\[\mu_t = \mu_{t-1} + \eta_t, \quad \eta_t \sim N(0, q) \qquad\Longrightarrow\qquad v_{t-1} \leftarrow v_{t-1} + q\]
With \(q > 0\) added every day, what happens to the posterior variance \(v_t\) in the long run?
Observation equation \(r_t = \mu_t + \varepsilon_t\), state equation \(\mu_t = \mu_{t-1} + \eta_t\): this is the local-level model, and the recursion you just wrote is the Kalman filter for it.
UnobservedComponents estimates both variances by maximum likelihood. First on the returns themselves.
Level variance 8.8e-13 — effectively zero. This fitted model estimates near-zero mean process variance; it does not prove that true expected return is constant. That is a finding, not a failure. Risk is a different story.
\(\sqrt{\pi/2}\,|r_t|\) is an unbiased daily-volatility proxy under normality. Give it a wandering level.
Filtered vol 1.2 % on 2 Jan 2020, 8.8 % on 16 Mar, back to 1.7 % by year-end. Signal-to-noise q = 0.025: the level moves, but each day’s |r| is mostly noise.
In steady state the filter’s update is \(m_t = m_{t-1} + K\,(y_t - m_{t-1})\) with gain \(K = \tfrac{-q + \sqrt{q^2 + 4q}}{2}\).
With \(q = 0.0251\), what is the steady-state gain \(K\) (approximately)?
0.146
Your five-line recursion reproduces filtered_state to 5e-07. \(K = 0.146\) is an EWMA with a 12.7-day span — RiskMetrics’ \(\lambda = 0.94\) (span 32) is the same filter with a smaller \(q\). Chapter 6 adds trends, seasons and regressors to this state.
Question: Was the risk estimate available before the decision?
Result: A training-boundary check and a corrected block-update experiment.
Builds on Allowing a latent quantity to change.
The earlier 2019–2020 local-level fit is retrospective: even its filtered states use variances estimated from that whole interval. For a forward exercise, estimate parameters on 2019 and carry the filter into 2020 without refitting.
There are 252 training and 253 forward days. The levels are 1.15%, 5.91% and 1.78% at the displayed dates. A close-based state is available only after that close; a trading rule uses an appropriate execution lag. Filtered states and smoothed states have different information sets.
Using a previous posterior as the next prior is valid when the next likelihood contributes new data, with the conditional dependence model specified.
Do not multiply yesterday’s posterior by a likelihood containing yesterday’s observations again. A rolling calendar alone does not prevent double-counting or future-labelled training rows.
The notebook’s practice: 20 days set the prior, then update every 10 days with the posterior of the previous block as the new prior. The starter resets the prior every block. Move the reset out of the loop so beliefs carry forward. sd_final should then equal \(\sigma/\sqrt{253}\) = 0.00126: a prior built from 20 days carries exactly 20 observations of information, so block updates end where a one-shot update would.
0.00199 ± 0.00106 exactly — the posterior is a sufficient summary of the past.UnobservedComponents to better than \(10^{-6}\); the gain \(K = 0.146\) is an EWMA with a 12.7-day span.Next: §5.5 — share evidence across stock signals, then evaluate the resulting investment decisions.
| Part | Question |
|---|---|
| 5.5.1 Targets and shrinkage across stocks | How much should a noisy stock estimate move towards its group? |
| 5.5.2 Feature effects and posterior covariance | How do shared features and stock effects combine? |
| 5.5.3 Predictive uncertainty and prior sensitivity | Is a positive expected return the same as a positive realised return? |
Question: How much should a noisy stock estimate move towards its group?
Result: Compare no pooling, complete pooling and partial pooling.
Begin with the definitions and assumptions; then trace the worked example.
The question is: which candidates have favourable expected next-month returns given their available characteristics?
Use three features: momentum, realised volatility and sector-relative momentum. Preserve stock/date columns. Keep future returns outside the feature matrix and retain pending candidates.
The nine-stock classroom panel has survivor membership, static sectors and assumed monthly availability. Its previous predictions are ignored. Its realised results have already been explored in earlier chapters.
No pooling estimates each stock separately. Complete pooling assumes a common effect. Partial pooling lets stocks differ while sharing a distribution:
\[a_i\sim N(0,\tau^2),\qquad \bar e_i\mid a_i\sim N(a_i,\sigma^2/n_i).\]
\[E[a_i\mid\bar e_i]=w_i\bar e_i,\qquad w_i=\frac{\tau^2}{\tau^2+\sigma^2/n_i}.\]
Here \(\bar e_i\) is a residual mean after a declared baseline. A large noisy residual mean need not imply a large persistent stock effect.
Stock A has only three monthly residuals averaging +8%. Stock C has sixty averaging −1%. Under the same prior and residual variance, which receives less weight on its own sample mean?
Stock A shrinks from 8% to 0.86%, with own-data weight 0.1071. Stock B becomes 1.14%; C becomes −0.71%. The largest raw sample mean is no longer the largest posterior mean.
The fixture is simulated to vary sample sizes transparently. It demonstrates a mechanism, rather than proving that the pooled ranking will earn money.
A simulated stock has three monthly residuals averaging 8%. Residual sd is fixed at 10%; the stock effect has prior mean zero.
Try: compare τ = 0.5%, 2% and 10%. Explain both the posterior mean and interval. Prior sensitivity is part of the result.
Question: How do shared features and stock effects combine?
Result: Fit a scaled feature model while retaining coefficient covariance.
Builds on Targets and shrinkage across stocks.
\[r_{i,t+1}=b_0+z_{it}^{\top}\beta+s_i^{\top}\gamma+a_i+\varepsilon_{it},\qquad \varepsilon_{it}\sim N(0,\sigma^2).\]
\(z\) contains three train-standardised features; \(s\) contains two sector indicators with Energy as reference. Nine stock effects share \(a_i\sim N(0,\tau^2)\).
Fixed exercise scales: intercept/features 5%, sector effects 3%, stock effects τ = 2%, residual sd σ = 10%. These are assumptions for sensitivity analysis, rather than universally appropriate market values.
The design has 15 columns. The function receives the training mean and standard deviation; it never estimates a new scale from prediction rows.
Ticker indicators represent this declared teaching universe. They do not establish historical membership. A production universe and sector history need separate point-in-time evidence.
For \(\theta\sim N(0,V_0)\) and \(y\mid\theta\sim N(X\theta,\sigma^2I)\):
\[V_n=(V_0^{-1}+X^{\top}X/\sigma^2)^{-1},\qquad m_n=V_nX^{\top}y/\sigma^2.\]
The same posterior mean minimises squared error plus a quadratic penalty. A prior changes parameter estimates and predictions, while the posterior covariance adds uncertainty around them.
The next function also calculates the normalised marginal likelihood needed in §5.6.
The function uses linear solves rather than explicitly inverting the precision matrix. All prior variances are strictly positive. Its evidence calculation includes normalising constants; an unnormalised posterior density cannot replace it.
There are 207 training rows over 23 outcome periods before the 1 January 2021 freeze. Posterior feature means are −0.0129, +0.0331 and +0.0050 per training standard deviation.
A coefficient’s sign is conditional on the included predictors, sector effects, stock effects and prior. It does not establish a causal trading mechanism.
Question: Is a positive expected return the same as a positive realised return?
Result: Separate parameter from predictive probability and test prior sensitivity.
Builds on Feature effects and posterior covariance.
For a candidate row \(x_*\):
\[E[r_*\mid D]=x_*^\top m_n,\quad \operatorname{Var}(E[r_*\mid\theta]\mid D)=x_*^\top V_nx_*.\] \[\operatorname{Var}(r_*\mid D)=\sigma^2+x_*^\top V_nx_*.\]
A credible interval for the conditional mean leaves out the new observation’s residual shock. A predictive interval includes it.
For AAPL, P(conditional mean > 0.2%) is 0.974, whereas P(next return > 0.2%) is 0.638.
The 0.2% screen is an assumed round-trip hurdle for this experiment, rather than a prediction of actual turnover costs. Cash earns zero. Both probabilities are conditional on the declared likelihood and prior.
A posterior gives P(conditional mean > cost) = 0.97. What follows?
Across τ = 0.5%, 2% and 8%, the maximum absolute stock deviation changes from 0.11% to 1.26% to 4.35%. AAPL remains the highest forecast in this snapshot.
An unchanged top name does not imply that predictions, confidence or portfolio weights are insensitive. Report what changed and what stayed stable.
Predictive variance must include both parameter uncertainty and residual variation. Fix the assignment; keep the conditional-mean variance separately.
Repair the omitted uncertainty term.
Next: §5.6 combines competing models and takes forecasts with stock and date columns through a declared portfolio policy.
| Part | Question |
|---|---|
| 5.6.1 Model evidence and predictive mixtures | How should competing models contribute to a forecast? |
| 5.6.2 Decision objectives and walk-forward scoring | How does uncertainty enter a dated decision rule? |
| 5.6.3 Holdings, accounts and incremental evidence | Does the resulting policy improve the complete account? |
Question: How should competing models contribute to a forecast?
Result: Evidence weights and a mixture that includes model disagreement.
Begin with the definitions and assumptions; then trace the worked example.
%%{init: {"flowchart": {"rankSpacing": 20}, "themeVariables": {"fontSize": "20px"}}}%%
flowchart LR
A["Chapter 2<br/>aligned features"] --> B["Training-only<br/>Bayesian fit"]
B --> C["Model weights<br/>predictive mixture"]
C --> D["Frozen rule<br/>weights and cash"]
D --> E["Next outcomes<br/>costs and NAV"]
E --> F["Chapter 3<br/>evidence review"]
The process must be auditable at each fitting cutoff. Posterior calculations do not repair future information in an input, an overlapping label or an incorrectly reconstructed trading account.
For a declared candidate set \(M_1,\ldots,M_K\):
\[p(M_k\mid D)=\frac{p(D\mid M_k)p(M_k)}{\sum_jp(D\mid M_j)p(M_j)},\qquad p(D\mid M_k)=\int p(D\mid\theta_k,M_k)p(\theta_k\mid M_k)d\theta_k.\]
The evidence averages fit over the prior; it is not the likelihood at the best-fitting parameter. Proper priors and their normalisation matter.
Our three models differ only in stock-effect prior sd τ = 0.5%, 2%, 8%, with equal prior model probabilities.
This section is standalone: each cell includes the setup functions it needs. Models are recomputed from the current training sample; future portfolio performance never determines their weights.
The next cell defines prediction without relying on a previous section.
\[p(r_*\mid D)=\sum_kw_k\,p(r_*\mid D,M_k),\qquad \bar m=\sum_kw_km_k.\] \[\operatorname{Var}(r_*\mid D)=\sum_kw_k(v_k+m_k^2)-\bar m^2.\]
A weighted average of standard deviations is not this variance. Averaging only \(v_k\) also misses disagreement between the component means.
Mixture tail probabilities average component tail probabilities; the mixture is generally not Normal.
logsumexp prevents underflow when multiplying many small densities. The returned probabilities distinguish a conditional-mean hurdle from an actual return hurdle.
The model weights are 0.7728, 0.2264 and 0.0008. The training sample favours stronger pooling within these assumptions.
This does not mean a 77.28% probability that the tightest-prior strategy will make money.
A model has posterior model probability 0.90 within a three-model comparison. What does 0.90 describe?
Question: How does uncertainty enter a dated decision rule?
Result: Chronological refits and predictive scores on mature outcomes.
Builds on Model evidence and predictive mixtures.
A general Bayesian decision chooses an action \(w\) to maximise:
\[E[U(w^{\top}r_{t+1}-C(w,w_{t^-}))\mid D_t].\]
Utility \(U\), predictive joint returns, transaction costs \(C\), constraints and drifted holdings all matter.
This exercise uses a simpler declared rule: rank posterior mean returns; reserve three 0.33 slots; retain a slot only if P(conditional mean > 0.2%) ≥ 0.95. Otherwise keep cash. This is an illustrative policy, not a solved optimal portfolio.
Training row counts are 315, 423 and 531. The three annual blocks produce 315 predictions: 306 evaluable, nine pending.
Features and priors are transformed inside the training window. January’s freeze uses labels strictly before 1 January. Refit from the original prior on the complete expanding history, avoiding double-counting.
You retain last year’s posterior and also refit on every historical observation, including last year’s data. What is the problem?
The mixture has RMSE 0.0952, versus 0.0930 for a training-mean baseline. Its exact-mixture mean log predictive density is 0.9315 in decimal-return units; higher is better for comparable models on the same outcomes.
This forecast does not beat the simple baseline on RMSE. A favourable portfolio ranking would not reverse that finding.
Question: Does the resulting policy improve the complete account?
Result: Reconciled policy comparisons with sensitivity and leakage checks.
Builds on Decision objectives and walk-forward scoring.
A permitted name retains its original 0.33 allocation. Blocked slots stay in cash. The equal-weight comparator invests 0.99 across all nine names.
Exposure, costs and drawdown must accompany returns. A lower-exposure portfolio is not directly evidence of better stock selection.
The loop below represents changing portfolio state. Cash earns zero, trades cost 10 bps per dollar bought or sold, opening NAV is 100, and there is no terminal liquidation. The 1% target cash reserve funds the specified fees.
At every month: closing NAV − opening NAV = security P&L − fees. A positive prediction never substitutes for a realised return.
With opening NAV 100, the net ending values are 183.90 equal-weight, 204.77 ranking alone and 171.27 Bayesian screening. Maximum drawdowns are 18.15%, 34.26% and 34.26%, respectively. Exposure in this table averages the same 34 realised months as the ledger; the earlier signal summary covers all 35 prediction months.
The Bayesian threshold is evaluated as part of the whole policy. It does not inherit a performance claim merely from using posterior probabilities.
Compare the threshold policy with ranking alone using paired monthly net returns, then apply Chapter 3’s dependence-aware uncertainty methods. Thirty-four periods and a survivor sample provide limited evidence.
Here the screen subtracts about 0.55 percentage points per month from ranking alone on paired net returns. It also changes exposure. Forecast RMSE 0.0952 is worse than the training-mean baseline 0.0930. These results support further investigation or rejection of this rule, rather than a claim that the Bayesian screen adds value. The examined historical sample is not untouched evidence.
The altered future does change later training samples when those outcomes become eligible. It must not change predictions from the earlier frozen fit.
This implementation check does not verify historical source availability or validate the chosen likelihood.
The starter includes every prediction in an RMSE denominator. Fix the evaluation rows without removing pending prediction candidates.
Keep all forecasts; restrict evaluation to valid outcomes.
Report the target and clock; likelihood and priors; pooling assumptions; model set and evidence weights; parameter versus predictive uncertainty; forecast baseline; frozen holding rule; exposure, costs and net accounts.
State which result would change your decision. Include prior sensitivity and unsuccessful comparisons. A posterior probability describes the specified model, rather than certifying economic profitability.
Next: Chapter 6 develops time-series dependence and dynamic risk models under the same availability discipline.
Warning
The Federal Reserve’s May 1999 testimony identified excessive leverage and weak counterparty risk management as central lessons of LTCM’s 1998 crisis. Counterparties had not adequately understood its large positions in illiquid markets. See the Federal Reserve testimony.
Course application: a narrow posterior conditional on a return model cannot describe every funding or liquidation scenario. Stress joint losses, changing correlations, liquidity and leverage alongside posterior predictive checks. This historical case does not establish that LTCM used the Bayesian models taught here.
The memo records a decision supported by reproducible evidence, including unfavourable results.
Question: Does the hierarchical predictive mixture improve the nine-stock allocation relative to equal weights and posterior-mean ranking alone? Statistical evidence: Compare priors, model weights, RMSE 0.0952 against the training-mean baseline 0.0930, and parameter versus predictive uncertainty. Policy: State the 0.2% expected-return hurdle, 95% posterior mean screen, three 0.33 slots, cash treatment and 10 bps trading fees. Economic evidence: Report all three net accounts, exposure, drawdown and paired monthly differences. Recommendation: Choose further research, revision or rejection from the evidence; a profitable illustrative path alone does not establish a deployable edge. Next evidence: A declared fresh universe/period, dependence-aware likelihood and a documented search history.
Three prompts that work with this chapter’s tools — and the pitfalls each one hides.
“Write a Metropolis sampler for this log-posterior and report the acceptance rate.” Good. Then check the trace yourself: an LLM will happily call a 0.98 acceptance rate “excellent” when it means the step size is far too small and the chain has explored nothing.
“Which prior should I use for ν?” It will suggest Exponential(1/29) + 2 or Gamma(2, 0.1) — both fine — but ask it to draw prior predictive samples and show you P(ν < 5). The prior you accept should be one you have seen, not one you were told.
“Is the Bayesian Sharpe of 2.38 better than the classical 0.77?” The copilot will say yes. It is answering a different question. Ask instead: “which σ does each number divide by, and which one matches my risk limit?” — the answer in this chapter was “they measure different risks”, and no model chooses your risk definition for you.
| Concept | Tool |
|---|---|
| Posterior ∝ likelihood × prior | np.exp(log_post - log_post.max()) on a grid |
| Student-t return model, Sharpe posterior | scipy.stats.t.logpdf, Metropolis loop with rng.uniform() |
| Credible interval | np.percentile(draws, [2.5, 97.5]) |
| Prior / posterior predictive check | rng.normal(...) on parameter draws; Bayesian p-value |
| Conjugate normal-linear posterior | Sn = inv(inv(S0) + X.T @ X / σ²), rng.multivariate_normal |
| Robust regression | minimize(nll_t) with student_t.logpdf; weights \((\nu+1)/(\nu+r^2)\) |
| Sequential update / local level | new likelihood contributions; training-only parameter estimation |
| Hierarchical investment signals | joint normal posterior; partial pooling and prior sensitivity |
| Predictive mixtures | integrated evidence; logsumexp; total predictive variance |
| Portfolio decisions | fixed posterior rule; chronological forecasts; cash, drift and fees |
In Colab the same models are four lines of PyMC each — now you know what pm.sample does.
Next: Chapter 6 — Time Series Models for Trading and Risk.
Your fund reports Sharpe ratios on a standard-deviation basis to investors but sizes positions on the t-scale σ. Which posterior should go in the marketing deck, and what has to be disclosed alongside it?
The prior predictive check rejected Prior I as “absurd”, yet with 200 points Prior I and Prior II give nearly identical posteriors. When does the choice of prior actually change a decision — and how would you demonstrate that to a sceptical PM?
The Student-t fit gave the March 2020 days weights near 0.02. A risk manager objects: “those are exactly the days I care about”. Is robust regression the wrong tool for risk, the right tool for alpha, or both?
The local-level MLE said AAPL’s mean return does not wander but its volatility does. What would you expect for a fund’s strategy returns, and how would you set \(q\) if you had only 60 days of live track record?
Why can the highest-evidence model still produce a portfolio rule that loses to a simple comparator? Separate model probability, prediction loss and economic utility.
If the likelihood treats stock-month observations as independent, what changes would you make for shared market shocks and overlapping features?
Use the investment research notebook for §§5.5–5.6.
ISOM 5650 · Prof. Xuhu Wan · HKUST Business School