Describe business data, quantify uncertainty, evaluate experiments and predictions, then examine portfolio risk
Prof. Xuhu Wan
ISOM, HKUST Business School · 2026 Edition
The methods apply across retail, operations, customer service and finance. Financial examples are transfer applications, not prerequisites for the statistical concepts.
Numbered parts preserve the teaching order. Examples run automatically after opening or refreshing; practice and repair cells run manually. Edit inputs in the current cell; refresh clears edits.
| Section | Concept | Tool |
|---|---|---|
| §3.1 | Feature Distributions and Diagnostics | Quantiles, ECDF, training-only rules |
| §3.2 | Sampling Uncertainty and Comparisons | Block bootstrap, HAC |
| §3.3 | Research Design and Multiple Testing | Search log, multiple testing |
| §3.4 | Association and Prediction Evaluation | Per-date Spearman IC, ICIR |
| §3.5 | Portfolio Losses and Tail Risk | Net losses, VaR, ES, drawdown, GEV/POT |
| Section | General application | Question |
|---|---|---|
| 3.1 | Customer waits and deliveries | Which population and distribution are described? |
| 3.2 | Operating days and service times | How uncertain is the estimated mean or difference? |
| 3.3 | Branch process changes and product experiments | What does the design and complete search support? |
| 3.4 | Demand scores and later sales | Which paired observations and cohorts are compared? |
| 3.5 | Portfolio losses | What do the tail, horizon and path imply? |
Before class, study the examples and bring one prediction, one uncertainty question and one risk limitation.
Download the guided notebook. Roles: analyst, validation reviewer and decision communicator.
Submit one runnable notebook and short decision memo using the existing course route. Include units, denominators, limitations and a proposed next test. Compare with another group before final revision.
Concepts and assumptions precede each application. Slides and Book share the diagnostics; the Book also develops BCa, experiment design and Bandits. All five sections remain assigned. Apply the same statistical methods to the class-specific discussion and project contexts.
| Part | Question |
|---|---|
| 3.1.1 Observations and distribution concepts | What population and distribution are we describing? |
| 3.1.2 Diagnose features across a fixed split | Does the later sample resemble development data? |
Question: What population and distribution are we describing?
Result: Distinguish a sample, an ECDF, a density and a fitted distribution.
A population distribution describes the random variable; a sample supplies observed units: customer visit, delivery, branch-week or trading day. Repeated observations within a branch or day can be dependent.
\[\mu=E[X],\qquad \bar x=\frac{1}{n}\sum_i x_i,\qquad s^2=\frac{1}{n-1}\sum_i(x_i-\bar x)^2.\]
The parameter \(\mu\) is unknown; \(\bar x\) is an estimator that changes across samples. Under i.i.d. sampling, \(s^2\) is unbiased for variance; \(s\) need not be unbiased for standard deviation.
Explain: Why can 639 rows describe a distribution without providing 639 independent observations?
\[F(x)=P(X\le x),\qquad \widehat F_n(x)=\frac{1}{n}\sum_i\mathbf1\{x_i\le x\}.\]
\[q_p=\inf\{x:F(x)\ge p\}.\]
The median is \(q_{0.5}\). Record the software quantile convention: interpolation can give an unobserved value. A 95th percentile is a location, not a confidence interval.
For a continuous variable, probability is area, not the height of its density:
\[P(a<X\le b)=\int_a^b f(x)\,dx,\qquad \widehat f_h(x)=\frac{1}{nh}\sum_i K\!\left(\frac{x-x_i}{h}\right).\]
Histogram bins and KDE bandwidth \(h\) affect apparent shape. Small \(h\) follows detail and noise; large \(h\) can merge modes.
Predict: Does a narrow density peak imply more probability than a wider, lower peak? Compare their areas.
Normal tails are thin; smaller Student-t \(\nu\) gives heavier tails. Its variance exists only for \(\nu>2\).
A Q–Q plot pairs sample and reference quantiles. Tail departures question the model; a straight centre does not validate extreme tails.
The KS distance is \(D=\sup_x|\widehat F_n(x)-F_0(x)|\). Its standard calibration assumes independent observations and a specified continuous \(F_0\). Fitting parameters on the same sample changes the null distribution; use a suitable calibrated test or simulation.
Check: A descriptive distribution fit is not proof of independent or stationary returns.
Question: Does the later sample resemble development data?
Result: A distribution comparison with fixed training-only cutoffs.
Builds on Observations and distribution concepts.
A later sample contains extreme feature values. Which clipping rule preserves the development boundary?
Use customer visits and delivery times to define the population, inspect distributions and compare a fixed later sample. The values below are invented for teaching; no company performance is claimed.
A service manager wants the mean wait of all customers. Would complaints alone represent that population?
Invented teaching data.
One row is one customer visit; waits are minutes. Complaint-only selection over-represents long waits in this fixture. A larger selected sample need not remove selection bias.
Four delivery times are 10, 20, 20 and 40 minutes. What fraction arrive within 20 minutes?
Invented teaching data.
The ECDF counts observations at or below the chosen cutoff, including ties. A sample quantile reverses the question; linear interpolation may give an unobserved time.
Two calls fall in each of three unequal-width bins. Do the bins need equal density heights?
Invented teaching data.
Each bin has probability 2/6. Density divides that probability by bin width, so all bar areas equal 1/3 while their heights differ. KDE bandwidth changes smoothing, not the observed calls.
Finance transfer example: Apply the same general statistical method to the existing market-data case.
Stock/date columns and training cutoffs are checked before any statistic. The manifest records the source and file hashes. This public current-survivor sample uses assumed availability clocks and static sectors; it demonstrates methods, not historical investability.
The feature table has 639 stock-month rows. How many independent experiments does that establish?
Which statement about an ECDF is correct?
The ECDF is descriptive under dependence. The i.i.d. DKW band cannot automatically certify these stock-months. Use density, Student-t and Q–Q diagnostics to investigate the shape.
Which pattern should trigger investigation of coverage and market conditions?
For a continuous variable, what gives the probability of falling between two values?
Follow-up: Worked examples: ECDF theory, KDE, Student-t and distribution diagnostics.
| Part | Question |
|---|---|
| 3.2.1 Sampling uncertainty and dependence | Why does the estimate change across samples? |
| 3.2.2 Uncertainty in repeated business measurements | Which observations should we resample together? |
Question: Why does the estimate change across samples?
Result: Explain a standard error, confidence interval and dependence assumption.
\[\operatorname{Bias}(\widehat\theta)=E[\widehat\theta]-\theta,\qquad \operatorname{MSE}(\widehat\theta)=\operatorname{Var}(\widehat\theta)+\operatorname{Bias}(\widehat\theta)^2.\]
Under i.i.d. sampling, \(\widehat{SE}(\bar X)=s/\sqrt n\). More customer rows within a branch or day are not necessarily independent observations; increasing \(n\) does not remove bias.
For an i.i.d. Normal sample with unknown variance, a mean interval is
\[\bar x\ \pm\ t_{n-1,\,0.975}\frac{s}{\sqrt n}.\]
A mean interval is not a prediction interval for an individual customer, delivery or future period. Coverage requires the sampling assumptions; larger samples do not repair selection bias or leakage.
Recompute \(\widehat\theta_b^*\) for each of \(B\) resamples; its 2.5th and 97.5th percentiles define the percentile interval. Longer blocks preserve longer local runs but provide fewer distinct blocks.
Assumptions: Enough data and a sufficiently stable process. More resamples reduce simulation noise; they do not create new historical information. BCa adds bias/acceleration adjustments and is developed in the Book.
For a stationary series with autocovariance \(\gamma_k\),
\[\operatorname{Var}(\bar X)=\frac{1}{n}\left[\gamma_0+2\sum_{k=1}^{n-1}(1-k/n)\gamma_k\right].\]
HAC means heteroskedasticity and autocorrelation consistent covariance estimation. In \(r_t=\mu+\varepsilon_t\), the OLS intercept estimates the mean measured outcome; HAC changes its uncertainty. For dependent daily operating measurements or monthly returns, inspect lag/block sensitivity. Chapter 4 develops OLS regression.
Question: Which observations should we resample together?
Result: A block-bootstrap interval checked against block length and HAC uncertainty.
Builds on Sampling uncertainty and dependence.
How should a bootstrap preserve short runs of dependent monthly observations?
Estimate mean service time, distinguish SD from SE and resample adjacent operating days. Choose the sampling unit and dependence assumptions before interpreting an interval.
Eight independent service times are observed. Does a 95% mean interval contain 95% of future service times?
Fixed-seed Normal simulation: true mean 20 min, SD 4 min.
SD describes individual service times; SE describes the sample mean across repeated samples. Under iid Normal sampling, 95% refers to repeated interval coverage of the fixed population mean, not individual customer coverage.
Six daily waiting-time deviations are −2, −1, 0, 1, 2 and 1 minutes. How does a two-day block preserve local order?
Invented teaching data.
Blocks preserve adjacency within a sampled run; block boundaries can jump and circular blocks wrap. This tiny fixture demonstrates mechanics, not a calibrated operational interval. Stable time behaviour and enough history remain assumptions; HAC is a different covariance estimate.
Finance transfer example: Apply the same general statistical method to the existing market-data case.
Must a block-bootstrap interval always be wider than an iid interval?
Each replicate has 34 month indices. Length 1 gives iid draws; length 3 retains three-month runs, wrapping at the last month. Independently sampled blocks assume a sufficiently stable process and cannot recreate unseen regimes or repair selection.
A 95% confidence interval for mean net return contains zero. What follows?
net.to_numpy() copies return values to a NumPy array so a two-dimensional index matrix can form resamples; the ledger retains the month labels.
34 months · selected sample · no adjustment for strategy search or unverified execution.
What does HAC change in an intercept-only regression?
sm names statsmodels.api; OLS means ordinary least squares. np.ones supplies the single intercept column. Chapter 4 develops regression models.
What does 95% coverage of a confidence-interval procedure mean?
Follow-up: Worked examples: Classical bootstrap, BCa, tests and rolling regime diagnostics.
| Part | Question |
|---|---|
| 3.3.1 Hypotheses and repeated comparisons | How can a search produce an attractive false finding? |
| 3.3.2 Pre-specified questions and a research log | What was chosen before the result was visible? |
Question: How can a search produce an attractive false finding?
Result: Distinguish a p-value, error rates and multiple-testing adjustments.
State \(H_0\) and \(H_1\) before testing. For example, \(H_0:\mu=0\) against \(H_1:\mu\ne0\) for a fixed strategy and return definition.
Type I rejects a true \(H_0\); Type II fails to reject a specified false \(H_0\). Power is \(1-\beta\) for the stated effect and design; failure to reject does not establish zero effect.
Independent true nulls give FWER \(1-(1-\alpha)^m\): 64% at \(m=20\), \(\alpha=0.05\). Bonferroni uses \(\alpha/m\) and needs no independence. BH finds the largest \(k\) with \(p_{(k)}\le kq/m\) and rejects through \(k\), under suitable dependence. Record the full search; neither method repairs leakage.
Random assignment helps identify causal effects; market backtests are observational. Fix the target, effect, features, split, costs, metrics and stopping rule before seeing results. Adaptive reward maximisation and fixed-rule evaluation answer different questions; the Book develops power, A/B, sequential testing and Bandits.
Question: What was chosen before the result was visible?
Result: A fixed comparison, complete search history and stopping rule.
Builds on Hypotheses and repeated comparisons.
You revise a model repeatedly after reading its OOS results. How should that period now be described?
Compare the same branches, state the counterfactual limits of before/after data, and account for every product experiment tried. Record the outcome metric, comparison and stopping rule in advance.
The same four branches are measured before and after a process change. What quantity should the paired test analyse?
Invented teaching data.
Pair by branch first; a negative change means fewer waiting minutes. The paired t-test assumes independent branch differences and, for an exact small-sample test, Normal differences. Before/after matching alone does not rule out seasonality or establish causation; a randomised comparison addresses a different design question.
Four product changes yield p-values .001, .02, .04 and .20. Does testing only the best result control the full search?
Invented teaching data.
Bonferroni controls the chance of any false rejection; BH controls the expected false-discovery proportion under its dependence conditions. Include all declared tests, retain valid base tests and pre-specify the metric and stopping rule. Peeking requires a sequential design; adaptive Bandits optimise reward rather than automatically supplying fixed-horizon A/B inference.
Finance transfer example: Apply the same general statistical method to the existing market-data case.
Fix the universe, timestamp contract, label horizon, primary metric, costs, model settings and evaluation dates before the result. Retain all candidates tried.
A holdout used repeatedly to revise features becomes development data. Randomisation assumptions from experiments do not transfer automatically to markets.
Which tested hypotheses belong in a multiple-testing family?
stats names scipy.stats, which supplies the one-sample test. multipletests applies family corrections; its first returned item contains the rejection flags.
One seeded draw · valid base tests · full declared family · suitable dependence.
Two strategies share evaluation dates. What is the natural comparison?
Log each dated hypothesis, configuration, source hashes, cutoff and all metrics. Apply the stated continue/reject rule; label new exploration separately. A winner crossing a threshold is not a valid retrospective stopping rule.
The Book works through A/B power, peeking, alpha spending and Bandits. Their randomised/stationary simulation assumptions do not establish a trading allocation policy.
Does Benjamini–Hochberg adjustment repair leaked future features?
A valid test reports p = 0.03. Which interpretation is correct?
Follow-up: Worked examples: A/B power, sequential testing, Bandits and multiple comparisons.
| Part | Question |
|---|---|
| 3.4.1 Covariance, correlation and dependence | Which kind of association does the statistic measure? |
| 3.4.2 Evaluate predictions within comparable cohorts | Does the score order outcomes within each date? |
Question: Which kind of association does the statistic measure?
Result: Choose a linear or rank measure and state what zero does not imply.
\[s_{xy}=\frac{1}{n-1}\sum_i(x_i-\bar x)(y_i-\bar y),\qquad r_{xy}=\frac{s_{xy}}{s_xs_y}.\]
Pearson lies between \(-1\) and \(1\). Constant inputs make it undefined; outliers, pair selection and pooled periods can change the answer.
Explain: A correlation of 0.6 does not mean a 60% probability of a correct trade or a 60% return.
Spearman is Pearson correlation of both rank vectors:
\[\rho_s=\operatorname{Corr}(\operatorname{rank}X,\operatorname{rank}Y).\]
Strictly increasing transformations preserve Spearman ranks. Without ties, \(\tau=(C-D)/\binom n2\); Kendall tau-b adjusts for ties. Both measures need the same valid pairs.
An IC is an association statistic applied to predictions and realised targets. This chapter uses per-date Spearman IC; state that definition whenever reporting IC.
For symmetric \(X\) and \(Y=X^2\), opposing slopes can cancel covariance. Population distance correlation detects broader dependence under finite-first-moment assumptions; sample coefficients do not prove independence, causation or tradable profit.
For \(T\) date-level IC values:
\[\overline{IC}=T^{-1}\sum_t IC_t,\qquad ICIR=\overline{IC}/s_{IC}.\]
This ICIR is not annualised. Dependence between dates affects uncertainty; inspect coverage, stability and a time-aware interval for mean IC.
Question: Does the score order outcomes within each date?
Result: Per-date IC with maturity, coverage and stability reported.
Builds on Covariance, correlation and dependence.
What is the sampling unit for the mean of monthly cross-sectional IC?
Inspect linear and rank association, then evaluate stores within each week. Keep predictions with pending sales; exclude them only from evaluation until the outcomes are observed.
Do higher demand scores accompany higher unit sales? Would Pearson detect a symmetric U-shaped relationship?
Invented teaching data.
Covariance has score-times-units dimensions. Pearson measures linear association; Spearman compares ranks and Kendall compares pair order. A zero Pearson coefficient can coexist with complete nonlinear dependence. Association alone establishes neither causation nor forecasting value on new data.
Stores are ranked before weekly sales arrive. Should a pending sales outcome remove a store from the earlier prediction list?
Invented teaching data.
Keep all seven predictions, but score only the six observed pairs. Compute one rank correlation per week before averaging weeks; pooled correlation and unequal weighting answer other questions. Retain counts, nonconstant vectors and stability. Finance calls this dated rank association IC; ICIR is its mean divided by its dated sample SD, not a success probability.
Finance transfer example: Apply the same general statistical method to the existing market-data case.
Pooled correlation mixes dates. Per-date IC asks whether scores order outcomes in the same cross-section; its dated average does not establish net profit.
Spearman IC requires which inputs?
Evaluation uses 306 predictions across 34 dates: nine stocks per date, three per sector. Sector-relative outcomes do not establish neutral holdings.
What happens to a candidate whose next-month label is pending?
Raw and sector-relative targets answer different questions. ICIR uses the dated sample standard deviation and remains unannualised; undefined dates must not be filled with zero.
Does positive sector-relative IC establish sector-neutral portfolio weights?
Inspect dated coverage and annual reversals. Use §3.2’s dependence-aware intervals for mean IC; compare these diagnostics with reconciled net-return evidence.
A population Pearson correlation is zero. What follows?
Follow-up: Worked examples: Pearson, Kendall, distance correlation and tail association.
| Part | Question |
|---|---|
| 3.5.1 Historical losses and path risk | Which loss horizon and denominator belong to the claim? |
| 3.5.2 EVT and block extremes | How does a block-extreme model differ from ordinary returns? |
| 3.5.3 Threshold excesses and daily tail risk | How do excesses above a threshold imply daily tail probabilities? |
| 3.5.4 Tail diagnostics and sensitivity | How sensitive is the extrapolation to sparse tail evidence? |
Question: Which loss horizon and denominator belong to the claim?
Result: Historical VaR, ES and drawdown on a reconciled account.
Define loss as \(L=-R\) for a stated horizon. A large positive value is a large loss.
\[\operatorname{VaR}_p=F_L^{-1}(p),\qquad \operatorname{ES}_p=\frac{1}{1-p}\int_p^1 F_L^{-1}(u)\,du.\]
For a continuous loss distribution, ES equals \(E[L\mid L\ge\operatorname{VaR}_p]\). VaR describes a cutoff; ES describes severity beyond it. Neither is a maximum possible loss.
The classroom code uses an interpolated sample quantile and averages observations at/above it. With finite samples or ties this convention can differ from exact fractional-tail quantile integration; report the actual tail count.
For net portfolio value \(V_t\) including the opening value,
\[H_t=\max_{s\le t}V_s,\qquad D_t=V_t/H_t-1,\qquad D_{\min}=\min_t D_t.\]
Drawdown measures loss from a running peak. We report its negative signed minimum; positive maximum-drawdown magnitude would be \(-D_{\min}\).
Specify horizon, fees, opening NAV and signed versus positive drawdown convention.
Block maxima: under suitable conditions, maxima of large blocks are approximated by a generalised extreme-value (GEV) distribution. Apply to losses; worst single-day loss in a month is not the compounded monthly loss.
Peaks over threshold: sufficiently high exceedances \(L-u\mid L>u\) can be approximated by a generalised Pareto distribution (GPD). Threshold choice trades approximation bias against a small tail sample.
The tail-shape parameter \(\xi\) distinguishes heavy unbounded tails (\(\xi>0\)), an exponential limit (\(\xi=0\)) and a bounded upper tail (\(\xi<0\)). Dependence may require declustering or adjusted inference.
Tail fits need stability checks and uncertainty. GEV and POT/GPD fitting, sensitivity and Q–Q diagnostics follow in this self-study section. Two tail months cannot validate an extreme-risk model.
The historical monthly ES uses two tail months. What is the appropriate interpretation?
Fees are already in net_return; do not subtract them twice. Do not use gross returns for a net-risk report. A distributional risk estimate is evidence for a risk review, not a guarantee that losses stop at that level.
Which risk statistic changes when return order changes but the set of returns stays fixed?
At 95%, only two of 34 months enter the tail mean. Changing the percentile exposes the small denominator; a fitted model adds assumptions rather than observed crises.
A GEV fitted to 20-day block minima describes what outcome?
The realised teaching path has signed minimum drawdown −0.1833. Review each leg, joint losses, exposure, concentration, borrowing and execution before a risk decision.
Which denominator belongs with tail co-exceedance?
The archive’s S&P/Tesla headers are reversed; price-series checks identify the columns. The read-time rename below corrects this example while preserving the CSV.
These thresholds use the full sample for description. Prospective monitoring requires prior-only thresholds and uncertainty; block extremes retain a single-day outcome.
For continuous losses, which description distinguishes VaR95 from ES95?
Question: How does a block-extreme model differ from ordinary returns?
Result: A GEV fit with explicit blocks, tail shape and support.
Builds on Historical losses and path risk.
The centre of a return distribution describes ordinary days. A risk review also asks how severe the rare losses could be, how frequently a threshold might be exceeded and whether joint stress defeats diversification.
Extreme-value theory (EVT) supplies limiting models for extremes and high-threshold excesses. It supports model-based extrapolation beyond observed quantiles, with assumptions and estimation uncertainty.
The Central Limit Theorem concerns suitably normalised sums or means. EVT concerns suitably normalised maxima or excesses. Averaging and taking a maximum answer different questions.
Check: A Normal-looking sample mean does not imply Normal one-day loss tails.
The worked example uses 20-day blocks; the diagram’s four-day blocks are illustrative. Block maxima and threshold excesses describe different samples and probabilities.
For a block maximum loss \(M\) and \(z=(m-\mu)/\sigma\),
\[G(m)=\exp\{-[1+\xi z]^{-1/\xi}\},\qquad 1+\xi z>0,\quad\sigma>0.\]
At \(\xi=0\), the limit is \(G(m)=\exp[-\exp(-z)]\) (Gumbel).
\(\mu\) is location, \(\sigma\) is scale, and \(\xi\) controls tail shape. Location is generally neither the mean nor the mode.
Python convention: scipy.stats.genextreme returns c = -xi. A negative c indicates a positive heavy-tail index.
\(\xi>0\): Fréchet; \(\xi=0\): Gumbel; \(\xi<0\): Weibull. These are fitted-family properties, not proof of the population’s tail.
For a positive heavy-tail index, the fitted model has a finite moment of order \(r\) only when \(r\xi<1\). Thus its variance requires \(\xi<1/2\).
An estimated index does not prove the population has or lacks a moment. Support, confidence intervals, dependence and stability all matter.
genextreme.fit returns c = -0.35. Which interpretation follows from this fitted family?
Predict: How many complete 20-day blocks come from 1,496 daily returns?
The largest daily log loss is 0.2563, a simple loss of about 22.6%. The same single-day outcome remains an extreme when selected from its 20-day block.
The fit gives \(\xi\approx0.354\), location 0.0136 and scale 0.0070. Its 95th block-extreme percentile is 0.0506. This is the fitted worst-day threshold exceeded in 5% of 20-day blocks, rather than a compounded 20-day loss.
Starting values seed optimisation; compare alternative starts and likelihoods, support and stability.
Question: How do excesses above a threshold imply daily tail probabilities?
Result: A POT/GPD calculation with the exceedance fraction retained.
Builds on EVT and block extremes.
Set a high loss threshold \(u\). Conditional on \(L>u\), define the positive excess \(Y=L-u\).
\[H(y)=1-(1+\xi y/\beta)^{-1/\xi},\qquad y\ge0,\quad1+\xi y/\beta>0.\]
\(\beta>0\) is excess scale. At \(\xi=0\), \(H(y)=1-e^{-y/\beta}\). With \(\xi<0\), excesses end at \(-\beta/\xi\).
genpareto uses c = xi, unlike GEV. Fix excess location to zero with floc=0. The threshold \(u\) belongs outside the excess distribution.
Try: Set ξ to −0.25, 0 and +0.50. Explain the endpoint and the change in the conditional 95th percentile while scale remains fixed.
The 1.5% threshold gives 91 excesses from 1,496 days: 6.08%, with \(\xi=0.4363\) and scale 0.00643. The whole 1985–1990 fit is descriptive; forecasts require prior-only observations.
Let \(p_u=N_u/n\). For \(q>1-p_u\) and \(\xi\ne0\),
\[\operatorname{VaR}_q=u+\frac{\beta}{\xi}\left[\left(\frac{p_u}{1-q}\right)^\xi-1\right].\]
For the fitted continuous GPD tail and \(\xi<1\),
\[\operatorname{ES}_q=\frac{\operatorname{VaR}_q+\beta-\xi u}{1-\xi}.\]
At \(\xi=0\), VaR becomes \(u+\beta\log[p_u/(1-q)]\), and ES becomes VaR \(+\beta\).
At \(\xi\ge1\), the fitted tail has infinite ES. The displayed finite estimates use the original fit with \(\xi<1\).
The fitted daily log-loss VaR99 is 0.0327 and ES99 0.0577. VaR transforms to a simple loss of about 3.21%; transforming the ES average does not give simple-loss ES.
Question: How sensitive is the extrapolation to sparse tail evidence?
Result: A risk statement supported by threshold, Q–Q and tail-shape diagnostics.
Builds on Threshold excesses and daily tail risk.
Why should we compare several plausible POT thresholds?
Threshold differences are sensitivity evidence. Check mean-excess plots, parameter stability and dependence-aware uncertainty; the three estimates are not a calibrated confidence interval.
The 1987 crash remains the most fragile observation. Clustered exceedances may require declustering or conditional-volatility modelling; record the changed target and denominator.
A 1-in-T-day return level \(x_T\) satisfies \(P(L>x_T)=1/T\) in the fitted stationary marginal model.
\[x_T=u+\frac{\beta}{\xi}\left[(p_uT)^\xi-1\right].\]
The 2,500-day level uses only 1,496 observations. Model estimates extrapolate; interpolated empirical quantiles stay within the sample extremes.
Check: “1-in-1,000” does not mean the next breach arrives exactly 1,000 days later. Clustering and changing regimes further weaken a simple waiting-time interpretation.
For descending positive losses \(L_{(1)}\ge\cdots\ge L_{(k+1)}>0\),
\[\widehat\xi_k=\frac1k\sum_{i=1}^k\log\frac{L_{(i)}}{L_{(k+1)}}.\]
Hill assumes a heavy, regularly varying tail. Plot \(\widehat\xi_k\) against \(k\): a stable region can support a threshold choice, but a plateau is not proof.
Too few extremes give noise; too many admit the body. GEV, GPD and Hill use different finite samples and need not agree exactly.
Continue in the Book: Run the Hill and return-level cells, then compare their assumptions and samples with the two fits above.
Which summary correctly reports the POT result?
Explain in your own words: What changes if the threshold rises? Which units differ from the monthly portfolio report? What additional evidence would you need for a prospective risk decision?
Follow-up: Worked examples: GEV, POT/GPD, Hill, extreme scores and return levels.
Report Expected Shortfall from the selected tail, rather than all monthly losses. Fix the assignment.
Ask for the columns that identify each table row, sampling unit, horizon and valid-pair count before any statistic.
Ask whether the resampling or test assumes independent observations, fitted parameters or an untouched holdout.
Ask for a claim supported by the output, its strongest limitation and the next test. Reproduce the numbers yourself before using the explanation.
A researcher tries many features, removes difficult months, changes a cutoff after reading OOS results and reports the winning Sharpe. This is a teaching scenario.
The accounting may reconcile while the claim remains unsupported: the search family is hidden, the holdout has become development data and the uncertainty calculation treats dependent periods as independent.
Repair: retain the complete search, fix the next evaluation protocol and assess the frozen candidate on new evidence.
Question and data: state the fixed baseline, public-sample limitations, timestamps and costs.
Evidence: report per-date IC and coverage, a dependence-aware interval, monthly net risk and the tail denominator.
Decision: explain whether the evidence supports further research, revision or rejection. Name one limitation that could change that judgement and a prospective next test.
Peer challenge: identify the strongest claim the evidence does not support; revise the memo before submission.
| Classroom unit | Evidence to retain |
|---|---|
| Distribution diagnosis | Quantiles, ECDF, training-only settings |
| Signal diagnosis | Dated IC, coverage, stability, defined targets |
| Backtest evidence | Block interval, sensitivity, complete search history |
| Tail risk | Net loss horizon, VaR/ES tail count, drawdown |
Next: Chapter 4 develops predictive models; Chapter 6 develops time dependence and risk dynamics.
ISOM 5650 · Prof. Xuhu Wan · HKUST Business School