Chapter 1 · Data Structures and Methods of Series
Section 1.3 · Chapter 1 · Learning Statistics with Python
Data Structures and Methods of Series
Prof. Xuhu Wan
ISOM, HKUST Business School · 2026 Edition
The index is the address book of a Series. You will rename it, reset it, and predict how many rows a .loc slice returns versus an .iloc slice — the answers differ by one.
to_str = lambda v: "ID-" + v. What is aSeries.rename(to_str).index[0]?
'ID-ONE'
A function maps every label; a dict maps only the ones you list; assigning .index replaces the whole thing (and must have the right length).
aep.rename_axis(“FancyName”) changes …
"FancyName""FancyName".reset_index()reset_index() turns a Series into a two-column DataFrame whose first column is named after the index (FancyName, or Datetime). drop=True throws the timestamps away and leaves a plain RangeIndex — useful after filtering, dangerous for time series.
In Colab
energy = isom5650.data.energy() returns the same PJM hourly load table (2004–2018) with Datetime as the index; aep = energy["AEP"].
How many rows do aep.loc[“2017-01-01 01:00”:“2017-01-01 05:00”] and aep.iloc[0:5] return?
Label slice ends at 05:00 (5 rows); position slice ends at 04:00 (positions 0–4). .loc is right-inclusive because you name the end you want; .iloc is right-exclusive because it is Python. iloc[:trainsize] is how you will carve a time-ordered training set.
The notebook slices SPY with loc["2020-01-01":"2020-12-31"] and gets 253 rows. How many rows does sp500.loc["2020"] return?
253
"2020" means the whole year (253 trading days), "2020-03" the whole month (22 days). The March 2020 low was 2 237.40 on the 23rd.
Jan 2020 trading days sit at positions 0 (Jan 2), 1 (Jan 3), 2 (Jan 6), 3 (Jan 7), 4 (Jan 8) …
Do sp500.loc[“2020-01-06”:“2020-01-08”] and sp500.iloc[2:5] return the same rows?
.loc gives 2 rows, .iloc gives 3.iloc includes Jan 9Same three closes (3246.28, 3237.18, 3253.05) — then iloc[2:4] silently drops Jan 8. A backtest that mixes the two rules is off by one day at every boundary, and the tests still pass.
.loc also accepts a boolean mask — or a function that builds one.
Q3 = 147.4; 4 998 products above it, both ways. Passing a function (compare, or a lambda) is handy inside a method chain when the Series has no name yet.
s[0] is dangerousSort the prices descending: top = price.sort_values(ascending=False). Its index is now 15511, 4071, 2218, … — integer labels out of order. Predict top[0].
The discovery: top[0] matched the label 0, not the first element. Bare [ ] guesses, and its guess depends on the index type. Always say .loc or .iloc — one extra word, zero ambiguity.
The notebook asks for MSFT’s worst days; microsoft.csv covers 2015–2018.
From MSFT daily returns r_msft, select the days with a loss worse than 5 % into worst using .loc and a mask. Expected 3 days; the worst is 2015-01-27 at −9.25 %.
You want MSFT days with a return strictly between +2 % and +5 %. Predict what happens if you drop the parentheses.
r[r > 0.02 & r < 0.05] …
TypeError — & binds tighter than >41 days in the band, 83 beyond ±2 %. Two rules you will live by: &, |, ~ — never and, or, not — and parentheses around every comparison.
price.sample(500, replace=True, random_state=0) …
frac is givenfrac=0.002 of 20 000 is 40 rows. With replacement, only 492 of the 500 draws are distinct — 8 products came up twice.
aSeries has labels jan, feb, march. What does aSeries.reindex(["jan", "Tom"]).tolist() print?
[100.0, nan]
reindex conforms the Series to the labels you give — existing ones are selected, new ones get NaN, and the dtype turns float. .loc with a missing label raises KeyError instead. reindex is how you align two calendars.
Add 100 to every price above 50 in the first five products. There is a right way (one .loc) and a wrong way (chained [ ][ ]). Predict the right way’s result.
p = [44.99, 24.81, 46.0, 100.0, 121.95]. After p.loc[p > 50] = p.loc[p > 50] + 100, what is p.tolist()?
[44.99, 24.81, 46.0, 200.0, 221.95]
The rule: any time you assign to a slice, use one .loc. Two brackets in a row return a copy — your write is silently discarded (pandas may warn, may not). This is the most common pandas bug there is.
The notebook practice pairs every Friday close with the following Monday for NVDA; nvda_spy_daily_2023_2024.csv holds two years of NVDA.
nvda.index.day_name() gives the weekday of every trading day. Select the Friday closes into fridays. Expected 102 Fridays. (Bonus: pair each with the following Monday via nvda.shift(-1).)
rename maps labels (function or dict); rename_axis names the index itself; .index = … overwrites.reset_index() moves the index into a column; drop=True discards it..loc[a:b] is right-inclusive, .iloc[a:b] is right-exclusive; "2020" slices the whole year..loc accepts masks and functions — & | ~ with parentheses; bare s[0] guesses label-or-position; sample(replace=True) may repeat rows.reindex selects by label and invents NaN; .loc raises KeyError on unknown labels; one .loc = to write, never chained [ ][ ].Next: §1.4 — dates, shifts, rolling windows, drawdowns and resampling.
Prof. Xuhu Wan · HKUST ISOM · Learning Statistics with Python