🪟 Window Functions

📚 What You'll Learn

By the end of this lesson, you will be able to:

Explain what a "window" is and how window functions differ from a plain aggregation
Use .rolling() for fixed-size sliding windows (moving averages, rolling std, and more)
Use .expanding() for cumulative, growing-from-the-start windows
Use .ewm() for exponentially weighted windows that favor recent observations
Apply any aggregation — mean, sum, min/max, std, custom — inside a window
Combine windows with .rank(), .shift(), and cumulative functions for real feature engineering

⏱️ Estimated Time: 45–60 minutes

🎯 Project: Build a moving-metrics toolkit — rolling averages and volatility, expanding running totals, EWM smoothing, and window-based rankings — on a numeric series.

Compute statistics over a moving slice of your data instead of the whole column at once.

🌟 Statistics That Slide

A normal aggregation collapses an entire column into a single number: one mean, one sum, one max. A window function is different — it computes that statistic over a moving slice of the data and returns a value for every position. The result lines up with your original rows, so you can attach a 7-day moving average, a running total, or a smoothed trend right next to the raw numbers.

Windows come in three flavors: a rolling window is a fixed-size frame that slides along; an expanding window starts small and grows to include everything seen so far; and an exponentially weighted window keeps all the history but gives recent points more say. Master these three and you can describe how any measurement behaves over its sequence, not just on average.

🔎 The Three Window Types

graph TD A[A numeric Series] --> B[.rolling(n)
fixed sliding frame] A --> C[.expanding()
grows from the start] A --> D[.ewm()
weighted toward recent] B --> E[same-length result
aligned to each row] C --> E D --> E style A fill:#f9f,stroke:#333,stroke-width:2px style E fill:#9f9,stroke:#333,stroke-width:2px

🪟 .rolling()

A fixed-size window slides across the series, one step at a time.

# 3-period moving average
s.rolling(window=3).mean()

# Rolling std (volatility)
s.rolling(window=5).std()

# Require full window before output
s.rolling(3, min_periods=3).sum()

📈 .expanding()

The window starts at the first row and grows to include every prior value.

# Running (cumulative) mean
s.expanding().mean()

# Running maximum so far
s.expanding().max()

# Same idea as cumsum for sums
s.expanding().sum()

⚖️ .ewm()

Every past point counts, but weights decay geometrically toward the present.

# Exponentially weighted mean
s.ewm(alpha=0.3).mean()

# Specify by span (like a 10-period EMA)
s.ewm(span=10).mean()

# Weighted volatility
s.ewm(span=10).std()

🎮 Interactive Sliding Window

Slide the frame across the data and choose an aggregation to see exactly what a rolling window computes at each step.

Window size 3 Position step 1

window result: -

📉 Rolling Average in Action

The classic use of a rolling window is smoothing. The faint line is raw, noisy data; the bold line is its 10-period rolling mean — the same signal with the jitter removed.

# Smooth noisy data with a moving average
df['smooth'] = df['value'].rolling(window=10).mean()

# Bollinger-style bands from a rolling window
roll = df['value'].rolling(20)
df['upper'] = roll.mean() + 2 * roll.std()
df['lower'] = roll.mean() - 2 * roll.std()

📈 Expanding Windows

An expanding window answers "what is the statistic using everything up to now?" Each bar below is a data point; the number above it is the running mean of all points seen so far — notice how it stabilizes as more data accumulates.

# Running average that never forgets
df['running_mean'] = df['value'].expanding().mean()

# Running total (equivalent to cumsum)
df['running_total'] = df['value'].expanding().sum()

# Best value seen so far
df['record'] = df['value'].expanding().max()

⚖️ Exponentially Weighted Windows

An EWM keeps the whole history but weights it with a decaying factor. The bars show how much influence each past observation (t, t-1, t-2, …) has on today's value — recent points dominate, distant ones fade.

# alpha controls how fast old data fades
df['ewm'] = df['value'].ewm(alpha=0.3).mean()

# Equivalent parameterizations
df['value'].ewm(span=10).mean()       # like a 10-period EMA
df['value'].ewm(halflife=5).mean()    # weight halves every 5 steps

🏅 Ranking Within Windows

Rankings are close cousins of window functions — they compare each value against the others in a group. Below, the same six scores are ranked several ways at once.

Name	Score	Rank	Dense Rank	Percentile	Quartile

# Standard competition rank (ties share a rank, gaps follow)
df['rank'] = df['score'].rank(ascending=False)

# Dense rank (no gaps after ties)
df['dense'] = df['score'].rank(method='dense', ascending=False)

# Percentile rank in [0, 1]
df['pct'] = df['score'].rank(pct=True)

💡 Pro Tips for Window Functions

Mind the warm-up: the first window-1 rolling results are NaN; use min_periods to relax this.
center=True aligns the window on each point instead of trailing behind it — great for visualization, risky for forecasting (it peeks ahead).
EWM never returns NaN after the first point, which makes it handy for gap-free smoothing.
Chain .shift() with a rolling result to build leak-free lag features for models.
Prefer built-in window methods over .apply() — they run in optimized C.

⚠️ Common Pitfalls to Avoid

Look-ahead bias: a centered window or forward fill can leak future data into a feature you use for prediction.
Forgetting to sort: rolling and expanding assume the rows are already in order.
Confusing rolling with resample: rolling keeps the same rows; resample changes the frequency.
Ignoring NaNs in the window: some functions skip them, changing effective window size.

📋 Quick Reference

Window Types:

.rolling() - Fixed-size sliding window
.expanding() - Cumulative window from start
.ewm() - Exponentially weighted window

Common Window Functions:

.mean() - Average within window
.sum() - Sum within window
.std() - Standard deviation
.var() - Variance
.min() / .max() - Minimum/Maximum
.median() - Median value
.quantile() - Quantile calculation
.apply() - Custom function
.corr() - Correlation
.cov() - Covariance

Shift Operations:

.shift(n) - Lag by n periods (positive n)
.shift(-n) - Lead by n periods (negative n)
.diff() - First difference
.pct_change() - Percentage change

Ranking Functions:

.rank() - Assign ranks
method='average' - Average rank for ties
method='min' - Minimum rank for ties
method='dense' - Dense ranking
pct=True - Percentile ranks
ascending=False - Reverse ranking

Cumulative Functions:

.cumsum() - Cumulative sum
.cumprod() - Cumulative product
.cummax() - Cumulative maximum
.cummin() - Cumulative minimum

📓 Learning Journal

Keep a learning journal — digital or physical. After this lesson, take a few minutes to write down:

Key concepts you learned
Techniques that clicked for you
Questions or confusion points to revisit
Ideas you want to try
Your progress and feelings about learning this

✍️ This lesson's prompt: Rolling, expanding, and EWM each "remember" the past differently. Pick a measurement you track over time and decide which window fits: would a fixed recent window, a running-since-the-start view, or a recency-weighted average tell the most honest story — and why?

📝 Lesson Summary

🎓 Key Takeaways

Window functions compute a statistic over a moving slice and return a value aligned to every row — unlike an aggregation that returns one number.
.rolling(n) uses a fixed sliding frame, .expanding() grows from the start, and .ewm() weights recent points more heavily.
Any aggregation — mean, sum, std, min/max, or a custom .apply() — can run inside a window.
Mind the warm-up NaNs, sort first, and avoid look-ahead bias when building features for prediction.

🎉 What You've Accomplished

You can now describe how a measurement behaves across its own sequence — smoothing noise with rolling means, tracking running totals with expanding windows, emphasizing recency with EWM, and ranking values within a window. These are the building blocks of time-aware feature engineering.

❓ Common Questions at This Stage

When should I use expanding() instead of rolling()?

Use rolling() when only the recent past matters and older data should drop out of the window (e.g. a 30-day moving average). Use expanding() when every observation so far should keep contributing — running totals, cumulative averages, or "best/worst to date" metrics.

What does alpha mean in .ewm()?

alpha (between 0 and 1) is the smoothing factor: it is the weight given to the most recent point, with every older point weighted by alpha·(1-alpha)^k. A larger alpha reacts faster to change; a smaller alpha smooths more. You can also specify the decay indirectly via span or halflife.

Why are the first few values of my rolling result NaN?

A rolling window needs a full window worth of observations before it can produce a value, so the first window-1 results are NaN. Set min_periods to a smaller number if you are willing to compute on a partial window at the start.

🔭 Looking Ahead

Window operations can get expensive on large data. Next you'll focus on performance — vectorization, efficient dtypes, and avoiding slow .apply() calls — so your rolling and grouped computations stay fast at scale.

✅ Before the Next Lesson

Add a rolling mean and a rolling std to a numeric series and plot them together.
Compare expanding().mean() with ewm(span=10).mean() on the same data.
Rank a column three ways (method='min', 'dense', pct=True) and note the differences.
Write your Learning Journal entry for this lesson.

🌟 Encouragement for the Journey

Window functions are the quiet workhorses of real analytics — the moving averages behind every dashboard and the smoothed lines behind every trend. Once you can slide a window in your head, a whole layer of data behavior becomes visible. Keep sliding!