Pandas: Shift Index Data with shift()

Shift index values forward or backward by a specified number of periods using the Pandas shift() method. Perform time-series lagging, create lead/lag columns for regression, and fill resulting missing values.

Show Table of Contents ↓

Introduction to shift() 🔝

The shift() method moves data values up or down along the index without changing the index labels themselves. Positive period values shift data downward (lagging), leaving NaN at the top, while negative values shift data upward (leading), leaving NaN at the bottom.

Create numerical sample series 🔝

We start with a sequential numerical Series to demonstrate shifting operations.

import pandas as pd

s = pd.Series([10, 20, 30, 40], name="VAL")
print(s)
assert s.shape == (4,)

Expected output

0    10
1    20
2    30
3    40
Name: VAL, dtype: int64

Basic shifting (lagging data) 🔝

Call shift(1) to push all values down by one row, leaving NaN in the first position.

shifted_down = s.shift(1)
print(shifted_down)
assert pd.isna(shifted_down.iloc[0]) and shifted_down.iloc[1] == 10.0

Expected output

0     NaN
1    10.0
2    20.0
3    30.0
Name: VAL, dtype: float64

Negative shifting (leading data) 🔝

Pass a negative integer to shift(-1) to pull values upward from future rows.

shifted_up = s.shift(-1)
print(shifted_up)
assert shifted_up.iloc[0] == 20.0 and pd.isna(shifted_up.iloc[3])

Expected output

0    20.0
1    30.0
2    40.0
3     NaN
Name: VAL, dtype: float64

Filling missing values with fill_value 🔝

Use the fill_value parameter to substitute generated NaN entries with a specified default number.

filled_shift = s.shift(1, fill_value=0)
print(filled_shift)
assert filled_shift.iloc[0] == 0

Expected output

0     0
1    10
2    20
3    30
Name: VAL, dtype: int64

Exercise: shift daily sales data 🔝

Given a Series sales = pd.Series([100, 200, 300], name="SALES"), shift the data by 1 period using fill_value=0. Verify that the second row equals 100.

Exercise solution 🔝

Solution creating the Series and applying shift().

sales = pd.Series([100, 200, 300], name="SALES")
answer = sales.shift(1, fill_value=0)
print(answer)
assert answer.iloc[1] == 100

Expected output

0      0
1    100
2    200
Name: SALES, dtype: int64

Practice in Google Colab 🔝

Interactive Colab Notebook

Practice shift() examples interactively directly in your browser:

View on GitHub

Reference: Pandas shift() documentation.

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author Watch on YouTube

Connect:

Subscribe to our YouTube Channel here