Pandas DataFrame Columns: List, Rename and Clean Headers

Use df.columns to inspect column names, check required fields and prepare a consistent table schema. It is an attribute, so write it without parentheses. Use df.columns.tolist() when you need an ordinary Python list.

Read column names with df.columns 🔝

columns is an attribute, not a function: use df.columns, without parentheses. It returns an Index containing the column labels. The number of labels is the number of columns, rather than the number of rows.

import pandas as pd

def student_data():
    return pd.DataFrame({
        "NAME": ["Ravi", "Raju", "Alex", "Ron", "King", "Jack"],
        "ID": [1, 2, 3, 4, 5, 6],
        "MATH": [80, 40, 70, 70, 60, 30],
        "ENGLISH": [80, 70, 40, 50, 60, 30]
    })

df = student_data()
print(df.columns)
print("Column count:", len(df.columns))
assert len(df.columns) == df.shape[1] == 4

Expected output

Index(['NAME', 'ID', 'MATH', 'ENGLISH'], dtype='str')
Column count: 4

Convert column labels to a list 🔝

Use df.columns.tolist() or list(df.columns). The older values-based form also works, but does not need to be used just to obtain a list. keys() returns the column Index for a DataFrame.

df = student_data()
print(df.columns.tolist())
print(list(df.columns))
print(df.columns.values.tolist())
print(df.keys())
assert df.keys().equals(df.columns)

Expected output

['NAME', 'ID', 'MATH', 'ENGLISH']
['NAME', 'ID', 'MATH', 'ENGLISH']
['NAME', 'ID', 'MATH', 'ENGLISH']
Index(['NAME', 'ID', 'MATH', 'ENGLISH'], dtype='str')

Get a label by position and select data 🔝

df.columns[2] is the third column label. Use that label to select a Series, or double brackets to retain a DataFrame. For positional selection of values, use iloc; it is also an indexer rather than a function.

df = student_data()
label = df.columns[2]
print("Third label:", label)
print(df[label].tolist())
print(df[[label]].head(2))
assert label == "MATH"

Expected output

Third label: MATH
[80, 40, 70, 70, 60, 30]
   MATH
0    80
1    40

Loop over names and pass columns to a function 🔝

Iterating over df.columns yields labels in order. Access the corresponding data with df[label]. For calculations over an entire table, prefer a suitable Pandas operation to a Python loop where possible.

df = student_data()
for label in df.columns:
    print(label)

def my_fun(table, label):
    print(label, table[label].tolist())

for label in df.columns:
    my_fun(df, label)

Expected output

NAME
ID
MATH
ENGLISH
NAME ['Ravi', 'Raju', 'Alex', 'Ron', 'King', 'Jack']
ID [1, 2, 3, 4, 5, 6]
MATH [80, 40, 70, 70, 60, 30]
ENGLISH [80, 70, 40, 50, 60, 30]

Create an empty table with named columns 🔝

Named columns define the header even when there are no rows. They do not automatically define useful numeric or date dtypes. Provide explicit typed Series when the empty table needs a schema.

empty = pd.DataFrame(columns=["A", "B", "C", "D", "E", "F", "G"])
print(empty)
print("Shape:", empty.shape)
typed = pd.DataFrame({"order_id": pd.Series(dtype="int64"), "amount": pd.Series(dtype="float64")})
print(typed.dtypes)

Expected output

Empty DataFrame
Columns: [A, B, C, D, E, F, G]
Index: []
Shape: (0, 7)
order_id      int64
amount      float64
dtype: object

Distinguish labels from column dtypes 🔝

df.columns describes the labels, while df.dtypes describes the data in each column. A date-looking string needs explicit conversion. An unambiguous format makes the intended day and month order clear; see to_datetime().

dates = pd.DataFrame({"NAME": ["Ravi", "Raju", "Alex"], "dt_start": ["1-1-2020", "2-1-2020", "5-1-2020"]})
print("Before:")
print(dates.dtypes)
dates["dt_start"] = pd.to_datetime(dates["dt_start"], format="%d-%m-%Y")
print("After:")
print(dates.dtypes)
print(dates["dt_start"].dt.strftime("%Y-%m-%d").tolist())

Expected output

Before:
NAME        str
dt_start    str
dtype: object
After:
NAME                   str
dt_start    datetime64[us]
dtype: object
['2020-01-01', '2020-01-02', '2020-01-05']

Add a column using a list or a scalar 🔝

A list must have one value per row. A scalar is repeated for every row. Assignment to an existing label replaces that column, so check names before adding data.

df = student_data()
classes = ["Four", "Three", "Five", "Six", "Two", "Three"]
df["my_class"] = classes
df["school"] = "Central"
print(df)
try:
    df["wrong_length"] = ["Four"] * 5
except ValueError:
    print("A five-item list cannot fill six rows.")

Expected output

   NAME  ID  MATH  ENGLISH my_class   school
0  Ravi   1    80       80     Four  Central
1  Raju   2    40       70    Three  Central
2  Alex   3    70       40     Five  Central
3   Ron   4    70       50      Six  Central
4  King   5    60       60      Two  Central
5  Jack   6    30       30    Three  Central
A five-item list cannot fill six rows.

Insert a column at a chosen position 🔝

insert(2, ...) places a new column before the current third column. It modifies the table and returns None. Duplicate names are rejected by default; allow_duplicates=True permits them, but usually makes later selection ambiguous.

df = student_data()
classes = ["Four", "Three", "Five", "Six", "Two", "Three"]
df.insert(2, "my_class", classes)
print(df.columns.tolist())
duplicate_demo = student_data()
duplicate_demo.insert(2, "NAME", "Four", allow_duplicates=True)
print("Duplicate names allowed:", duplicate_demo.columns.tolist())
assert not duplicate_demo.columns.is_unique

Expected output

['NAME', 'ID', 'my_class', 'MATH', 'ENGLISH']
Duplicate names allowed: ['NAME', 'ID', 'NAME', 'MATH', 'ENGLISH']

Use assign() to return a new table 🔝

assign() returns a DataFrame rather than changing the source table. A scalar can be supplied through assignment, insert or assign; reset the source for each example so repeated labels do not cause an insertion error.

df = student_data()
df2 = df.assign(my_class=["Four", "Three", "Five", "Six", "Two", "Three"])
print("Source:", df.columns.tolist())
print("Result:", df2.columns.tolist())
a = student_data()
a["my_class"] = "Four"
b = student_data()
b.insert(2, "my_class", "Four")
c = student_data().assign(my_class="Four")
print("Scalar assignment:", a["my_class"].tolist())
print("Scalar insert:", b.columns.tolist())
print("Scalar assign:", c["my_class"].tolist())

Expected output

Source: ['NAME', 'ID', 'MATH', 'ENGLISH']
Result: ['NAME', 'ID', 'MATH', 'ENGLISH', 'my_class']
Scalar assignment: ['Four', 'Four', 'Four', 'Four', 'Four', 'Four']
Scalar insert: ['NAME', 'ID', 'my_class', 'MATH', 'ENGLISH']
Scalar assign: ['Four', 'Four', 'Four', 'Four', 'Four', 'Four']

Add values from a dictionary by matching names 🔝

Assigning a dictionary directly to a column aligns its keys with row labels; it does not automatically match a NAME column. Use map() to look up a value for each name. Missing keys produce missing values that should be reviewed.

df = student_data()
class_by_name = {"Ravi": "Four", "Raju": "Three", "Alex": "Five", "Ron": "Six", "King": "Two", "Jack": "Eight"}
df["my_class"] = df["NAME"].map(class_by_name)
print(df[["NAME", "my_class"]])
assert df["my_class"].notna().all()
# A Series assignment uses index alignment, not list position:
aligned = student_data()
aligned["note"] = pd.Series({1: "review", 0: "ready"})
print(aligned[["ID", "note"]].head(3))

Expected output

   NAME my_class
0  Ravi     Four
1  Raju    Three
2  Alex     Five
3   Ron      Six
4  King      Two
5  Jack    Eight
   ID    note
0   1   ready
1   2  review
2   3     NaN

Add sequential IDs after resetting the index 🔝

reset_index() can retain the old row labels as a column. A sequence generated after reset follows the current row order. It is not a persistent database identifier.

df = student_data().reset_index()
df = df.rename(columns={"index": "New_ID"})
df["New_ID"] = df.index + 1000
print(df[["New_ID", "NAME"]])

Expected output

   New_ID  NAME
0    1000  Ravi
1    1001  Raju
2    1002  Alex
3    1003   Ron
4    1004  King
5    1005  Jack

Delete a column with drop() 🔝

Use drop(columns=...) to make the intent clear. The returned table has fewer columns; the source remains unchanged unless you assign it back. Unknown labels raise KeyError by default. See drop() for row and column removal.

df = student_data().assign(**{"Page Value": 0})
result = df.drop(columns="Page Value")
print(result.columns.tolist())
print("Source retains Page Value:", "Page Value" in df.columns)
# Equivalent explicit-axis, in-place form:
df.drop(labels="Page Value", axis=1, inplace=True)
assert df.columns.tolist() == result.columns.tolist()

Expected output

['NAME', 'ID', 'MATH', 'ENGLISH']
Source retains Page Value: True

Rename one column or replace every label 🔝

Use rename() for selected labels. Assigning df.columns replaces all labels position by position, so the list length must match. It does not reorder the data. errors="raise" helps catch a misspelled source label.

df = student_data().assign(my_class="Four")
renamed = df.rename(columns={"my_class": "my_class4"}, errors="raise")
print(renamed.columns.tolist())
report = pd.DataFrame({"old_page": ["/python/"], "old_views": [12], "old_users": [8], "old_avg": [1.5]})
report.columns = ["Page", "p_view", "u_view", "avg"]
print(report)
try:
    report.columns = ["Page", "views"]
except ValueError:
    print("Provide exactly four labels for this four-column table.")

Expected output

['NAME', 'ID', 'MATH', 'ENGLISH', 'my_class4']
       Page  p_view  u_view  avg
0  /python/      12       8  1.5
Provide exactly four labels for this four-column table.

Check whether columns exist before selection 🔝

Membership checks are exact and case-sensitive. For several required labels, report all missing names together before accessing the data. Do not silently fill a missing required column with defaults.

df = student_data()
if "Gender" in df.columns:
    print("Gender column is present")
else:
    print("Gender column is not present")
required = ["NAME", "MATH", "ENGLISH"]
missing = [label for label in required if label not in df.columns]
print("Missing required columns:", missing)
print(df.loc[:, required].head(2))

Expected output

Gender column is not present
Missing required columns: []
   NAME  MATH  ENGLISH
0  Ravi    80       80
1  Raju    40       70

Practical workflow: normalize sales headers 🔝

CSV and Excel headers may have spaces or inconsistent case. For string headers, strip surrounding spaces, convert to lowercase and replace runs of whitespace with underscores. These are explicit naming rules, not a guarantee that two business fields mean the same thing. str.replace() explains regex replacement.

sales = pd.DataFrame({" Order ID ": [101, 102], "Product Name": ["Notebook", "Pen"], " Amount ": [12.5, 3.5]})
cleaned = sales.copy()
normalized = cleaned.columns.str.strip().str.lower().str.replace(r"\s+", "_", regex=True)
if not normalized.is_unique:
    raise ValueError("Header normalization would create duplicate names")
cleaned.columns = normalized
required = ["order_id", "product_name", "amount"]
missing = [name for name in required if name not in cleaned.columns]
if missing:
    raise ValueError(f"Missing required columns: {missing}")
ordered = cleaned.loc[:, required]
print("Original:", sales.columns.tolist())
print("Cleaned:", ordered.columns.tolist())
print(ordered)
print(ordered.to_csv(index=False).rstrip())
assert ordered.columns.tolist() == required

Expected output

Original: [' Order ID ', 'Product Name', ' Amount ']
Cleaned: ['order_id', 'product_name', 'amount']
   order_id product_name  amount
0       101     Notebook    12.5
1       102          Pen     3.5
order_id,product_name,amount
101,Notebook,12.5
102,Pen,3.5

Detect duplicate names before they cause ambiguity 🔝

Cleaning can turn different labels into the same name. Stop and decide whether to rename, combine or remove those fields. A duplicated label can make df["amount"] return a DataFrame instead of a Series. Do not automatically discard a column just because its name is repeated.

raw = pd.DataFrame([[10, 12]], columns=[" Amount ", "amount"])
candidate = raw.columns.str.strip().str.lower()
duplicates = candidate[candidate.duplicated(keep=False)].tolist()
print("Conflicting cleaned labels:", duplicates)
duplicate_table = pd.DataFrame([[10, 12]], columns=["amount", "amount"])
print("Selection type:", type(duplicate_table["amount"]).__name__)
assert not candidate.is_unique

Expected output

Conflicting cleaned labels: ['amount', 'amount']
Selection type: DataFrame

Reorder known columns or create optional ones 🔝

Selection with loc raises an error for missing labels. reindex(columns=...) can create a missing optional column filled with missing values. Validate required fields first, then use reindex deliberately for optional fields.

df = student_data()
reordered = df.loc[:, ["ID", "NAME", "ENGLISH", "MATH"]]
print(reordered.columns.tolist())
with_optional = df.reindex(columns=["ID", "NAME", "Gender"])
print(with_optional.head(2))
assert with_optional["Gender"].isna().all()

Expected output

['ID', 'NAME', 'ENGLISH', 'MATH']
   ID  NAME  Gender
0   1  Ravi     NaN
1   2  Raju     NaN

Handle numeric and MultiIndex labels 🔝

Column labels need not be strings. Check their type before applying string cleaning. MultiIndex columns use tuples; select a full tuple for a particular field. Flattening with a separator can cause collisions, so validate the resulting names.

numeric = pd.DataFrame([[5, 6]], columns=[10, 20])
print("Numeric labels:", numeric.columns.tolist())
multi = pd.DataFrame([[12, 3]], columns=pd.MultiIndex.from_tuples([("sales", "amount"), ("sales", "units")]))
print("Tuple labels:", multi.columns.tolist())
print("Amount:", multi[("sales", "amount")].tolist())
flat = ["_".join(map(str, label)) for label in multi.columns]
assert len(flat) == len(set(flat))
multi.columns = flat
print("Flattened:", multi.columns.tolist())

Expected output

Numeric labels: [10, 20]
Tuple labels: [('sales', 'amount'), ('sales', 'units')]
Amount: [12]
Flattened: ['sales_amount', 'sales_units']

Common questions and next steps 🔝

Why does df.columns() fail? columns is an attribute; remove the parentheses. Why does a renamed column disappear? Check that you assigned the returned table and used the exact old label. Why is a list assignment rejected? Its length must match the row count; header replacement must match the column count. Why are mapped values missing? Check the mapping keys and the lookup column. Where can I inspect the whole table? Use info(), dtypes and describe(). For a wider view of labels and dimensions, explore DataFrame attributes.

Exercise: validate and prepare an export 🔝

Create a two-row table with headers Customer Name, Order Total and Order ID, including surrounding spaces. Normalize the headers, confirm the three required fields exist, rename order_total to amount and reorder the output as order_id, customer_name, amount. Preserve the input headers and verify that the total amount remains 30.

Exercise solution 🔝

Validate the header names before creating the export. Renaming and reordering columns should not change the numerical data.

original = pd.DataFrame({" Customer Name ": ["Alex", "Ravi"], " Order Total ": [10, 20], " Order ID ": [1, 2]})
result = original.copy()
labels = result.columns.str.strip().str.lower().str.replace(r"\s+", "_", regex=True)
assert labels.is_unique
result.columns = labels
assert {"customer_name", "order_total", "order_id"}.issubset(result.columns)
result = result.rename(columns={"order_total": "amount"}, errors="raise")
result = result.loc[:, ["order_id", "customer_name", "amount"]]
print(result)
print("Amount total:", result["amount"].sum())
assert result["amount"].sum() == 30
assert original.columns[0] == " Customer Name "

Expected output

   order_id customer_name  amount
0         1          Alex      10
1         2          Ravi      20
Amount total: 30

Practice in Google Colab 🔝

Open in Google Colab View on GitHub
Run the examples in order, change the column names and inspect how validation and selection change. Save your own copy to keep edits. All sample tables are included in the notebook.

Continue with data cleaning, reading CSV files, reading Excel files and exporting to Excel.

Reference: Pandas DataFrame.columns documentation.

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author   Watch on YouTube

Connect: