Pandas Categorical Data: Categories, Order and Reports

Create categorical columns, define category order, review unknown values and build complete category reports.

Convert the original grades with astype() 🔝

Categorical data stores values using a defined category set. Preserve the original grade sample and use astype() to convert it. These grades are synthetic practice data.

import pandas as pd
grades = pd.DataFrame({"grade": ["a", "c", "b", "b", "b", "c"]})
grades["grade"] = grades["grade"].astype("category")
print(grades.dtypes)
print(grades["grade"].cat.categories.tolist())
assert isinstance(grades["grade"].dtype, pd.CategoricalDtype)

Expected output

grade    category
dtype: object
['a', 'b', 'c']

Create the original categorical Series 🔝

dtype="category" infers categories from known values and leaves them unordered. Codes are internal category positions, not numeric scores.

sample = pd.Series(["d", "a", "b", "c", "a", "c"], dtype="category")
print(sample)
print("Codes:", sample.cat.codes.tolist())
assert sample.cat.categories.tolist() == ["a", "b", "c", "d"]

Expected output

0    d
1    a
2    b
3    c
4    a
5    c
dtype: category
Categories (4, str): ['a', 'b', 'c', 'd']
Codes: [3, 0, 1, 2, 0, 2]

Create categorical bands with the original cut() example 🔝

Preserve all six students and marks. The original bins use right-closed intervals: (1,50], (50,70], (70,100]. Check boundaries before applying these rules to other marks. More on cut().

students = pd.DataFrame({"NAME": ["Ravi", "Raju", "Alex", "Ron", "King", "Jack"],
    "ID": [1, 2, 3, 4, 5, 6], "MATH": [80, 40, 70, 70, 60, 30],
    "ENGLISH": [80, 70, 40, 50, 60, 30]})
students["my_cut"] = pd.cut(students["MATH"], bins=[1, 50, 70, 100])
print(students[["NAME", "MATH", "my_cut"]])
print(students["my_cut"].dtype)
assert isinstance(students["my_cut"].dtype, pd.CategoricalDtype)
assert students["my_cut"].notna().all()

Expected output

   NAME  MATH     my_cut
0  Ravi    80  (70, 100]
1  Raju    40    (1, 50]
2  Alex    70   (50, 70]
3   Ron    70   (50, 70]
4  King    60   (50, 70]
5  Jack    30    (1, 50]
category

Define a meaningful category order 🔝

An explicit CategoricalDtype establishes a stable schema. ordered=True makes sorting and comparisons use the supplied order rather than alphabetical order.

priority_type = pd.CategoricalDtype(["low", "medium", "high"], ordered=True)
priority = pd.Series(["high", "low", "medium"], dtype=priority_type)
print(priority.sort_values().tolist())
print("Above low:", (priority > "low").tolist())
assert priority.sort_values().tolist() == ["low", "medium", "high"]

Expected output

['low', 'medium', 'high']
Above low: [True, False, True]

Review values outside the category set 🔝

Casting to a restricted set converts unknown values to missing. Inspect raw values first so conversion does not hide data-quality problems. Missing values have category code -1.

raw = pd.Series(["low", "urgent", None], dtype="string")
unknown = raw.notna() & ~raw.isin(priority_type.categories)
converted = raw.astype(priority_type)
print("Unknown raw values:", raw.loc[unknown].tolist())
print("Missing after conversion:", converted.isna().tolist())
print("Codes:", converted.cat.codes.tolist())
assert raw.loc[unknown].tolist() == ["urgent"]
assert converted.cat.codes.tolist() == [0, -1, -1]

Expected output

Unknown raw values: ['urgent']
Missing after conversion: [False, True, True]
Codes: [0, -1, -1]

Add and rename categories explicitly 🔝

Assigning a new value requires adding that category first. These operations return new objects; retain reassignment. remove_unused_categories() can simplify a local result but changes its category schema.

extended = priority.cat.add_categories(["urgent"])
extended.iloc[0] = "urgent"
renamed = extended.cat.rename_categories({"low": "routine"})
print(renamed.tolist())
print(renamed.cat.remove_unused_categories().cat.categories.tolist())
assert priority.iloc[0] == "high" and renamed.iloc[1] == "routine"

Expected output

['urgent', 'routine', 'medium']
['routine', 'medium', 'urgent']

Choose observed or complete category reports 🔝

Set observed explicitly in groupby to make the report policy clear across Pandas versions. Include unobserved categories for a complete schema; dropna determines whether missing keys form a group.

orders = pd.DataFrame({"priority": pd.Series(["low", "high", "low"], dtype=priority_type),
    "amount": [10, 20, 5]})
observed = orders.groupby("priority", observed=True)["amount"].sum()
complete = orders.groupby("priority", observed=False)["amount"].sum()
print(observed)
print("Complete categories:")
print(complete)
assert complete.loc["medium"] == 0 and complete.sum() == 35

Expected output

priority
low     15
high    20
Name: amount, dtype: int64
Complete categories:
priority
low       15
medium     0
high      20
Name: amount, dtype: int64

Common mistakes 🔝

Category codes are not measured quantities. Category order must match your data specification. Review unknown values before casting; adding categories is different from renaming them. Categorical storage can save memory for repeated labels but may cost more for high-cardinality data. Numeric IDs are not automatically useful categorical fields. See dtypes, select_dtypes() and groupby().

Exercise: a complete grade report 🔝

Using the original grades, define unordered categories a, b, c and d. Count each category, including d even though it has no records. Verify that counts reconcile with six input rows.

Exercise solution 🔝

Use a defined category schema and an explicit observed policy.

grade_type = pd.CategoricalDtype(["a", "b", "c", "d"])
answer = grades.assign(grade=grades["grade"].astype(grade_type))
counts = answer.groupby("grade", observed=False).size()
print(counts)
assert counts.tolist() == [1, 3, 2, 0] and counts.sum() == len(grades)

Expected output

grade
a    1
b    3
c    2
d    0
dtype: int64

Practice in Google Colab 🔝

Open in Google Colab View on GitHub

Run examples in order, inspect formatted dates and try the exercise. Save a copy to keep edits.

Reference: Pandas Categorical data documentation.

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author Watch on YouTube

Connect:

Subscribe to our YouTube Channel here