Extracting GitHub Repository Statistics into a Pandas DataFrame

To fetch live repository metrics (such as stars, forks, and open issues) from a public Representational State Transfer Application Programming Interface (REST API) and convert them directly into a Pandas DataFrame, use the Python requests library. Run this snippet instantly inside Google Colab:

import requests
import pandas as pd

# Fetch public repository details from GitHub REST API
url = "https://api.github.com/repos/pandas-dev/pandas"
headers = {"Accept": "application/vnd.github.v3+json"}
response = requests.get(url, headers=headers)

if response.status_code == 200:
    repo_data = response.json()
    
    # Extract key metrics into a DataFrame
    df = pd.DataFrame([{
        'Repository': repo_data['full_name'],
        'Stars': repo_data['stargazers_count'],
        'Forks': repo_data['forks_count'],
        'Open_Issues': repo_data['open_issues_count'],
        'Language': repo_data['language']
    }])
    
    print(df)
else:
    print(f"Error fetching GitHub data: {response.status_code}")

Prerequisites & Required Libraries Top ↑

Ensure you have the required libraries installed in your Python environment or Google Colab notebook:

!pip install --upgrade requests pandas

Flattening Nested Contributor or Release Data Top ↑

Endpoints returning lists of items (such as repository contributors or recent issues) require pd.json_normalize() to flatten nested JSON structures into tabular columns:

# Fetch top contributors for a repository
url = "https://api.github.com/repos/pandas-dev/pandas/contributors?per_page=5"
response = requests.get(url)

if response.status_code == 200:
    contributors = response.json()
    df_contrib = pd.json_normalize(contributors)
    print(df_contrib[['login', 'contributions']])

Handling Rate Limits & Headers Top ↑

GitHub imposes unauthenticated rate limits (typically 60 requests per hour). Always include appropriate User-Agent headers and handle response codes gracefully:

headers = {
    "User-Agent": "Plus2Net-Data-Tutorial",
    "Accept": "application/vnd.github.v3+json"
}
response = requests.get("https://api.github.com/rate_limit", headers=headers)
print(response.json())

Saving Repository Metrics (SQLite, CSV, Excel) Top ↑

Once your repository statistics are loaded into a DataFrame, you can persist them across multiple storage formats:

• SQLite Database: Use pandas to_sql().
• CSV Files: Export using pandas to_csv().
• Excel Workbooks: Save using pandas to_excel().

import sqlite3

# Save to CSV
df.to_csv("github_repo_stats.csv", index=False)

# Save to SQLite
conn = sqlite3.connect("github_data.db")
df.to_sql("repo_metrics", conn, if_exists="replace", index=False)
conn.close()
print("GitHub metrics successfully saved!")

Interactive Companion Jupyter Notebook

Explore live GitHub API extraction code cells, test rate-limit checks, and run self-validating data assertions in Google Colab.

View on GitHub

Frequently Asked Questions Top ↑

Related Python & Pandas Tutorials

• Pandas Input and Output Guide
• Extracting Live API Data into a Pandas DataFrame
• Extracting Live Weather Data into a Pandas DataFrame
• Getting Started with Google Colab Notebooks

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author   Watch on YouTube

Connect: