Skip to main content

A Python package to collect and wrangle YouTube video and channel statistics using the YouTube Data API v3

Project description

yt-stats-wrangler

A flexible and easy-to-use Python package to collect and wrangle YouTube video and channel statistics using the YouTube Data API v3.

Built with extensibility and usability in mind, yt-stats-wrangler supports a range of outputs (raw JSON, pandas, polars) and offers features for analyzing video metadata, statistics, and comments.

You'll need a developer key for the YouTube API V3 to use this package. To generate an API key for your google developer account, please see the official YouTube API v3 documentation for more information.

Github repository for the project can be found here

PyPI page for the project can be found here


Features

  • Gather public video metadata for one or more YouTube channels
  • Collect video statistics (views, likes, comments, etc.)
  • Retrieve top-level comments from videos
  • Output to multiple formats: raw, pandas, polars
  • Flexible column formatting: raw, lower_case, UPPER_CASE, mixedCase
  • Quota tracking support for efficient API usage
  • Optional DataFrame utilities (reordering, parsing datetime)

More coming soon!


Installation

pip install yt-stats-wrangler

yt-stats-wrangler is designed to work independent of any data libraries in python for users who prefer a lightweight solution. pandas, polars are treated as optional dependencies. To use this package with those libraries, ensure they are installed in your enviornment or specify their installation like below:

To use with pandas:

pip install yt-stats-wrangler[pandas]

To use with polars:

pip install yt-stats-wrangler[polars]

Quick Start

The following example is a quick start in gathering all videos present on the CDCodes YouTube channel, and outputting the results into a pandas dataframe with formatted column names.

from yt_stats_wrangler.api.client import YouTubeDataClient
import os

# Get your API key (recommended to store as environment variable)
api_key = os.getenv("YOUTUBE_API_V3_KEY")
client = YouTubeDataClient(api_key=api_key, max_quota=10000) # set to -1 for unlimited

# Get all videos from a channel
videos = client.get_all_video_details_for_channel(
    channel_id="UCB2mKxxXPK3X8SJkAc-db3A",
    key_format="upper",        # Format dictionary keys
    output_format="pandas"     # Output as pandas DataFrame
)

print(videos.head())

Checkout example_notebooks for more ways to use the package


Quota Exhaustion Behavior

When a configured key returns 403 quotaExceeded, the wrangler:

  1. Marks that key as fully spent in its in-memory counter.
  2. Attempts to rotate to a sibling key (if multiple were provided to __init__).
  3. Raises QuotaExceededError from the failing call.

QuotaExceededError exposes keys_remaining: bool so iterating methods can decide whether to keep going on the new key or abort. Callers using single-call methods (get_channel_id_from_handle_v2, get_channel_statistics, etc.) should catch QuotaExceededError directly.

Importable from yt_stats_wrangler.api.client:

from yt_stats_wrangler.api.client import QuotaExceededError

Supported Methods

YouTubeDataClient

Method Description Estimated Quota Cost
get_channel_id_from_handle(handle) Get a channel's ID from a YouTube handle (e.g. '@cdcodes') 100
get_channel_ids_from_handles(handles) Get multiple channel IDs from a list of YouTube handles 100 per handle
get_channel_statistics(channel_id) Get high-level stats for a single channel (subscribers, total views, total posts) 1
get_channel_statistics_for_channels(channel_ids) Get high-level stats for multiple channels 1 per channel
get_all_video_details_for_channel(channel_id) Fetch video metadata for a single channel 1 per 50 videos
get_all_video_details_for_channels(channel_ids) Fetch video metadata for multiple channels 1 per 50 videos, per channel
get_video_stats(video_ids) Get public statistics for one or more videos 1 per 50 video IDs
get_top_level_video_comments(video_id) Get top-level comments for a video 1 per 100 comments page
get_top_level_comments_for_video_ids(video_ids) Get top-level comments for multiple videos 1 per 100 comments page, per video
get_all_video_comments(video_id) Get all comments (top-level + replies) for a video 1 per 100 top-level comments + 1 per 100 replies
get_all_comments_for_video_ids(video_ids) Get all comments (top-level + replies) for multiple videos Varies by number of videos and replies

Output Formats

You can return data in one of the following formats:

  • raw: List of dictionaries (default)
  • pandas: Requires optional pandas dependency
  • polars: Requires optional polars dependency
  • pyspark : Requires optional polars dependency, implementation available but not thoroughly tested as of v0.2.0

Key and Column Formatting

For user-friendly keys and columns, pass the key_format argument:

  • raw: Keep keys as-is from the API (default)
  • lower: lowercase with underscores to separate words
  • upper: UPPERCASE with underscores to separate words
  • mixed: camel_Case with underscores to separate words

Development & Contributing

Please see the contribution documentation for best practice on contributing to the package.

See the full CHANGELOG.md for a list of updates and changes.

License

This project has an MIT license. You're welcome to build on this package, integrate it into your own data pipelines, or extend it for custom use cases (e.g., adding other output formats or integrating it into production systems). Just make sure to preserve the license file and give credit in your derived works. It is open source, open to new contributors and can be used for any purpose (personal, academic, commercial, etc.)

Author

Created and maintained by Christian Dueñas
GitHub: @ChristianD37

Additonal Contributors: Coming Soon!


Tutorial and Example Notebooks

Check out examples of the package at work in the example_notebooks directory.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

yt_stats_wrangler-0.4.0.tar.gz (22.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

yt_stats_wrangler-0.4.0-py3-none-any.whl (14.5 kB view details)

Uploaded Python 3

File details

Details for the file yt_stats_wrangler-0.4.0.tar.gz.

File metadata

  • Download URL: yt_stats_wrangler-0.4.0.tar.gz
  • Upload date:
  • Size: 22.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.15

File hashes

Hashes for yt_stats_wrangler-0.4.0.tar.gz
Algorithm Hash digest
SHA256 40a22ba5bfd7e4d7e6c0e2a798c02d5d81c9103fde36699a199d9ff0c72c6e53
MD5 5d527c95b921a1170317fe01075fe13e
BLAKE2b-256 d6595e022632e5a6ba99e82e768b829d68d1ff67683c43568c6696f8e754500d

See more details on using hashes here.

File details

Details for the file yt_stats_wrangler-0.4.0-py3-none-any.whl.

File metadata

File hashes

Hashes for yt_stats_wrangler-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 25e7eb8f308b28d0dc018e5e6fd975465a4158130fdb51b183cb553589e9aa6c
MD5 5e3e8b98c66d31e8a050c52f8879c6f0
BLAKE2b-256 02f4c172ddc6a2a9cf575ae523204354dde3e39c85c6fd15ada4fbf060e62df7

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page