mcmaster-scraper 0.2.1

Fetch product tables from a McMaster-Carr URL as a DataFrame for complex filtering and calculations

McMaster-Scraper

A Python library for fetching product tables from a McMaster-Carr URL as a DataFrame for complex filtering and calculations.

Features

• Caches data locally to speed up future calls
• Supports both sync/async APIs
• Works in Python files and Jupyter notebooks
• Includes convenience functions to quickly retrieve product tables from multiple URLs
• Typed functions for type-checking compatibility

Install

McMaster-Scraper is available on PyPi:

pip install mcmaster-scraper

McMaster-Scraper requires Playwright to fetch the product tables. It is already included as a dependency. However, you will need to install the browsers manually:

playwright install

Quick Start

To use the Sync API, import the sync_api module and call get_products_from_url(s):

from mcmaster_scraper.sync_api import get_products_from_url

url = "https://www.mcmaster.com/products/screws/socket-head-screws-2~/steel-socket-head-screws~~/"
data = get_products_from_url(url) # Returns a DataFrame with all the products from the URL

... # Do stuff with the DataFrame (filter, perform calculations, etc.)

Using the Async API is similar, import the async_api module and await the function call:

from mcmaster_scraper.async_api import get_products_from_url

url = "https://www.mcmaster.com/products/screws/socket-head-screws-2~/steel-socket-head-screws~~/"
data = await get_products_from_url(url) # Returns a DataFrame with all the products from the URL

... # Do stuff with the DataFrame (filter, perform calculations, etc.)

Docs

API Reference

The API reference can be found on GitHub Pages.

Examples

An example script can be found in docs/example.py.

Known Limitations

Some pages generate server-side HTML rather than client-side rendering a JSON response. These pages are currently unsupported but may be supported in the future. UnsupportedOperation will be raised for such pages.

Disclaimer

This library is for responsible data extraction only. Do not:

• Scrape beyond reasonable rates
• Violate Terms of Service
• Circumvent access controls
• Use data for unauthorized commercial purposes

Legal Notice

This library is provided as-is. Authors are not liable for any legal, technical, or business consequences resulting from misuse of this library. Users assume full responsibility for compliance with applicable laws, regulations, and website policies.

By using this library, you acknowledge and agree to these responsibilities.

License

MIT