instagram-graphql-scraper

Collects posts from a public Instagram profile by capturing the browser's profile GraphQL request, then replaying subsequent pages through one requests.Session.

Installation

pip install -r requirements.txt
# or
pip install -e .

Usage

from instagram_graphql_scraper import InstagramGraphqlScraper

scraper = InstagramGraphqlScraper(driver_path="/path/to/chromedriver")
try:
    posts = scraper.get_user_posts("username", max_pages=2, max_posts=24)
finally:
    scraper.close()

The browser flow opens the profile, dismisses an optional login dialog, scrolls once, clears Selenium Wire requests, and clicks the dynamic show-more-posts button once. The captured request supplies the URL, headers, cookies, form payload, variables, and doc_id; no dynamic GraphQL values are hardcoded.

days_limit keeps posts newer than N days and stops pagination early once older posts are reached, using each post's accessibility_caption date; posts without a parseable date are kept. Combine with max_pages or max_posts for additional bounds.

For bounded asynchronous Embed enrichment (opt-in, not run automatically by get_user_posts):

import asyncio
from instagram_graphql_scraper import InstagramEmbedClient

async def enrich(posts):
    async with InstagramEmbedClient(max_concurrency=10) as client:
        return await client.fetch_many(posts)

details = asyncio.run(enrich(posts))

The batch client preserves input order, deduplicates shortcodes per run, and returns a per-post error result instead of failing the whole batch. The single-post normalized CLI remains available:

python3 instagram_context_json.py "https://www.instagram.com/p/SHORTCODE/embed/captioned/"
python3 instagram_context_json.py "https://www.instagram.com/p/SHORTCODE/embed/captioned/" --full

Tests

python3 -m pytest -q

The manual browser check is intentionally not part of CI:

python3 manual_integration.py 1989ivyshao --driver-path /path/to/chromedriver --max-pages 2

Documentation

See docs/ for the system architecture diagram, class diagram, and a bilingual (繁體中文 / English) SRE testing runbook covering environment setup, parameter boundaries, and known issues.

Do not place cookies, CSRF/LSD tokens, or captured payloads in source, fixtures, or logs.

License

MIT, see LICENSE.