scrapeMM 0.1.0

LLM-friendly scraper for media and text from social media and the open web.

scrapeMM: Multimodal Web Retrieval

Simple web scraper to asynchronously retrieve webpages and access social media contents, fetching text along with media, i.e., images and videos.

This library aims to help developers and researchers to easily access multimodal data from the web and use it for LLM processing.

Usage

from scrapemm import retrieve

url = "https://example.com"
result = retrieve(url)
result.render()

How it works

Input:                                  Output:
URL (string)   -->   retrieve()   -->   MultimodalSequence

The MultimodalSequence is a sequence of Markdown-formatted text and media provided by the ezMM library.

Web scraping is done with Firecrawl.

Supported Proprietary APIs

• ✅ X/Twitter
• ✅ Telegram
• ⏳ Facebook
• ⏳ Instagram
• ⏳ Threads
• ⏳ TikTok