🔮 morphscrapper

A sophisticated, multi-modal web scraping and media conversion suite for Python.

📖 Overview

morphscrapper is a modular Python package designed to extract and convert diverse content types from the web. It bridges the gap between raw web scraping and advanced media processing, allowing developers to extract text, images, audio, and video resources, and immediately perform operations like OCR (Optical Character Recognition), speech recognition, and video manipulation.

It features a robust Python API, a full-featured Command-Line Interface (CLI), and a modern, light-themed Neo-Brutalism web dashboard for interactive scraping and conversion.

🛠️ Tech Stack & Dependencies

morphscrapper leverages modern, reliable Python libraries:

Layer	Technology	Icons / Badges
Core Scraping	requests, BeautifulSoup4
OCR Engine	pytesseract, Pillow (Tesseract OCR wrapper)
Speech-to-Text	SpeechRecognition, pydub
Video Processing	moviepy (FFmpeg-powered)
Web Dashboard	Flask, Vanilla HTML5, CSS3, JavaScript

✨ Features

• 📄 Document Text Scraping — Extracts clean, structural content from websites.
• 🖼️ Image Scraping & Filtering — Locates all image formats, with options to download specific extensions or package them into ZIPs on-demand.
• 🎵 Audio & Video Extraction — Finds direct media tracks and embeds.
• 🔤 Optical Character Recognition (OCR) — Extracts text from local or scraped image files using Tesseract OCR.
• 🎤 Speech Transcription — Transcribes WAV/MP3 files into readable text.
• 🎞️ Video Manipulation — Converts video clips to audio tracks, extracts frames at custom intervals, or transcribes speech within video tracks.
• 💻 Desktop CLI — Clean command parser with JSON, CSV, and TXT exporting formats.
• 🌐 Web UI — A modern web dashboard built around high-contrast Neo-Brutalism principles with drag-and-drop uploads.

🚀 Installation

Core Package (Basic Scraping)

pip install morphscrapper

With Optional Media Processing Features

# To install with OCR, Audio, and Video conversion dependencies:
pip install "morphscrapper[all]"

System Dependencies

For full conversion capabilities, make sure the system binaries are installed on your OS:

# Ubuntu/Debian
sudo apt update
sudo apt install -y tesseract-ocr ffmpeg

# macOS
brew install tesseract ffmpeg

💻 Usage & Quick Start

1. Python API

from morphscrapper import scrape_text, scrape_images, image_to_text

# Scrape document content from a website
result = scrape_text("https://example.com")
if result["success"]:
    print(f"Page Title: {result['title']}")
    print(result["text"])

# Scrape images
img_result = scrape_images("https://example.com")
print(f"Found {img_result['count']} images.")

# Perform local OCR (requires Tesseract)
ocr_text = image_to_text("receipt.png")
if ocr_text["success"]:
    print(ocr_text["text"])

2. Command-Line Interface (CLI)

# Scrape text from a URL and export as CSV
morph scrape text https://example.com --output csv

# Scrape and download images
morph scrape images https://example.com --download

# Extract audio track from an MP4 file
morph convert video-to-audio movie.mp4 --save-as audio.wav

# Start the Web GUI
morph web

🌐 Neo-Brutalism Web Dashboard

Run morph web or python -m morph.cli.main web to launch the local web server. The dashboard provides:

1. Interactive Scrapers: Instant scraping for text, images, audio, or video.
2. On-Demand Package Download: Download your scraping results as TXT, CSV, JSON, or package your images/audio/video files into a ZIP archive instantly.
3. Media Converters: Drag-and-drop any local file below the scraper to instantly convert images to text, audio to text, or videos to frames and audio tracks, featuring real-time visual media players and galleries.

📁 Repository Architecture

morphscrapper/
├── morph/                    # Main package source
│   ├── cli/                  # Command-line entry points
│   ├── core/                 # Settings and static configuration
│   ├── scrapers/             # Scraper scripts (text, image, audio, video)
│   ├── converters/           # Media converters (OCR, speech, video)
│   ├── utils/                # Downloaders, exporters, validators
│   └── web/                  # Web server backend and static assets
├── docs/                     # Full manuals (usage, install, API guides)
├── examples/                 # Code usage demonstrations
├── tests/                    # Unit testing suite
├── pyproject.toml            # Package configuration
├── setup.py                  # Package installer script
└── README.md                 # Project README

📄 License

This project is licensed under the MIT License. See the LICENSE file for details.