chuscraper 0.16.4

A blazing fast, async-first, undetectable webscraping/web automation framework

🕷️ Chuscraper

LLM + CDP powered stealth-focused web scraping & automation framework
You Only Scrape Once — data extraction made smarter, faster, and more resilient.

---

🚀 What is Chuscraper?

Chuscraper is a Python web scraping & automation library that uses CDP (Chrome DevTools Protocol) and LLMs to extract structured data, interact with pages, and automate workflows — with a heavy focus on Anti-Detection and Stealth.

With AI-powered extraction, you tell it what to extract — it figures out how.

---

🌟 Features

🕵️‍♂️ Stealth & Anti-Detection

• Hides navigator.webdriver, user agent rotation
• Canvas/WebGL noise + hardware spoofing
• Timezone & geolocation spoofing

🤖 AI-Driven Data Extraction

• Semantic extraction using LLMs
• Converts HTML into structured JSON/Pydantic

🧠 Autonomous Navigation

• Intelligent pilot (ai_pilot) that clicks/types until goal achieved

⚡ Async + Fast

Built on async CDP, low overhead, no heavy browser bundles.

🔄 Flexible Outputs

Supports JSON, CSV, Markdown, Excel, Pydantic, and more.

🌐 Integrations

• LLM Providers: OpenAI, Gemini, Anthropic, Ollama
• Frameworks: LangChain, LlamaIndex, Agno, Crew.ai

---

📦 Installation

pip install chuscraper

# For AI Capabilities
pip install chuscraper[ai]

[!TIP] Use within a virtual environment to avoid conflicts.

---

💻 Quick Start (Async)

import asyncio
from chuscraper import start

async def main():
    browser = await start(headless=False)
    page = await browser.get("https://www.makemytrip.com/")
    # ... see full examples in examples/ directory

🔥 New: Check out our Real-World Scraper Examples for Amazon, Flipkart, Walmart, Google & more! page = await browser.get("https://www.makemytrip.com/")

# Tell the AI what to extract
print("AI is navigating...")
await page.ai_pilot("Search hotels in Goa for next weekend")

# Extract structured data
result = await page.ai_extract("Get the first 3 hotels with prices")
import json
print(json.dumps(result, indent=2))

await browser.stop()

if name == "main": asyncio.run(main())

---

## 🤖 AI Usage with Providers
Chuscraper supports multiple providers out-of-the-box.

### 1. Gemini (Native)
```python
from chuscraper.ai.providers import GeminiProvider
provider = GeminiProvider(api_key="YOUR_GEMINI_API_KEY")
await page.ai_extract("Extract data", provider=provider)

2. OpenAI

from chuscraper.ai.providers import OpenAIProvider
provider = OpenAIProvider(api_key="YOUR_OPENAI_API_KEY")
await page.ai_extract("Extract data", provider=provider)

🛡️ Stealth & Anti-Detection Proof

We don't just claim to be stealthy; we prove it. Below are the results from top anti-bot detection suites, all passed with 100% "Human" status.

👉 View Full Visual Proofs & Screenshots Here

Detection Suite	Result	Status
SannySoft	No WebDriver detected	✅ Pass
BrowserScan	100% Trust Score	✅ Pass
PixelScan	Consistent Fingerprint	✅ Pass
IPHey	Software Clean (Green)	✅ Pass
CreepJS	0% Stealth / 0% Headless	✅ Pass
Fingerprint.com	No Bot Detected	✅ Pass

🌍 Real-World Protection Bypass

We tested chuscraper against live websites protected by major security providers:

Provider	Target	Result
Cloudflare	Turnstile Demo	✅ Solved Automatically
DataDome	Antoine Vastel Research	✅ Accessed
Akamai	Nike Product Page	✅ Bypassed

---

📖 Documentation

Full technical guides are available in the docs/ folder:

• English (Main)
• Project API Guide
• Stealth Comparison

Translations (Chinese, Japanese, etc.) coming soon.

---

🛠️ Contributing

Want to contribute? Open an issue or send a pull request — all levels welcome! Please follow the CONTRIBUTING.md guidelines.

---

📜 License

Chuscraper is licensed under the MIT License.

Made with ❤️ by [Toufiq Qureshi]