ChinaCrawl

The Firecrawl alternative that works in China. 12-in-1 web data engine — zero external API cost.

中国版 Firecrawl。12 合 1 Web 数据引擎 — 零外部付费依赖。

Why ChinaCrawl?

Firecrawl is the best web scraping engine from Silicon Valley. But it's blocked in China and has no Chinese search optimization. ChinaCrawl fills this gap.

Capabilities (12-in-1)

extract-llm uses local Ollama (qwen2.5:7b) for semantic extraction. Zero API cost, unlimited calls.

Quick Start

Install

# Basic (scrape / search / map / crawl / download / monitor)
pip install chinacrawl

# Full (includes Playwright for browser interaction)
pip install chinacrawl[full]

# LLM extraction (optional, requires Ollama)
ollama pull qwen2.5:7b

3 Lines to Start

from chinacrawl import scrape, search_web, extract_llm

# Any URL -> Markdown
result = scrape("https://example.com")
print(result.content[:200])

# Web search
results = search_web("Python web scraping", max_results=5)
for r in results:
    print(f"{r.title}: {r.url}")

# LLM-powered structured extraction (zero cost!)
data = extract_llm("https://news.ycombinator.com", {
    "top_story": "h1",
    "all_headlines": "a.storylink"
})
print(data.data)

CLI

python chinacrawl.py  # Run built-in 11-capability test suite

Architecture

ChinaCrawl
├── Scrape:   jina.ai + trafilatura  ->  Markdown
├── Search:   SearXNG (self-hosted or public)
├── Map:      trafilatura link discovery
├── Crawl:    BFS recursive extraction
├── Download: Map + Scrape -> local {url}.md files
├── Monitor:  Hash + AI Judge (noise-stripped)
├── Extract:  CSS selector + Ollama qwen (semantic)
└── Interact: Playwright browser automation

Use Cases

• Financial data — batch scrape central bank / SEC announcements
• E-commerce intel — monitor competitor pricing on JD / Tmall
• Legal compliance — archive government regulations with version diff
• Bid monitoring — real-time government procurement alerts (AI de-noised)
• RAG pipelines — any website -> offline markdown -> Dify / LangChain

Disclaimer

ChinaCrawl is a data extraction tool, NOT an AI service.

IMPORTANT: Users assume all legal responsibility. ChinaCrawl provides general-purpose data collection capabilities. Users must:

• Respect target websites' robots.txt and terms of service
• Comply with applicable laws (including China's Cybersecurity Law, Data Security Law, Personal Information Protection Law)
• Obtain necessary permissions and consent
• Not use this tool for any illegal purpose

ChinaCrawl authors and contributors bear no legal liability for users' actions.

The extract-llm feature depends on user-installed Ollama models. Users must ensure their chosen model's license (Apache 2.0, MIT, Llama Community, etc.) permits their intended use. ChinaCrawl defaults to qwen2.5:7b (Apache 2.0, no restrictions).

Respect robots.txt and terms of service. For legitimate use only.

License

GNU Affero General Public License v3.0 — Free to use, modify, and distribute. Modified versions served over a network must disclose source code.

Roadmap

• 11-in-1 core capabilities
• SearXNG search integration
• AI Judge noise filtering
• Chinese search engine config (Baidu / Sogou)
• PyPI release
• Docker one-click deployment
• Managed API (chinacrawl.cloud)

_{Built by Xiao Hei · ChinaCrawl · 2026}