crawlix 0.1.0

One API. Any backend. Full browser automation to lightweight scraping.

crawlix

One API. Any backend. Full browser automation to lightweight scraping.

PyPI: pip install crawlix Author: keylordelrey License: MIT Python: 3.10+

---

What crawlix is

crawlix is a Python browser automation and web scraping library with a unified API across multiple backends. The same code works whether you are doing simple HTTP scraping or full Playwright-powered browser automation — you switch backends, not code.

from crawlix import Browser

with Browser() as b:
    page = b.open("https://example.com")
    print(page.find("h1").text)

with Browser(backend="playwright") as b:
    page = b.open("https://example.com")
    page.click("#login")
    page.type("#email", "[e-mail]")
    page.submit("form")
    page.wait_for(".dashboard")
    page.screenshot("result.png")

---

Install

pip install crawlix
pip install crawlix[playwright]
pip install crawlix[selenium]
pip install crawlix[async]
pip install crawlix[full]

---

Core Design Rules

1. Same API across all backends — switching backend never requires rewriting user code
2. Auto-detect best available backend — no config needed, crawlix figures it out
3. Zero hard dependencies — pip install crawlix always succeeds
4. Fail with helpful errors — BackendError tells you exactly what to install
5. Context manager always — resources always cleaned up properly
6. Stealth on by default — realistic headers, UA rotation, no bot fingerprint

---

Backend Priority

playwright > selenium > requests+bs4 (core)

Override anytime:

Browser(backend="playwright")
Browser(backend="requests")
Browser(backend="selenium")

---

Quick Examples

from crawlix import Browser, get, fetch

with Browser() as b:
    page = b.open("https://news.ycombinator.com")
    for item in page.find_all(".titleline > a"):
        print(item.text, item.attr("href"))

data = get("https://api.github.com/users/keyreyla").json()
html = fetch("https://example.com")

For async:

import asyncio
from crawlix.async_api import AsyncBrowser

async def main():
    async with AsyncBrowser() as b:
        page = await b.open("https://example.com")
        print(page.html)

asyncio.run(main())

---

API Overview

Browser

Browser(backend="auto", headless=True, stealth=True, timeout=30, proxy=None, locale="en-US", user_agent=None)
b.open(url) -> Page
b.new_page() -> Page
b.close()
b.backend_name -> str
b.supports_js -> bool

Page (all methods return self for chaining)

page.find(selector) -> Element | None
page.find_all(selector) -> list[Element]
page.click(selector) -> Page
page.type(selector, text) -> Page
page.screenshot(path=None) -> bytes
page.html -> str
page.text -> str
page.json() -> dict
page.links() -> list[str]

Element

el.text -> str
el.attr(name) -> str
el.attrs -> dict
el.find(selector) -> Element | None
el.click() -> Element
bool(el)  # always True

---

Exceptions

from crawlix.exceptions import CrawlixError, BackendError, TimeoutError, NavigationError, SelectorError, NetworkError, JavaScriptError

---

Development

git clone https://github.com/keyreyla/crawlix.git
cd crawlix
pip install -e ".[full]"
pytest