Lexoid turns messy documents and live websites into agent-ready context.

• parse(): High-accuracy document and webpage parsing across static, local LLM, and frontier LLM pipelines.
• browse() (preview): Goal-driven, evidence-grounded browser automation for live web workflows.

Documentation

Why Lexoid?

Useful information is buried in documents and scattered across websites. Getting it out shouldn't require expensive a custom pipeline for every task.

• Extract what matters: Turn documents and web pages into usable text and structured data for downstream workflows.
• Automate the search, retain the evidence: Automate browser tasks with evidence.
• Choose your approach: Use static parsing, hosted LLMs, or supported local models.
• Build openly: Collaborate under the Apache 2.0 license.

Installation

Installing with pip

pip install lexoid

To use LLM-based parsing, define the following environment variables or create a .env file with the following definitions

OPENAI_API_KEY=""
GOOGLE_API_KEY=""

For local inference with Ollama, no API key is required. Install Ollama, pull the target model, and keep the local server running:

ollama pull gemma4
export OLLAMA_BASE_URL=127.0.0.1:11434
ollama list
ollama serve

# docker
Reference: https://docs.ollama.com/docker#run-model-locally
CPU example (will most likely be slower; remember to adjust `OLLAMA_TIMEOUT` as needed)
- docker run -d -v ollama:/root/.ollama -p 11434:11434 -e OLLAMA_BASE_URL=0.0.0.0 -e OLLAMA_TIMEOUT=240 --name ollama ollama/ollama
- docker exec -it ollama ollama pull gemma4:latest

For webpage retrieval and URL-to-PDF conversion with parse(), install Playwright's Chromium browser binaries. The Playwright Python package is installed with Lexoid:

playwright install --with-deps --only-shell chromium

To use the autonomous browser tasks shown below with browse(), install the optional agent dependencies and Patchright's Chromium binaries:

pip install "lexoid[browse]"
patchright install --with-deps chromium

Building .whl from source

make build

Creating a local installation

To install dependencies:

make install

or, to install with dev-dependencies:

make dev

To activate virtual environment:

source .venv/bin/activate

Usage

Example Notebook

Example Colab Notebook

Here's a quick example to parse documents using Lexoid:

from lexoid.api import parse

parsed_md = parse("https://www.justice.gov/eoir/immigration-law-advisor", parser_type="AUTO")["raw"]
# or
pdf_path = "path/to/immigration-law-advisor.pdf"
parsed_md = parse(pdf_path, parser_type="LLM_PARSE")["raw"]
# or
pdf_path = "path/to/immigration-law-advisor.pdf"
parsed_md = parse(pdf_path, parser_type="STATIC_PARSE")["raw"]

print(parsed_md)

Parameters

• path (str): The file path or URL.
• parser_type (str, optional): The type of parser to use ("LLM_PARSE" or "STATIC_PARSE"). Defaults to "AUTO".
• pages_per_split (int, optional): Number of pages per split for chunking. Defaults to 4.
• max_threads (int, optional): Maximum number of threads for parallel processing. Defaults to 4.
• **kwargs: Additional arguments for the parser.

Web Pages and Browser Tasks

Use parse() when you have a URL to retrieve and parse. Set as_pdf=True to render the page to PDF before parsing; pass save_dir to keep the generated PDF after the call returns:

from lexoid.api import parse

# Retrieve and parse a webpage
page = parse("https://example.com")

# Render the webpage to PDF, then parse the PDF
page_pdf = parse(
    "https://example.com",
    as_pdf=True,
    save_dir="output/",
    save_filename="example.pdf",
)
print(page_pdf["pdf_path"])

Web Operator (Preview): for browser tasks

Use browse() when the task requires goal-driven interaction with a website, including searching, applying filters, and completing multi-step non-binding quote, estimate, eligibility, or lookup forms with values supplied in the request. It returns an evidence-backed answer; it does not convert the source URL to PDF:

import asyncio
from lexoid.api import browse

async def main():
    result = await browse(
        "Visit https://example.com and find the current enterprise plan price."
    )
    print(result.answer)

asyncio.run(main())

Note: The navigator is instructed not to log in, create accounts, accept legal agreements, purchase, bind coverage, pay, or book, and to stop when required information, authentication, or consequential consent is needed. These are navigator instructions, not semantic enforcement for every site control; ambiguous actions should be reviewed. Popups are handled by purpose: promotional overlays can be dismissed, and cookie banners use reject or necessary-only options when available. CAPTCHA solving is planned; until it is implemented, a CAPTCHA that prevents progress is reported as a blocker.

Command Line Usage

Lexoid provides a command-line interface for document parsing without writing Python code.

Installation

The CLI is automatically available after installing Lexoid:

pip install lexoid
lexoid --help

Alternatively, use with Python module syntax:

python -m lexoid --help

Parse Documents

Convert documents to markdown or JSON:

# Parse to stdout (default markdown)
lexoid parse --input document.pdf

# Save to file
lexoid parse --input document.pdf --output output.md

# Output as JSON (includes metadata, segments, token usage)
lexoid parse --input document.pdf --format json --output result.json

# Use specific parser (STATIC_PARSE, LLM_PARSE, or AUTO)
lexoid parse --input document.pdf --parser-type STATIC_PARSE

# Use specific LLM model
lexoid parse --input document.pdf --model gpt-4o

# Enable verbose logging
lexoid parse --input document.pdf --verbose

Extract Structured Data with JSON Schema

Extract data conforming to a JSON schema:

# Inline schema
lexoid schema \
  --input document.pdf \
  --schema '{"type": "object", "properties": {"title": {"type": "string"}}}' \
  --output result.json

# Schema from file
lexoid schema \
  --input document.pdf \
  --schema schema.json \
  --output result.json

# Specify LLM provider
lexoid schema \
  --input document.pdf \
  --schema schema.json \
  --api openai \
  --model gpt-4o

Convert to LaTeX

Convert documents to LaTeX format:

# Convert to stdout
lexoid latex --input document.pdf

# Save to file
lexoid latex --input document.pdf --output output.tex

# Use specific model
lexoid latex --input document.pdf --model gpt-4o

Command-line Options

Common Options

• --input, -i: Input file path (required) - Supports PDF, images, HTML, DOCX, XLSX, PPTX, or URLs
• --output, -o: Output file path (optional) - If not specified, output is printed to stdout
• --verbose, -v: Enable detailed logging

Parse Command

lexoid parse --help

• --parser-type, -p: Parser type - AUTO (default), LLM_PARSE, or STATIC_PARSE
• --model, -m: LLM model name (default: gemini-3.5-flash-lite, thinking: medium)
• --pages-per-split: Pages per chunk (default: 4)
• --max-processes: Parallel processes (default: 4)
• --framework: Static parser framework - pdfplumber or paddleocr
• --format: Output format - markdown (default, plain markdown text) or json (full result with metadata, segments, token usage)

Schema Command

lexoid schema --help

• --schema, -s: JSON schema (file path or inline JSON, required)
• --model, -m: LLM model (default: gpt-4o-mini)
• --api: API provider - openai, gemini, anthropic, ollama, etc. (auto-detected if not specified)
• --example-schema: Provide example data for the schema
• --fill-single-schema: Auto-fill single schemas

LaTeX Command

lexoid latex --help

• --model, -m: LLM model (default: gpt-4o-mini)
• --api: API provider - openai, gemini, anthropic, ollama, etc. (auto-detected if not specified)

Supported API Providers

• Google
• OpenAI
• Hugging Face
• Together AI
• OpenRouter
• Fireworks
• Ollama

Ollama Local Parsing

Lexoid supports local LLM_PARSE inference through Ollama. The initial recommended model is gemma4:latest.

from lexoid.api import parse

result = parse(
    "path/to/document.pdf",
    parser_type="LLM_PARSE",
    api_provider="ollama",
    model="gemma4:latest",
    max_processes=1,
)

print(result["raw"])

Notes:

• Ollama uses the default local endpoint http://localhost:11434 unless OLLAMA_BASE_URL is set.
• Lexoid forces max_processes=1 for Ollama-backed parsing to avoid local multiprocess contention.
• AUTO routing does not select Ollama in this first version; choose it explicitly with api_provider="ollama".

Benchmark

Results aggregated across 14 documents.

Note: Benchmarks are currently done in the zero-shot setting.

Rank	Model	SequenceMatcher Similarity	Cosine- Similarity	Time (s)	Cost ($)
1	gemini-3.5-flash-lite (thinking: medium)	0.959 (±0.071)	0.986 (±0.018)	9.84	0.00659
2	claude-sonnet-5-5 (thinking: low)	0.956 (±0.084)	0.991 (±0.010)	6.85	0.01478
3	claude-opus-5-5 (thinking: low)	0.952 (±0.088)	0.983 (±0.022)	9.13	0.02805
4	claude-haiku-5-5 (thinking: medium)	0.949 (±0.074)	0.980 (±0.025)	7.96	0.00121
5	gpt-6.1-sol (thinking: low)	0.947 (±0.100)	0.986 (±0.014)	31.41	0.02349
6	gemini-3.8-flash (thinking: low)	0.932 (±0.117)	0.975 (±0.032)	20.73
7	gpt-6-astra (thinking: medium)	0.931 (±0.125)	0.986 (±0.014)	30.83	0.14173
8	AUTO + gemini-3.5-flash-lite (thinking: medium)	0.928 (±0.112)	0.989 (±0.015)	8.93	0.005315
9	AUTO + claude-haiku-5-5 (thinking: medium)	0.925 (±0.086)	0.982 (±0.023)	8.05	0.001215
10	gemini-4-argon (thinking: high)	0.890 (±0.229)	0.969 (±0.071)	69.63
11	gpt-5.2	0.890 (±0.193)	0.975 (±0.036)	33.32	0.03959
12	mistral-ocr-latest	0.882 (±0.106)	0.932 (±0.091)	5.75	0.00121
13	gpt-5.5	0.874 (±0.209)	0.939 (±0.138)	72.11	0.14495
14	gpt-6-luna (thinking: low)	0.869 (±0.146)	0.953 (±0.053)	13.35	0.00096
15	claude-3-5-sonnet-20241022	0.858 (±0.184)	0.930 (±0.098)	17.32	0.01804
16	gpt-5.4-mini	0.835 (±0.210)	0.948 (±0.066)	13.14	0.00902
17	gpt-5-mini	0.819 (±0.201)	0.917 (±0.104)	52.84	0.00811
18	gpt-5	0.807 (±0.215)	0.919 (±0.088)	98.12	0.05505
19	gpt-5.4	0.803 (±0.238)	0.936 (±0.150)	31.98	0.03887
20	claude-sonnet-4-20250514	0.801 (±0.188)	0.905 (±0.136)	22.02	0.02056
21	claude-opus-4-20250514	0.789 (±0.184)	0.886 (±0.148)	29.55	0.09513
22	accounts/fireworks/models/llama4-maverick-instruct-basic	0.772 (±0.203)	0.930 (±0.117)	16.02	0.00147
23	gemini-3-flash-preview	0.766 (±0.293)	0.858 (±0.210)	39.38	0.00969
24	claude-opus-4-8	0.764 (±0.254)	0.863 (±0.154)	11.10	0.03195
25	claude-sonnet-4-6	0.757 (±0.302)	0.843 (±0.206)	16.50	0.01804
26	gpt-4.1-mini	0.754 (±0.249)	0.803 (±0.193)	23.28	0.00347
27	accounts/fireworks/models/llama4-scout-instruct-basic	0.754 (±0.243)	0.942 (±0.063)	13.36	0.00087
28	gpt-4o	0.752 (±0.269)	0.896 (±0.123)	28.87	0.01469
29	gpt-4o-mini	0.728 (±0.241)	0.850 (±0.128)	18.96	0.00609
30	claude-haiku-4-5-20251001	0.683 (±0.300)	0.841 (±0.187)	7.86	0.00504
31	claude-3-7-sonnet-20250219	0.646 (±0.397)	0.758 (±0.297)	57.96	0.01730
32	gpt-4.1	0.637 (±0.301)	0.787 (±0.185)	35.37	0.01498
33	google/gemma-3-27b-it	0.604 (±0.342)	0.788 (±0.297)	23.16	0.00020
34	ds4sd/SmolDocling-256M-preview	0.603 (±0.292)	0.705 (±0.262)	507.74	0.00000
35	gpt-5.4-nano	0.600 (±0.309)	0.856 (±0.119)	22.51	0.00321
36	microsoft/phi-4-multimodal-instruct	0.589 (±0.273)	0.820 (±0.197)	14.00	0.00045
37	qwen/qwen-2.5-vl-7b-instruct	0.498 (±0.378)	0.630 (±0.445)	14.73	0.00056

Citation

If you use Lexoid in production or publications, please cite accordingly and acknowledge usage. We appreciate the support 🙏