Skip to main content

Local OpenAlex database with 284M+ works, abstracts, and semantic search

Project description

OpenAlex Local

Local OpenAlex database with 284M+ scholarly works, abstracts, and semantic search.

PyPI version Documentation Tests Python License

Why OpenAlex Local?

Built for the LLM era - features that matter for AI research assistants:

Feature Benefit
284M Works More coverage than CrossRef
Abstracts ~45-60% availability for semantic search
Concepts & Topics Built-in classification
Author Disambiguation Linked to institutions
Open Access Info OA status and URLs

Perfect for: RAG systems, research assistants, literature review automation.

Installation
pip install openalex-local

From source:

git clone https://github.com/ywatanabe1989/openalex-local
cd openalex-local && make install

Database setup (~300 GB, ~1-2 days to build):

# Check system status
make status

# 1. Download OpenAlex Works snapshot (~300GB)
make download-screen  # runs in background

# 2. Build SQLite database
make build-db

# 3. Build FTS5 index
make build-fts
Python API
from openalex_local import search, get, count

# Full-text search (title + abstract)
results = search("machine learning neural networks")
for work in results:
    print(f"{work.title} ({work.year})")
    print(f"  Abstract: {work.abstract[:200]}...")
    print(f"  Concepts: {[c['name'] for c in work.concepts]}")

# Get by OpenAlex ID or DOI
work = get("W2741809807")
work = get("10.1038/nature12373")

# Count matches
n = count("CRISPR")
CLI
openalex-local search "CRISPR genome editing" -n 5
openalex-local search-by-doi W2741809807
openalex-local search-by-doi 10.1038/nature12373
openalex-local status  # Configuration and database stats

With abstracts (-a flag):

$ openalex-local search "neural network" -n 1 -a

Found 1,523,847 matches in 45.2ms

1. Deep learning for neural networks (2015)
   OpenAlex ID: W2741809807
   Abstract: This paper presents a comprehensive overview of deep learning
   techniques for neural network architectures...
HTTP API

Start the FastAPI server:

openalex-local relay --host 0.0.0.0 --port 31292

Endpoints:

# Search works (FTS5)
curl "http://localhost:31292/works?q=CRISPR&limit=10"

# Get by ID or DOI
curl "http://localhost:31292/works/W2741809807"
curl "http://localhost:31292/works/10.1038/nature12373"

# Batch lookup
curl -X POST "http://localhost:31292/works/batch" \
  -H "Content-Type: application/json" \
  -d '{"ids": ["W2741809807", "10.1038/nature12373"]}'

# Database info
curl "http://localhost:31292/info"

HTTP mode (connect to running server):

# On local machine (if server is remote)
ssh -L 31292:127.0.0.1:31292 your-server

# Python client
from openalex_local import configure_http
configure_http("http://localhost:31292")

# Or via CLI
openalex-local --http search "CRISPR"
MCP Server

Run as MCP (Model Context Protocol) server:

openalex-local mcp start

Local MCP client configuration:

{
  "mcpServers": {
    "openalex-local": {
      "command": "openalex-local",
      "args": ["mcp", "start"],
      "env": {
        "OPENALEX_LOCAL_DB": "/path/to/openalex.db"
      }
    }
  }
}

Remote MCP via HTTP:

# On server: start persistent MCP server
openalex-local mcp start -t http --host 0.0.0.0 --port 8083
{
  "mcpServers": {
    "openalex-remote": {
      "url": "http://your-server:8083/mcp"
    }
  }
}

Diagnose setup:

openalex-local mcp doctor        # Check dependencies and database
openalex-local mcp list-tools    # Show available MCP tools
openalex-local mcp installation  # Show client config examples

Available tools:

  • search - Full-text search across 284M+ papers
  • search_by_id - Get paper by OpenAlex ID or DOI
  • enrich_ids - Batch lookup with metadata
  • status - Database statistics
Related Projects

crossref-local - Sister project with CrossRef data:

Feature crossref-local openalex-local
Works 167M 284M
Abstracts ~21% ~45-60%
Update frequency Real-time Monthly
DOI authority Yes (source) Uses CrossRef
Citations Raw references Linked works
Concepts/Topics No Yes
Author IDs No Yes
Best for DOI lookup, raw refs Semantic search

When to use CrossRef: Real-time DOI updates, raw reference parsing, authoritative metadata. When to use OpenAlex: Semantic search, citation analysis, topic discovery.

Documentation

Full documentation available at openalex-local.readthedocs.io

Data Source

Data from OpenAlex, an open catalog of scholarly works. Updated monthly from their snapshot.


SciTeX
AGPL-3.0 · ywatanabe@scitex.ai

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

openalex_local-0.3.1.tar.gz (43.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

openalex_local-0.3.1-py3-none-any.whl (46.0 kB view details)

Uploaded Python 3

File details

Details for the file openalex_local-0.3.1.tar.gz.

File metadata

  • Download URL: openalex_local-0.3.1.tar.gz
  • Upload date:
  • Size: 43.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.0rc1

File hashes

Hashes for openalex_local-0.3.1.tar.gz
Algorithm Hash digest
SHA256 0b3bf6ed1b450cb8a92e281e02efb68d5a4b37293c8d6a76ae260e1439fb4c2a
MD5 838b976f8c87ed978d033e1fe00dbe32
BLAKE2b-256 deb76693280d15db79b88737b672d19f08f607f100daeb59b1842bdd9d87f4ac

See more details on using hashes here.

File details

Details for the file openalex_local-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: openalex_local-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 46.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.0rc1

File hashes

Hashes for openalex_local-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 ca766b2ddf09d3d3eb5fe9c7b3442c02910aa1c2b1efe570e78abad135c2fd13
MD5 295fbd93c11ced05a10fb8a4513d552f
BLAKE2b-256 d71bd7d0c5f9e8d0de849e8ce6fa3c745acafd20a5584b3427c887b6d8ea68c2

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page