SciTeX Scholar (scitex-scholar)

Scientific paper search, enrichment, PDF download, and library management for reproducible research.

Full Documentation · uv pip install scitex-scholar[all]

Problem and Solution

Problem

Literature management spans many tools and APIs: searching databases, resolving DOIs, downloading PDFs through institutional access, enriching BibTeX metadata, and keeping a reproducible, deduplicated library. Each step speaks a different library, auth flow, and data format.

Solution

scitex-scholar provides a unified workflow:

• Search across CrossRef, Semantic Scholar, PubMed, arXiv, and OpenAlex
• Resolve DOIs from titles; enrich BibTeX with abstracts, citation counts, impact factors (JCR 2024), PMIDs, and arXiv IDs
• Download PDFs through institutional access (OpenAthens / SSO) with Playwright browser automation
• Organize papers in a primary-hash library with per-project symlinks at ~/.scitex/scholar/library/. One-button maintenance via library refresh (reconcile → regenerate readable names → optional rsync to remote hosts)
• Highlight each sentence of a PDF by rhetorical role — claim, method, limitation, supportive citation, contradicting citation — via Claude
• Automate the same operations from the CLI, a Python API, or the SciTeX MCP server

Demo

flowchart LR
    Q["search query<br/>or .bib / DOI"] --> S["Scholar.search() /<br/>bibtex import"]
    S --> E["metadata_engines<br/>(CrossRef · OpenAlex · arXiv ·<br/>PubMed · Semantic Scholar)"]
    E --> P["Papers<br/>(deduped, enriched)"]
    P --> U["url_finder<br/>(translators + heuristics)"]
    U --> A["auth<br/>(OpenAthens / EZProxy /<br/>Shibboleth)"]
    A --> D["pdf_download<br/>(Playwright strategies)"]
    D --> L["~/.scitex/scholar/library/<br/>MASTER/&lt;HASH&gt;/"]
    P -.-> H["pdf_highlight<br/>(claim / method / limitation)"]
    L -.-> M["MCP: scholar_* tools<br/>(scitex serve)"]

_{Figure 1. End-to-end demo flow: one paper fetch call runs enrich, URL resolution, authentication, download, and storage into the primary-store library.}

scitex-scholar paper fetch --doi 10.1038/... exercises the full chain in one call: enrich → resolve URL → authenticate → download → store under MASTER/<HASH>/ with metadata + PDF + per-project symlinks.

Installation

uv pip install "scitex-scholar[all]"

Architecture

flowchart TB
    CLI["scitex-scholar CLI<br/>(noun-verb groups)"] --> CORE["core/<br/>Scholar · Paper · Papers"]
    MCP["MCP tools<br/>scholar_* via scitex serve"] --> CORE
    GUI["Django GUI<br/>_django/"] --> CORE
    CORE --> SEARCH["search_engines<br/>CrossRef · OpenAlex · arXiv ·<br/>PubMed · Semantic Scholar"]
    CORE --> META["metadata_engines<br/>DOI · abstracts · citations · IF"]
    SEARCH --> META
    META --> URL["url_finder + auth<br/>translators · OpenAthens / SSO"]
    URL --> PDF["pdf_download<br/>Playwright strategies"]
    PDF --> STORE["storage/<br/>primary store + project symlinks"]
    STORE --> HL["pdf_highlight<br/>claim / method / limitation"]
    STORE --> INT["integration/<br/>Zotero · Mendeley"]

_{Figure 2. Module architecture: the CLI, MCP, and GUI surfaces drive core/; federated search and enrichment feed URL discovery and authenticated download into the primary-store library.}

The CLI is a thin layer over the Python API: every scitex-scholar <noun> <verb> command dispatches into one of core/, pipelines/, storage/, or auth/. The MCP server (_mcp/) exposes the same handlers as scholar_* tools consumed by the unified scitex serve server.

4 Interfaces

from scitex_scholar import Scholar, Paper, Papers, apply_filters, to_bibtex

scholar = Scholar()
papers = scholar.search("deep learning EEG", year_min=2020)   # auto-enriched
papers.save("results.bib")

# Filter + export
top = apply_filters(papers, min_citations=50, min_impact_factor=5.0)
print(to_bibtex(top))

# Discover everything
scitex-scholar --help
scitex-scholar --help-recursive          # full overview, every leaf
scitex-scholar --version                  # or -V

# Paper(s)
scitex-scholar paper fetch --doi 10.1038/nature12373 --project demo
scitex-scholar paper fetch-batch --dois 10.1038/xxx --dois 10.1126/yyy --project demo --num-workers 4

# BibTeX file
scitex-scholar bibtex import --bibtex refs.bib --project demo --output refs.enriched.bib

# PDF post-processing
scitex-scholar pdf highlight paper.pdf

# Library — daily workflow
scitex-scholar library list                                   # all projects
scitex-scholar library list neurovista                        # one project, per-paper
scitex-scholar library open-urls neurovista --watch           # browser + auto-import
scitex-scholar library refresh neurovista                     # one-button maintenance
scitex-scholar library refresh neurovista --sync spartan      # +rsync push (repeatable)

# Library — manual PDF import (when DBs haven't indexed yet, or no auto-download)
scitex-scholar paper fetch --doi 10.1002/epi.70076 \
    --pdf-main ~/Downloads/Liu_2026.pdf \
    --pdf-supple ~/Downloads/MOESM1_ESM.pdf \
    --attachment ~/Downloads/dataset.csv \
    --project neurovista

# Library — layout / share / integrity
scitex-scholar library bind neurovista ~/proj/neurovista      # one symlink, no data move
scitex-scholar library export neurovista --format bibtex
scitex-scholar library audit-files --project neurovista
scitex-scholar library db build --dry-run
scitex-scholar library db audit --json

# Auth (institutional SSO — OpenAthens / EZProxy / Shibboleth)
scitex-scholar auth status              # exit 0 if any session valid, 1 otherwise
scitex-scholar auth login               # trigger SSO flow now (debug-friendly)
scitex-scholar auth logout -y           # clear cached cookies (--yes required)
scitex-scholar auth refresh             # logout + login

# MCP server
scitex-scholar mcp start
scitex-scholar mcp list-tools --json

# Shell completion
scitex-scholar install-shell-completion --shell bash
scitex-scholar print-shell-completion --shell bash

# Skills + Python API introspection
scitex-scholar dev skills list
scitex-scholar list-python-apis -v

Core API

_{Table 1. Public Python API symbols and their purposes.}

Sources: core/, search_engines/, metadata_engines/, pdf_download/, pipelines/, browser/, auth/, storage/, pdf_highlight/, _mcp/.

Semantic PDF Highlighting

Overlay colour-coded highlights on a PDF that separate what the paper claims from its methods, self-admitted limitations, and stance toward related work. Highlights are standard PDF annotation objects placed on a copy of the source — the original bytes are unchanged and any viewer can show or strip them.

_{Table 2. Highlight colours and their rhetorical categories.}

A compact colour legend + signature (model name, timestamp) is stamped in the lower-right corner of the last page. See docs for full details.

export ANTHROPIC_API_KEY=sk-ant-...
scitex-scholar pdf highlight paper.pdf        # sentence-level, Haiku, writes paper.highlighted.pdf
scitex-scholar pdf highlight paper.pdf --stub # offline keyword heuristic (no API calls)

from scitex_scholar.pdf_highlight import highlight_pdf
result = highlight_pdf("paper.pdf", output_path="paper.highlighted.pdf")
print(result.counts(), result.annotations_added)

Also exposed as the scholar_highlight_pdf MCP tool (unified scitex serve server) and as the semantic-highlight agent skill (see skills documentation).

Storage layout

~/.scitex/scholar/library/
├── MASTER/<HASH>/               # Canonical per-paper storage (metadata.json + PDF)
└── <project>/<human-label> -> ../MASTER/<HASH>

Cache and auth state live under ~/.scitex/scholar/cache/ (URL resolver, Chrome profiles, OpenAthens cookies). Override with SCITEX_DIR.

License

AGPL-3.0-only.

Part of SciTeX

scitex-scholar is part of SciTeX. Install via the umbrella with pip install scitex[scholar] to use as scitex.scholar (Python) or scitex scholar ... (CLI).

Four Freedoms for Research

0. The freedom to run your research anywhere — your machine, your terms.
1. The freedom to study how every step works — from raw data to final manuscript.
2. The freedom to redistribute your workflows, not just your papers.
3. The freedom to modify any module and share improvements with the community.

AGPL-3.0 — because we believe research infrastructure deserves the same freedoms as the software it runs on.