arXiv Scanner

A command-line tool that fetches and shows new arXiv papers for a chosen domain (archive, for example cs) and subdomain (category, for example cs.CV).

It uses only the Python standard library, so there is nothing to pip install. It needs Python 3.9 or later and runs on Windows, macOS and Linux.

Installation

Install from PyPI:

pip install arxivscanner
arxivscanner --list        # check it works: prints every known domain and subdomain

To upgrade later, run pip install -U arxivscanner. To get the latest unreleased code, run pip install git+https://github.com/shubham10divakar/arxivscanner.git.

This installs an arxivscanner command. python -m arxivscanner … works the same way. Use python3 -m pip on macOS or Linux if pip points at an older Python.

To work on the code instead, clone the repo and install it in editable mode:

git clone https://github.com/shubham10divakar/arxivscanner.git
cd arxivscanner
pip install -e .

Without installing, python run.py … from the repo folder also works.

How to use it

1. Interactive mode (easiest)

Run the tool with no arguments and answer the prompts:

arxivscanner

1. Pick a domain. Type its number (for example 1 for cs).
2. Pick one or more subdomains. Type numbers such as 8 or 8,23, or type 0 for the whole domain. You can also type arXiv codes directly (cs.CV cs.LG).
3. Pick a mode. 1 shows today's announcement. 2 shows papers submitted in the last N days, and asks for N.

2. Command-line flags

Pass -c with one or more codes to skip the prompts:

arxivscanner -c cs.CV                          # today's Computer Vision list
arxivscanner -c cs.CV --type new               # only brand-new submissions (no cross-lists or updates)
arxivscanner -c cs.CV cs.LG --short            # two subdomains, abstracts trimmed
arxivscanner -c cs                             # the whole Computer Science domain
arxivscanner -c cs.CV --mode recent --days 3   # everything submitted in the last 3 days
arxivscanner -c cs.CV --mode recent --source oai   # same, read straight from OAI-PMH
arxivscanner -c cs.CV --md cv.md --json cv.json   # also save the results to files
arxivscanner --from-file saved_feed.xml        # parse a saved RSS, API or OAI-PMH XML file offline

Which mode should I use?

When is there something new? arXiv announces Sunday to Thursday at 20:00 US Eastern time, which is about 05:30 IST the next morning. There are no announcements on Friday or Saturday nights US Eastern, so the Saturday and Sunday (IST) feeds are empty. On those days, use --mode recent --days 4 or more. A paper only becomes visible once it is announced, so papers submitted on Friday or over the weekend first show up after Sunday night's announcement, in both modes. cs.CV usually has 150–300 papers per announcement.

Example output

arXiv · cs.CV (Computer Vision and Pattern Recognition)
Announcement: Mon, 28 Sep 2026 00:00:00 -0400
3 papers  new: 1  cross: 1  replace: 1

  1. 2609.00001v1 [new]
     Sample Paper A: Looped Vision Transformers for Fine-Grained Recognition
     Alice Author, Bob Builder, Chandra Kumar
     cs.CV, cs.LG · 2026-09-28
     We study looped vision transformers and show strong results on fine-grained benchmarks.
     https://arxiv.org/abs/2609.00001  https://arxiv.org/pdf/2609.00001

Codes

Run arxivscanner --list to see all built-in codes. They cover cs, eess, stat, math, q-bio, q-fin, econ, astro-ph, cond-mat, physics, nlin, quant-ph, gr-qc, hep-th and hep-ph. Any other valid arXiv code also works, even if it isn't in the list. Some common ones:

Rate limiting

The arXiv search API (export.arxiv.org) throttles hosts. While it does, it answers HTTP 406 Not Acceptable to any query it hasn't cached, and waiting a few minutes often doesn't clear it. So with the default --source auto, the tool doesn't retry a 406. It switches straight to arXiv's OAI-PMH feed (oaipmh.arxiv.org), which gives the same papers with the same fields:

Querying the arXiv API for cs.AI, last 4 day(s) …
  ! arXiv API refused the query (HTTP 406 Not Acceptable from export.arxiv.org); switching to OAI-PMH …
  cs:cs:AI: scanned 659 records, 172 submitted in window

OAI-PMH lists every record changed since a date, including older papers that only got a new version. The tool keeps only papers whose first version was submitted inside the window. For a 4-day cs.AI window this gave exactly the same 172 papers as the API.

Busy responses (429, 503) and network errors are retried with back-off, honouring Retry-After. The tool waits 3 seconds between requests, as arXiv's terms ask. If a later API page fails for another reason, it keeps the papers it already has and prints a warning. The RSS feed (today mode) is rarely affected.

Development

python -m unittest discover -s tests     # offline tests, using the XML fixtures in tests/fixtures
python -m build                          # builds dist/arxivscanner-<version>.tar.gz and .whl (pip install build)

The version lives in arxivscanner/__init__.py (__version__).

Design

arxivscanner ──► cli.py ──► fetchers.py ──► arXiv (RSS / API / OAI-PMH)
                   │             │
                   │             └─► models.Paper   (one normalised record)
                   ├─► taxonomy.py  (domain → subdomain names, picker)
                   └─► display.py   (terminal view, JSON / Markdown export)

Roadmap

• 0.1 (current) Fetch, display and export by domain and subdomain. 0.1.1 adds the OAI-PMH fallback for --mode recent.
• 0.2 Remember papers already seen (a local SQLite or JSON file) so each run shows only unseen papers.
• 0.3 Keyword or interest filtering and ranking (title and abstract match, later embeddings).
• 0.4 Daily automation (a scheduled task or cron) and a digest by email, Telegram or HTML.
• 0.5 Dashboard view with bookmarking.