arXiv Scanner

A command-line tool that fetches and shows new arXiv papers for a chosen domain (archive, for example cs) and subdomain (category, for example cs.CV).

It uses only the Python standard library, so there is nothing to pip install. It needs Python 3.9 or later and runs on Windows, macOS and Linux.

Installation

Install from GitHub with pip:

pip install git+https://github.com/shubham10divakar/arxivscanner.git
arxivscanner --list        # check it works: prints every known domain and subdomain

This installs an arxivscanner command. python -m arxivscanner … works the same way. Use python3 -m pip on macOS or Linux if pip points at an older Python.

To work on the code instead, clone the repo and install it in editable mode:

git clone https://github.com/shubham10divakar/arxivscanner.git
cd arxivscanner
pip install -e .

Without installing, python run.py … from the repo folder also works.

How to use it

1. Interactive mode (easiest)

Run the tool with no arguments and answer the prompts:

arxivscanner

1. Pick a domain. Type its number (for example 1 for cs).
2. Pick one or more subdomains. Type numbers such as 8 or 8,23, or type 0 for the whole domain. You can also type arXiv codes directly (cs.CV cs.LG).
3. Pick a mode. 1 shows today's announcement. 2 shows papers submitted in the last N days, and asks for N.

2. Command-line flags

Pass -c with one or more codes to skip the prompts:

arxivscanner -c cs.CV                          # today's Computer Vision list
arxivscanner -c cs.CV --type new               # only brand-new submissions (no cross-lists or updates)
arxivscanner -c cs.CV cs.LG --short            # two subdomains, abstracts trimmed
arxivscanner -c cs                             # the whole Computer Science domain
arxivscanner -c cs.CV --mode recent --days 3   # everything submitted in the last 3 days
arxivscanner -c cs.CV --md cv.md --json cv.json   # also save the results to files
arxivscanner --from-file saved_feed.xml        # parse a previously saved RSS/API XML file offline

Which mode should I use?

When is there something new? arXiv announces Sunday to Thursday at 20:00 US Eastern time, which is about 05:30 IST the next morning. There are no announcements on Friday or Saturday nights US Eastern, so the Saturday and Sunday (IST) feeds are empty. On those days, use --mode recent --days 3. cs.CV usually has 150–300 papers per announcement.

Example output

arXiv · cs.CV (Computer Vision and Pattern Recognition)
Announcement: Mon, 28 Sep 2026 00:00:00 -0400
3 papers  new: 1  cross: 1  replace: 1

  1. 2609.00001v1 [new]
     Sample Paper A: Looped Vision Transformers for Fine-Grained Recognition
     Alice Author, Bob Builder, Chandra Kumar
     cs.CV, cs.LG · 2026-09-28
     We study looped vision transformers and show strong results on fine-grained benchmarks.
     https://arxiv.org/abs/2609.00001  https://arxiv.org/pdf/2609.00001

Codes

Run arxivscanner --list to see all built-in codes. They cover cs, eess, stat, math, q-bio, q-fin, econ, astro-ph, cond-mat, physics, nlin, quant-ph, gr-qc, hep-th and hep-ph. Any other valid arXiv code also works, even if it isn't in the list. Some common ones:

Rate limiting

The arXiv API sometimes rate-limits in bursts, answering 406 or 429 for a few minutes. The tool retries with back-off for up to about 4 minutes per request and waits 3 seconds between pages, as arXiv's terms ask. If a later page still fails, it keeps the papers it already fetched and prints a warning. If the first request fails, wait a few minutes and run the command again. The RSS feed (today mode) is rarely affected.

Development

python -m unittest discover -s tests     # offline tests, using the XML fixtures in tests/fixtures
python -m build                          # builds dist/arxivscanner-<version>.tar.gz and .whl (pip install build)

The version lives in arxivscanner/__init__.py (__version__).

Design

arxivscanner ──► cli.py ──► fetchers.py ──► arXiv (RSS / API)
             │             │
             │             └─► models.Paper   (one normalised record)
             ├─► taxonomy.py  (domain → subdomain names, picker)
             └─► display.py   (terminal view, JSON / Markdown export)

Roadmap

• 0.1 (current) Fetch, display and export by domain and subdomain.
• 0.2 Remember papers already seen (a local SQLite or JSON file) so each run shows only unseen papers.
• 0.3 Keyword or interest filtering and ranking (title and abstract match, later embeddings).
• 0.4 Daily automation (a scheduled task or cron) and a digest by email, Telegram or HTML.
• 0.5 Dashboard view with bookmarking.