alvitr-sentinel 1.0.2

Automated Scientific Literature Monitoring System

AlvitrSentinel

Automated Scientific Literature Monitoring System

AlvitrSentinel is a command-line tool that automates the daily monitoring of scientific literature. It fetches articles from RSS feeds of major journals, uses AI to generate summaries, classify topics, and score relevance to your research interests, then produces interactive HTML reports for convenient review and manual archiving to reference managers such as Zotero.

The language of LLM prompts has been set to be Chinese. If you need to use English, please modify ai_engine.py.

Skills

I also support AI skills in AlvitrSentinelSkills.

Features

• RSS Feed Fetching — Supports ETag conditional requests for efficient caching, automatic retries with exponential back-off, and configurable timeouts.
• Three-Layer Abstract Acquisition — (1) RSS-embedded description, (2) web-page capture of the abstract section, (3) title-only fallback with appropriate labeling.
• AI-Powered Processing — Batch LLM calls via any OpenAI-compatible API to generate Chinese-language summaries, assign topic tags, and produce a 1-5 relevance score with rationale.
• Dual HTML Reports — A daily briefing stratified by relevance tier, and a cumulative single-page dashboard with full-text search, date-range filtering, topic filtering, and paginated results.
• Optional Embedding Vectors — Compute dense vectors for semantic search and article clustering using a separate embedding model.
• Pure File-Based Storage — JSON files partitioned by date; no database required.
• Flexible Configuration — YAML config covering AI models, research interests, topic taxonomy, feed list, and all pipeline parameters.

Installation

pip install alvitr-sentinel

Or install from source:

git clone https://github.com/Secretloong/AlvitrSentinel.git
cd alvitr-sentinel
pip install -e .

Quick Start

# 1. Initialize project (creates directories, config, and feed list)
alvitr-sentinel --init

# 2. Edit configuration
#    - Set your LLM API key
#    - Describe your research interests
#    - Customize the feed list
vim config.yaml
vim journal-feeds.txt

# 3. Run the full pipeline
alvitr-sentinel

Command-Line Usage

Command	Description
alvitr-sentinel	Full pipeline run
alvitr-sentinel --init	Initialize project directory
alvitr-sentinel --fetch-only	Fetch and parse only, skip AI (useful for testing)
alvitr-sentinel --no-ai	Skip AI processing, still store and generate reports
alvitr-sentinel --report-only	Regenerate reports from existing data
alvitr-sentinel --date 2026-03-20	Process a specific date
alvitr-sentinel --days 60	Override dashboard display window
alvitr-sentinel -v	Enable verbose logging
alvitr-sentinel --version	Show version

Configuration

The config.yaml file controls all aspects of the pipeline:

Section	Key Fields	Description
ai.llm	base_url, api_key, model	LLM inference model for summarization and scoring
ai.embedding	enabled, base_url, model	Optional embedding model for semantic search
research_interests	(free text)	Natural-language description of your research focus; directly affects relevance scoring
topic_taxonomy	(list)	Topic labels the AI assigns to each article
filter	min_relevance, highlight_threshold	Controls report filtering and highlighting
fetcher	timeout, retries, user_agent	Feed fetching behavior
parser	min_abstract_length, scrape_abstract	Abstract extraction settings

Pipeline Architecture

journal-feeds.txt
       |
       v
  +---------+     +---------+     +-----------+     +---------+     +--------+
  | Fetcher | --> | Parser  | --> | AIEngine  | --> | Storage | --> | Report |
  +---------+     +---------+     +-----------+     +---------+     +--------+
  RSS Feed        Metadata         Summarize,        JSON by         HTML
  fetch with      extraction,      classify,         date,           daily +
  ETag cache      3-layer           score             dedup index     dashboard
                  abstract

Output

• Daily Briefing: data/reports/daily/YYYY-MM-DD.html — Articles grouped by relevance tier (high / medium / low) with AI summaries, topic tags, and direct links to original papers.
• Cumulative Dashboard: data/reports/dashboard.html — A self-contained single-page application with search, filtering, sorting, and pagination across all collected articles.

Supported AI Services

Any OpenAI-compatible API endpoint works, including:

• Alibaba Cloud DashScope (Qwen series)
• SiliconCloud / SiliconFlow
• OpenAI
• Azure OpenAI
• Local LLM servers (vLLM, Ollama, etc.)

Module Overview

Module	Description
fetcher.py	RSS/Atom feed fetcher with conditional request caching and retry logic
parser.py	Article metadata extraction with three-tier abstract acquisition
ai_engine.py	AI summarization, classification, and relevance scoring engine
storage.py	JSON file storage layer with deduplication index
report_generator.py	Jinja2-based HTML report generator (daily briefing + dashboard)
pipeline.py	CLI entry point and pipeline orchestrator

Requirements

• Python >= 3.9
• feedparser, requests, openai, jinja2, pyyaml, beautifulsoup4, lxml, tqdm

License

MIT License