PeerSight - AI Academic Paper Reviewer
Version: 0.1.1
PeerSight is an AI-powered command-line tool designed to automate and enhance the academic paper review process. It acts like a virtual journal reviewer, leveraging local Large Language Models (LLMs) via Ollama to provide structured peer reviews of research papers supplied as plain text files.
The goal is to simulate the output of an expert reviewer – clean, concise, and decision-oriented – without exposing the underlying step-by-step reasoning process of the LLM.
✨ Features
- 📄 Accepts academic papers as plain text files (
.txt). - 🧠 Uses a locally hosted LLM via Ollama for review generation.
- ⚙️ Configurable LLM model, API endpoint, and generation parameters (temperature, top_k, top_p) via
.envfile and CLI flags. - 📝 Generates structured reviews including:
## Summary## Strengths(considering novelty, significance, methodology, clarity, evidence)## Weaknesses / Areas for Improvement(considering same criteria)## Recommendation(Accept, Minor Revision, Major Revision, Reject)
- 🧹 Post-processes LLM output to remove extraneous thinking steps or conversational filler.
- 💾 Outputs review in human-readable Text format (default) or machine-readable JSON format (
--jsonflag). - M️ Warns if input paper exceeds a configurable length threshold (
MAX_PAPER_LENGTH_WARN_THRESHOLD). - 💻 Simple and clean Command-Line Interface (CLI).
- 📦 Installable Python package via
pip. - ✅ Includes unit tests and code quality checks (
pytest,black,ruff,pre-commit). - 🔬 Experimental: Attempts to extract references from the paper for future analysis (
--check-referencesflag).
⚙️ Configuration
PeerSight uses a .env file in the project root directory for base configuration. Create this file if it doesn't exist (it's ignored by git).
.env File Example:
# --- REQUIRED ---
# Specify the Ollama model to use by default
# Make sure Ollama is serving this model!
OLLAMA_MODEL="llama3:latest" # Or deepseek-coder:latest, mistral:latest, etc.
# --- OPTIONAL ---
# Override the default Ollama API endpoint if needed
# OLLAMA_API_URL="http://custom-host:11434/api/generate"
# Override default generation parameters
# OLLAMA_TEMPERATURE=0.5 # Default: 0.7 (Lower = more deterministic)
# OLLAMA_TOP_K=40 # Default: -1 (disabled) (Consider only top K tokens)
# OLLAMA_TOP_P=0.9 # Default: -1.0 (disabled) (Consider tokens cumulative prob > P)
# Warning threshold for long papers (characters)
# MAX_PAPER_LENGTH_WARN_THRESHOLD=20000 # Default: 15000
Command-Line Overrides:
You can override the model, API URL, temperature, top-k, and top-p for a single run using CLI flags:
--model "model-name:tag"
# e.g., --model "llama3:8b"
# e.g., --model "deepseek-coder:latest"
--api-url "http://host:port/api/endpoint"
# e.g., --api-url "http://localhost:11434/api/generate"
-t TEMPERATURE or --temperature TEMPERATURE (e.g., -t 0.5)
# e.g., --temperature 0.3
# e.g., -t 0.7
--top-k K (e.g., --top-k 40)
# e.g., --top-k 20
# e.g., --top-k 0
--top-p P (e.g., --top-p 0.9)
# e.g., --top-p 0.95
CLI flags take precedence over .env settings.
🚀 Installation
Ensure you have Python >= 3.9 and pip installed. Ollama must also be installed and running separately (ollama.com).
- Clone the repository:
git clone https://github.com/RAHAMNIabdelkaderseifelislem/PeerSight
cd PeerSight
- Create and activate a virtual environment:
python -m venv venv
# Windows
.\venv\Scripts\activate
# macOS/Linux
# source venv/bin/activate
-
Install the package:
- For regular use:
pip install .
- For development (includes testing and quality tools):
pip install -e .[dev]
-
Set up .env: Copy the example above into a .env file in the project root and set your desired OLLAMA_MODEL.
- (Crucial) Ensure Ollama is running and the model specified in your config (OLLAMA_MODEL) is downloaded and available (e.g., ollama run llama3).
🛠️ Usage
Once installed, use the peersight command.
Basic:
- Review paper, print Text output to console
peersight path/to/your/paper.txt
- Get help/see all options
peersight --help
- Check version
peersight --version
Reference Handling (Experimental):
# Attempt to extract references (results currently logged)
peersight --check-references path/to/your/paper.txt
# Combine with other flags
peersight -v --check-references --json path/to/paper.pdf -o review_with_refs.json**Reference Handling (Experimental):**
```bash
# Attempt to extract references (results currently logged)
peersight --check-references path/to/your/paper.txt
# Combine with other flags
peersight -v --check-references --json path/to/paper.pdf -o review_with_refs.json
Output Control:
- Save Text review to a file
peersight path/to/paper.txt -o output/review.md
- Output JSON to console
peersight --json path/to/paper.txt
- Save JSON review to a file
peersight --json path/to/paper.txt -o output/review.json
LLM Control:
- Use a different model for this run
peersight --model "mistral:latest" path/to/paper.txt
- Adjust temperature for less randomness
peersight -t 0.3 path/to/paper.txt
- Use nucleus sampling
peersight --top-p 0.95 path/to/paper.txt
Combine options
- Use a different model, save JSON output to a file, and set temperature
peersight --model "llama3:8b" -t 0.5 --json path/to/paper.txt -o reviews/paper1_llama3.json
Verbose Logging:
- See DEBUG level logs (useful for troubleshooting)
peersight -v path/to/paper.txt
📝 Output Formats
-
Text (Default): Human-readable Markdown-like text with standard headers (## Summary, etc.). Suitable for quick reading or pasting into documents.
-
JSON (--json): Structured JSON object with keys summary, strengths, weaknesses, and recommendation. Useful for programmatic access or integration with other tools.
{
"summary": "The paper discusses...",
"strengths": "- Strength 1...\n- Strength 2...",
"weaknesses": "- Weakness 1...\n- Weakness 2...",
"recommendation": "Minor Revision"
}
⚠️ Limitations
-
Input Length: Very long papers may exceed the LLM's context window or Ollama's processing limits, potentially leading to poor results or errors (a warning is issued). Chunking is not yet implemented.
-
LLM Consistency: The quality and adherence to instructions depend heavily on the specific LLM used (model, size) and its inherent variability. Results may differ between runs, especially with higher temperatures.
-
Factuality/Citation Check: PeerSight does not verify the factual accuracy of the paper's claims or check citations. It simulates the review based on the provided text only.
-
Deep Understanding: As an AI simulation, it lacks the deep domain expertise and nuanced understanding of a human expert in a specific field.
-
No Feedback Loop: Does not currently support incorporating reviewer comments back into revisions.
-
Reference Extraction: The current reference extraction (
--check-references) is highly experimental and uses basic heuristics. It will struggle with diverse or poorly formatted reference lists in plain text or PDFs. Accuracy is not guaranteed. Full reference validation and analysis are future work.
🔧 Development
- Install dev dependencies:
pip install -e .[dev]
- Install pre-commit hooks:
python -m pre_commit install
- Run tests:
pytest
- Run linters/formatters:
ruff check --fix . and black .
Metadata
Release files for peersight 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| peersight-0.1.1.tar.gz | 31.5 kB | Without provenance | Details for peersight-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| peersight-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for peersight-0.1.1-py3-none-any.whl |
Total release size: 57.4 kB
Release files / peersight-0.1.1.tar.gz
| Download URL | peersight-0.1.1.tar.gz |
|---|---|
| Size | 31.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7ce0986543de00b3918b2f7eae030e141ffeeeead350d8d0670e1528835ef3fe
|
|
BLAKE2b-256 checksum How to use checksums |
c6ad5eea001c4912dee317d542eabbb64ca1141919ec241ce740d35c282dc7c2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.8
|
Release files / peersight-0.1.1-py3-none-any.whl
| Download URL | peersight-0.1.1-py3-none-any.whl |
|---|---|
| Size | 25.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f93daa76e631e2e5db1b85dbab9319db6ef3a2e2c78e2666fc50e58f41950032
|
|
BLAKE2b-256 checksum How to use checksums |
7c04b9ba5cba901a77c147447e8a07dc8050d4fd6fea56bfcdc306970f1d5a25
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.8
|
Security for peersight 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.