pyreqscan 1.0.0

A smart CLI tool that scans Python projects for imports and auto-generates requirements.txt with latest PyPI versions

PyReqScan

A smart CLI tool that scans Python projects for import statements and auto-generates requirements.txt with the latest PyPI versions.

✨ Features

• 🔍 AST-Based Scanning — Parses Python files using the Abstract Syntax Tree for accurate import detection (no regex hacks)
• 🧠 Smart Stdlib Detection — Dynamically identifies standard library modules using sys.stdlib_module_names (Python 3.10+) with fallbacks
• 🌐 PyPI Version Resolution — Fetches the latest stable version for each package from PyPI's JSON API
• ⚡ Concurrent Resolution — Resolves multiple package versions in parallel using ThreadPoolExecutor
• 📦 Import → PyPI Mapping — Handles packages where import name ≠ PyPI name (cv2 → opencv-python, PIL → Pillow, yaml → PyYAML)
• 🚫 Smart Exclusions — Automatically skips venv/, node_modules/, __pycache__/, .git/, etc.
• 🛠️ Environment Setup — One-command virtual environment creation with auto-installation (--init-env --install).
• 🆚 VS Code Integration — Automatically configures VS Code (.vscode/settings.json) to activate your .venv on new terminals.
• 🎨 Colorized Output — Rich terminal output with progress indicators
• 🖥️ Cross-Platform — Works on Windows, macOS, and Linux

📦 Installation

Using pip

pip install pyreqscan

Using pipx (recommended for CLI tools)

pipx install pyreqscan

From source

git clone https://github.com/crrrowz/pyreqscan.git
cd pyreqscan
pip install -e .

🚀 Usage

Basic Usage

# Scan current directory and generate requirements.txt
pyreqscan

# Scan a specific project directory
pyreqscan ./my-project

# Preview without writing (dry run)
pyreqscan --dry-run

# Generate requirements.txt AND automatically pip install them
pyreqscan . --install

# Create a virtual environment (.venv) and configure VS Code auto-activation
pyreqscan . --init-env

# Full Setup: Create venv, configure VS Code, generate requirements, and install them
pyreqscan . --init-env --install

Auto-Activation Wrapper Scripts

If you want a one-click setup that also activates the environment in your current terminal session, you can use the included wrapper scripts from the scripts/ directory:

# Windows (PowerShell)
.\scripts\setup-env.ps1

# macOS / Linux (Must use 'source' to modify current shell)
source scripts/setup-env.sh

Version Pinning Styles

# Compatible release (default): ~=2.31.0
pyreqscan --pin compatible

# Exact version: ==2.31.0
pyreqscan --pin exact

# Minimum version: >=2.31.0
pyreqscan --pin minimum

# No version pinning
pyreqscan --pin none

Offline Mode

# Skip PyPI version lookups (just list package names)
pyreqscan --no-version

Other Options

# Custom output file
pyreqscan -o deps.txt

# Verbose output (show discovered packages & progress)
pyreqscan -v

# Don't include header comments
pyreqscan --no-header

# Don't backup existing requirements.txt
pyreqscan --no-backup

# Disable colors
pyreqscan --no-color

📋 Example Output

Running pyreqscan on a Flask project:

📦 PyReqScan — Scanning for imports...

  Scanned 12 Python files
  Found 8 third-party imports (15 stdlib skipped)

🔍 Resolving latest versions from PyPI...

  [1/8] ✓ Flask → 3.0.0
  [2/8] ✓ requests → 2.31.0
  [3/8] ✓ SQLAlchemy → 2.0.23
  [4/8] ✓ celery → 5.3.6
  [5/8] ✓ redis → 5.0.1
  [6/8] ✓ gunicorn → 21.2.0
  [7/8] ✓ python-dotenv → 1.0.0
  [8/8] ✓ PyYAML → 6.0.1

  Resolved 8/8 versions

✅ Generated requirements.txt with 8 dependencies

Generated requirements.txt:

# Auto-generated by pyreqscan on 2024-12-24 12:00:00 UTC
# Scanned 12 Python files
# Found 8 third-party dependencies
#

celery~=5.3.6
Flask~=3.0.0
gunicorn~=21.2.0
python-dotenv~=1.0.0
PyYAML~=6.0.1
redis~=5.0.1
requests~=2.31.0
SQLAlchemy~=2.0.23

⚙️ CLI Reference

usage: pyreqscan [OPTIONS] [DIRECTORY]

Scan Python projects for imports and auto-generate requirements.txt.

positional arguments:
  directory              Directory to scan (default: current directory)

options:
  -h, --help             Show this help message
  -V, --version          Show version number
  -o, --output FILE      Output file path (default: requirements.txt)
  --pin STYLE            Version pinning: exact, compatible, minimum, none
  --no-version           Skip PyPI version resolution (offline mode)
  --no-header            Don't include header comments
  --dry-run              Preview output without writing to file
  --no-backup            Don't backup existing requirements.txt
  --include-stdlib       Include standard library modules in output
  --no-color             Disable colorized output
  -v, --verbose          Show detailed scanning information
  --install              Auto-install packages after generating
  --init-env             Create a .venv virtual environment
  --env-path PATH        Virtual environment directory name (default: .venv)

🧠 How It Works

┌─────────────────┐     ┌──────────────────┐     ┌────────────────┐
│  1. SCAN        │ ──▶ │  2. RESOLVE      │ ──▶ │  3. GENERATE   │
│                 │     │                  │     │                │
│ Walk .py files  │     │ Query PyPI API   │     │ Format output  │
│ Parse with AST  │     │ Get latest       │     │ Pin versions   │
│ Extract imports │     │ stable versions  │     │ Write file     │
│ Filter stdlib   │     │ (concurrent)     │     │ Backup old     │
│ Detect local    │     │                  │     │                │
└─────────────────┘     └──────────────────┘     └────────────────┘

Stdlib Detection Strategy

Priority	Method	Python Version
1	sys.stdlib_module_names	3.10+
2	stdlib-list package	3.8 - 3.9
3	importlib.metadata heuristic	All
4	Minimal built-in fallback	All

Import Name Resolution

Many PyPI packages use different names for installation vs import:

Import Name	PyPI Package
cv2	opencv-python
PIL	Pillow
sklearn	scikit-learn
bs4	beautifulsoup4
yaml	PyYAML
dateutil	python-dateutil
dotenv	python-dotenv
attr	attrs

PyReqScan ships with 100+ known mappings and also uses importlib.metadata.packages_distributions() (Python 3.11+) for dynamic runtime resolution.

🐍 Python API

from pyreqscan import scan_directory, resolve_package_version, generate_requirements
from pyreqscan.resolver import resolve_multiple

# Scan a project
result = scan_directory("./my-project")

# Get third-party imports
for name, info in result.third_party.items():
    print(f"{info.pypi_name}: imported in {len(info.source_files)} files")

# Resolve versions
pypi_names = [info.pypi_name for info in result.third_party.values()]
versions = resolve_multiple(pypi_names)

# Generate requirements.txt content
content = generate_requirements(result, versions, pin_style="exact")
print(content)

🔒 Default Scan Exclusions

Category	Patterns
Version Control	.git, .hg, .svn
Python	__pycache__, .pytest_cache, .mypy_cache, .eggs
Virtual Env	venv, .venv, env, ENV, site-packages
Node.js	node_modules, .npm, .yarn
Build	dist, build, out, target
IDE	.idea, .vscode, .vs

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Inspired by pipreqs and the need for a modern, accurate, AST-based requirements generator.