Atrium Job Scraper

A powerful, polite, and modular scraper that collects internship and job listings for the Atrium student community from various portals (Internshala, Naukri, LinkedIn, Indeed).

Features a beautiful interactive Terminal User Interface (TUI) and a full FastAPI Web Dashboard!

🚀 Installation

pip install atrium-job-scraper

Note: For the Indeed scraper, Google Chrome (or Chromium) must be installed on your machine. Selenium Manager will automatically handle the driver.

⚙️ Configuration

Copy the example environment file to configure your filters:

cp .env.example .env

Key .env settings to customize your search:

KEYWORDS=react,node.js,python,java
LOCATION=india
REMOTE_ONLY=false
INTERNSHIP_ONLY=false
MAX_PAGES=5

(You can also override these settings directly via CLI flags).

💻 Usage Guide

1. The Interactive Terminal (TUI) & Web UI 🌟

Launch the interactive opencode-style terminal alongside the live Web UI dashboard:

atrium-scraper

• Terminal Commands: Use /scrape naukri or /scrape all to fetch jobs. Prefix with ! to run standard bash commands (e.g., !ls). Type exit or Ctrl+C to quit.
• Web UI: Automatically launches at http://localhost:8000.

2. Standalone Web UI

If you only want the Web Dashboard without the interactive terminal:

uvicorn app:app --host 0.0.0.0 --port 8000

3. Traditional CLI (atrium-cli)

Perfect for cron jobs, automation, or quick one-off commands:

# Scrape specific sources
atrium-cli --source internshala
atrium-cli --source linkedin

# Scrape all supported sources
atrium-cli --source all

# Override your .env filters for a single run
atrium-cli --source all --keyword "frontend developer" --location "india"

📁 Output

By default, all scraped jobs are saved as both .json and .csv files inside the output/ directory, and duplicates are automatically removed. Detailed logs can be found in logs/scraper.log.