market-price-tracker 0.1.0

Stock Scraper SDK

Stock Scraper SDK

A comprehensive Python SDK to scrape, store, and stream live stock prices. This project uses Selenium for scraping Google Finance, FastAPI for REST and WebSocket interfaces, and SQLite for historical data storage.

Folder Structure

stockscraper/
├── README.md               # Project documentation
├── pyproject.toml          # Project configuration and metadata
├── run.py                  # Entry point to start the FastAPI server
├── stocks.db               # SQLite database containing stock prices history
├── chromedriver-mac-arm64/ # Chromedriver executable (used for Selenium scraping)
├── sdk/                    # Core SDK Module
│   ├── api/
│   │   ├── main.py         # FastAPI application and WebSocket endpoint setup
│   │   └── routes.py       # REST API endpoints (/price/{ticker}, /historical/{ticker})
│   ├── cli/
│   │   └── main.py         # CLI wrapper to fetch stock price from the command line
│   ├── streaming/
│   │   ├── manager.py      # StockStreamer class to handle live WebSocket price pushing
│   │   └── server.py       # Core WebSocket connection lifecycle methods
│   ├── client.py           # StockClient acting as the main interface for fetching and saving prices
│   ├── database.py         # SQLAlchemy configuration & Session maker
│   ├── models.py           # SQLAlchemy Database models (e.g. StockPrice)
│   └── scraper.py          # Selenium logic to scrape stock prices directly from Google search

Features

• Live Data Scraping: Uses a headless Selenium browser to bypass simple bot protections and fetch real-time search results from Google.
• REST API: Standard endpoints to query the latest price or get a historical summary of a ticker.
• WebSocket Streaming: Continuous real-time stream of the stock's price to an active WebSocket client.
• Persistent Storage: Uses SQLAlchemy and SQLite to record all requested stock prices into stocks.db.
• Command Line Interface (CLI): Easy checking of stock prices directly from your terminal.

Installation

1. Requirements

Ensure you have Python 3.13+ installed. Since this project uses Selenium, ensure you have Chrome installed on your machine. The chromedriver-mac-arm64 directory contains the chrome driver for Apple Silicon Macs, but the modern Selenium framework should automatically handle driver management.

2. Setup

Clone the repository and install the dependencies. The recommended approach is using a virtual environment:

# Create and activate a virtual environment
python3 -m venv venv
source venv/bin/activate  # On macOS/Linux

# If using standard pip, install basic requirements
# Note: You may need to install fastapi, uvicorn, selenium, sqlalchemy, and websockets manually if they are not listed in a requirements.txt file.
pip install fastapi uvicorn selenium sqlalchemy websockets

Usage

1. Running the CLI Interface

You can quickly get a stock's current price via the terminal by using the cli module.

python3 -m sdk.cli.main NSE:BEL

2. Running the REST & WebSocket API Server

To start the FastAPI server with hot-reloading:

python3 run.py

This will run the server on http://127.0.0.1:8000.

3. API Endpoints

Once the server is running, the following interfaces are available:

REST API

• GET /price/{ticker}: Fetch the immediate current price of a stock (e.g., http://127.0.0.1:8000/price/NSE:BEL).
• GET /historical/{ticker}?days=30: Fetch the historically saved prices for a specific stock over a set amount of days.
• GET /health: Server health-check.

WebSockets

• ws://127.0.0.1:8000/ws/{ticker}: Connect to a WebSocket stream that will emit JSON payloads continuously with updated stock information.

npm install -g wscat

Example connecting to the WebSocket using wscat:

wscat -c ws://127.0.0.1:8000/ws/NSE:BEL