vdweb 0.1.0

Excel for Developers - A fast, local-first web GUI for exploring data with VisiData

VisiLens

Excel for Developers

Open 1M+ rows in seconds. Local, fast, simple.

A high-performance, local-first web GUI for exploring datasets. Instantly view and filter CSV, Parquet, Excel, and JSON files using the power of VisiData.

Features • Installation • Usage • Architecture • Contributing

---

🚀 Why VisiLens?

Data exploration shouldn't require writing boilerplate Pandas code or waiting for heavy spreadsheet software like Excel to load. VisiLens is a modern CSV viewer and Parquet explorer that combines the raw speed of VisiData with a lightweight web interface.

• ⚡️ Load 1M rows in under 2 seconds: Powered by VisiData's highly optimized engine.
• 🔒 All local: Your data never leaves your machine. No cloud uploads.
• 🛠 Zero config: CLI-first workflow. Pipe data in, explore, and get back to coding.
• 🔌 Universal Support: Open CSV, TSV, JSON, Parquet, Excel, SQLite, and 50+ other formats.

✨ Features

• Instant Data Visualization: Just run vdweb data.csv to visualize large datasets instantly.
• Backend-powered Sorting & Filtering: Perform complex queries on millions of rows using the VisiData engine.
• Lightweight Data Grid: A virtualized React-based table view for smooth scrolling.
• Zero Config: No database setup required. Works as a standalone CSV/Parquet viewer.

📂 Supported Formats

VisiLens leverages VisiData's loaders to support a wide range of formats out of the box:

• Tabular: .csv, .tsv, .xlsx (Excel), .parquet
• Structured: .json, .jsonl, .yaml
• Database: .sqlite, .db
• Code: .pcap (Wireshark), .xml, .html tables

📊 Benchmarks

We take performance seriously. Here is how VisiLens compares when opening a 1,000,000 row CSV dataset on a standard MacBook Air (M2).

Tool	Load Time (1M Rows)	Memory Footprint	Interactive Sorting
VisiLens	~1.7s	Minimal (< 50MB Total)	Instant (Backend: < 0.4s)
Excel	> 30s (Often Fails)	High (Blocking RAM)	Slow/Unresponsive
Pandas-based GUI	> 15s (Cold Start)	High (Entire DF in RAM)	Sluggish (Non-Virtualized)
Jupyter (print df)	Fast	Medium	Static Text

Test Data: 1M rows, 3 columns (Mixed types). Numbers are from my MacBook Air M2 during real development use.

📦 Installation

VisiLens is available as a Python package.

pip install vdweb

Note: VisiLens requires Python 3.10 or higher.

💻 Usage

Command Line Interface

The primary way to use VisiLens is via the command line.

# Open a CSV file
vdweb data.csv

# Open a Parquet file
vdweb large-dataset.parquet

# Open an Excel file
vdweb spreadsheet.xlsx

# Launch without opening the browser automatically
vdweb data.json --no-browser

# Specify a custom port
vdweb data.csv --port 9000

Web Interface

Once launched, VisiLens opens in your default browser (usually http://localhost:8000).

1. View Data: Scroll through your dataset efficiently.
2. Sort: Click column headers to sort ascending/descending.
3. Filter: Use the filter inputs to search within columns.
4. Load New Data: (Coming Soon) Drag and drop files directly into the window.

🏗 Architecture

VisiLens is built on a robust modern stack designed for performance:

• Backend: FastAPI server bridges VisiData and the browser.
• Communication: WebSockets stream slices on demand.
• Frontend: React grid renders only what you see.

🗺 Roadmap

We are actively working on making VisiLens the ultimate local data companion.

• v0.1: Core Engine, Virtual Scrolling, Sorting, Filtering.
• Jupyter Integration: Launch VisiLens directly from a notebook cell (visilens.view(df)).
• Drag-and-drop file loading
• Plotting: Quick histograms and scatter plots via Vega-Lite.
• Editing: Edit cells and save changes back to CSV/Parquet.
• SQL Support: Connect directly to SQLite/Postgres/DuckDB.

🛠 Development

Want to contribute? Great! Here's how to set up the development environment.

Prerequisites

• Python 3.10+
• Node.js 18+
• npm or pnpm

Setup

1. Clone the repository

   git clone https://github.com/PStarH/VisiLens.git
   cd VisiLens
   

2. Backend Setup

   # Create virtual environment
   python -m venv .venv
   source .venv/bin/activate  # or .venv\Scripts\activate on Windows
   
   # Install dependencies
   pip install -e ".[dev]"
   

3. Frontend Setup

   cd frontend
   npm install
   

4. Run Locally

Terminal 1 (Backend):

uvicorn backend.main:app --reload --port 8000

Terminal 2 (Frontend):

cd frontend
npm run dev

5. Build frontend assets (optional)

If you prefer to run only the Python CLI (without a separate Vite dev server), you can build the frontend once:

cd frontend
npm run build

This produces a production bundle under frontend/dist/ which is copied into vdweb/static/ for releases. End users then just run:

vdweb path/to/data.csv

🤝 Contributing

For Contributors: where things live

• Python package (vdweb/): This is the installable package published to PyPI. The CLI entrypoints vdweb / visilens both resolve to vdweb.cli:main as configured in pyproject.toml.
• Dev backend (backend/): A separate FastAPI app used only for local development (uvicorn backend.main:app). It mirrors the behavior of the packaged backend but is not what users import when they install vdweb.
• Core logic: The VisiData-powered data access layer lives in vdweb/core.py (and is mirrored in backend/core.py for the dev app). If you want to change how data is loaded/sorted/filtered, start here.

Typical contributor workflow

1. Edit backend / core logic in vdweb/ (and update backend/ if needed for dev parity).
2. Run the dev backend + frontend locally as described in Development.
3. If you change the React app and want those changes to ship, run npm run build in frontend/ so the bundle that will be copied into vdweb/static/ is up to date.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

Made with ❤️ by PStarH and the Open Source Community.