flatseek 0.1.6

Scalable engine to ingest, process, and query large datasets—transactions, logs, events, and analytics—directly from flat files.

Flatseek

Full-text search over flat files and remote datasets. No JVM. No cluster. No operational overhead.

Explore, filter, and aggregate CSV/JSON like a spreadsheet with search-grade power — run locally or query directly from Public/HuggingFace datasets.

Demo: flatlens.demo.flatseek.io  ·  Docs: flatseek.io/docs  ·  Dashboard: flatlens  ·  Benchmark: flatbench  ·  Hosted Datasets: HuggingFace

---

Why Flatseek

• Exact-by-design engine. Query results are deterministic with full-count accuracy on flat files.
• No infra required. Run locally as a single binary or deploy as a lightweight API sidecar.
• Millisecond search performance. Built for interactive exploration over large datasets.
• Zero-ops distribution. Store and query indexes directly from HuggingFace, S3, or Vercel Blob.
• Instant dataset access. Paste a dataset URL in Flatlens and start searching immediately — no ingestion pipeline needed.
• Built-in analytics dashboard. Spreadsheet-like experience with Kibana-style filtering, aggregation, and exploration — closer to Microsoft Excel for structured search workflows.
• Developer-first embedding. Use as a Python library or full API without running a cluster or managing nodes.

---

Search engine for public datasets instantly

Flatseek can query indexes hosted remotely — no need to deploy or pay for expensive storage infrastructure. Build your Flatseek index locally, upload it to any free-cheap publicly accessible storage service, and instantly search and aggregate data from anywhere.

Supported providers:

• HuggingFace Datasets / Buckets
• S3-compatible storage
• Vercel Blob
• Static HTTP hosting

Samples

Dataset	Rows
Local Sample	500 searchable rows
https://huggingface.co/buckets/flatseek/flatdata	10K searchable rows
https://huggingface.co/datasets/flatseek/sample-articles	100K articles
https://huggingface.co/datasets/flatseek/sample-adsb	100K ADS-B records
https://huggingface.co/datasets/flatseek/sample-encrypted	100K encrypted rows Password: flatseek

Encrypted datasets are supported with per-request passphrases.

---

Performance

500K rows, article schema, SSD. Flatseek is 2× faster on search than Elasticsearch with exact counts where others silently miss.

Metric	Flatseek	Elasticsearch
Search p50	7.9ms	16.1ms
Range query hits	501,011 (exact)	505,044
Build 500K rows	216s	113s

Full comparison (tantivy, typesense, whoosh, zincsearch): docs/benchmark.md or bench.flatseek.io

---

Quick Start

# 1. Install
curl -fsSL flatseek.io/install.sh | sh

# 2. Build index
flatseek build ./data.csv -o ./data

# 3. Serve API + dashboard
flatseek serve -d ./data

# Pack index into single portable file
flatseek pack ./data -o ./data.fsk

# Serve API + dashboard from single portable index file
flatseek serve data.fsk


# → API:
# http://localhost:8000

# → Dashboard:
# http://localhost:8000/dashboard

# 4. Query
flatseek search ./data "program:raydium AND amount:>1000000"

# Query from single portable file
flatseek search data.fsk "program:raydium AND amount:>1000000"

---

Core Capabilities

• Full text search — tokenized, trigram-backed wildcard (*kube*)
• Range queries — exact counts on numeric, date, keyword fields
• Aggregations — terms, stats, min/max, cardinality, date histogram
• Array fields — matches any element (tags:graphql)
• Nested objects — dot-path queries (address.city:Jakarta)
• Boolean operators — AND, OR, NOT with grouping
• Encryption at rest — ChaCha20-Poly1305
• Parallel indexing — multi-worker builds
• Remote querying — search HuggingFace datasets without downloading
• REST API — Elasticsearch-compatible endpoints

See docs/ for full details.

---

Docs

Guide	Description
Quick Start	Install, index, query — in 5 minutes
Indexing	Formats, column types, parallel builds, encrypt
Query Language	Full syntax reference
CLI Reference	All CLI commands
REST API	API endpoints
Remote Storage	HuggingFace, S3, Vercel Blob
Schemas	Supported Column Types
Architecture	Structural and behavioral map
Internals	Deep technical breakdown

---

Install

Recommended — one-liner

curl -fsSL flatseek.io/install.sh | sh

Includes API server + Flatlens dashboard (http://localhost:8000/dashboard).

PyPI

pip install flatseek

CLI only. For Flatlens dashboard:

git clone https://github.com/flatseek/flatlens

From source

git clone https://github.com/flatseek/flatseek.git

cd flatseek
pip install -e .

Requirements: Python ≥ 3.10, macOS / Linux / WSL.

---

Contributing

PRs welcome. Run tests:

pytest src/flatseek/test/test_search.py -v   # accuracy tests
pytest src/flatseek/test/test_api.py         # API smoke tests
pytest src/flatseek/test/test_cli.py         # CLI integration

---

License

Apache 2.0. See LICENSE.