Flatseek

Powerful disk-first full-text search without the infrastructure.

Explore, filter, and search structured data at any scale — from live application data to historical archives, and AI-ready datasets.

Demo: flatlens.demo.flatseek.io  ·  Docs: flatseek.io/docs  ·  Dashboard: Flatlens  ·  Benchmark: flatbench  ·  Hosted Datasets: HuggingFace

See it work

Search a 14 GB index hosted on HuggingFace through a serverless API running with only 2 GB of memory.

# demo CLI
flatseek search https://huggingface.co/datasets/flatseek/public-dataset/resolve/main/6.3M-books.fsk \
  "title:dune"

[reading remote index] ✓
[searching remote] ✓

Found: 1,018 matches

{'name':'Statisticity','author':['Yaron Glazer'],'genres':['Science Fiction'],'...':'...'}

# demo API

curl -X 'GET' \
  'https://api.demo.flatseek.io/6.3M-books/_search?q=dune&from=0&size=20&bucket=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Fflatseek%2Fpublic-dataset' \
  -H 'accept: application/json'

{"hits":{"total":1018,"hits":[{"_source":{"name":"Statisticity","author":["Yaron Glazer"],"...":"..."}}]},"took":1}

That query runs against 6.3M books dataset hosted on HuggingFace, and only pulls the byte ranges it needs to answer it. The same index also powers the Flatlens - live dashboard — try it with zero installation.

The problem

Distributed search engines like Elasticsearch and OpenSearch are excellent at large-scale production systems—but they assume always-on infrastructure.

For many workloads, that means paying for 24/7 servers, storage, replicas, and operational overhead even when you're simply searching documents.

Whether you're powering a production API, exploring a dataset, preparing training data, building a RAG pipeline, or publishing a searchable archive, the infrastructure often costs more than the search itself.

Lightweight alternatives avoid the cluster, but they're usually limited to local files. If your data lives in S3, HuggingFace, or object storage, you still need to download it before you can search it, making remote search harder than it should be.

What Flatseek does differently

Flatseek uses one query language across two index formats, so the same search engine works for both live production data and portable datasets.

Both formats expose the same query language, embedded library, CLI, and REST API—switching between mutable production data and immutable archives doesn't require changing your application.

The .fsk format packages an entire index into a single portable file, making it easy to publish on object storage, CDN, or platforms with file-count limits while preserving the same query capabilities.

Performance at a glance

500K documents • Article schema • SSD • Compared against Elasticsearch

Full comparison including tantivy, Typesense, Whoosh, ZincSearch: docs/benchmark.md or bench.flatseek.io

Core capabilities

See docs/ for full details.

Installation

Recommended — one-liner

curl -fsSL flatseek.io/install.sh | sh

Includes: • CLI • REST API • Flatlens dashboard (http://localhost:8000/dashboard)

PyPI

pip install flatseek

CLI only. For the Flatlens dashboard:

git clone https://github.com/flatseek/flatlens

From source

git clone https://github.com/flatseek/flatseek.git
cd flatseek
pip install -e .

Requirements: Python ≥ 3.10, macOS / Linux / WSL.

Quick start

# Generate 100K dummy data
flatseek generate -r 100000 -s article -f csv -o ./data.csv

# Build index
flatseek build ./data.csv -o ./data

# Query via CLI
flatseek search ./data "program:raydium AND amount:>1000000"

# Serve API + dashboard
flatseek serve -d ./data

# Pack index into single portable .fsk file
flatseek pack ./data -o ./data.fsk

# Query from portable file
flatseek search data.fsk "program:raydium AND amount:>1000000"

# Serve API + dashboard from portable file
flatseek serve data.fsk

# Query a .fsk archive directly from HTTP — no full download
flatseek search https://huggingface.co/datasets/owner/repo/resolve/main/data.fsk "program:raydium AND amount:>1000000"

# Serve a .fsk archive directly from HTTP — no full download
flatseek serve https://huggingface.co/datasets/owner/repo/resolve/main/data.fsk

# Export results
flatseek export ./data "program:raydium AND amount:>1000000" -f jsonl --out exported.jsonl
flatseek export data.fsk "program:raydium AND amount:>1000000" -f jsonl --out exported.jsonl

→ API: http://localhost:8000 → Dashboard: http://localhost:8000/dashboard

Remote datasets

Build an index once, upload it to any HTTP-accessible storage, and query it from anywhere using the same embedded library, CLI, or REST API.

Flatseek reads only the byte ranges required to answer each query, so even large indexes can be searched remotely without downloading the entire file.

Supported providers:

• HuggingFace Datasets / Buckets
• S3-compatible storage (Amazon S3, MinIO, Cloudflare R2)
• Vercel Blob
• Any static HTTP server or CDN

Try it now: Use the live dashboard at flatlens.demo.flatseek.io to explore remote .fsk indexes directly from your browser—no installation required.

Example indexed datasets

All datasets below are hosted on HuggingFace. The Flatseek API serving the dashboard is deployed on a free Vercel hobby account — and it searches a 14 GB index in under 5 seconds by fetching only the byte ranges it needs.

Distribution & Licensing

Flatseek supports four distribution models, depending on your distribution and access-control requirements.

Key difference — time-limited vs. license-based: when a time-limited archive expires, you must repack it from existing non-fsk index to extend access. When a license-based archive expires, you issue a new HMAC token — the index itself stays unchanged.

The credentials below are public demo credentials for the samples above — not secrets.

Full Documentation

Contributing

PRs welcome. Run tests:

pytest src/flatseek/test/test_search.py -v   # accuracy tests
pytest src/flatseek/test/test_api.py         # API smoke tests
pytest src/flatseek/test/test_cli.py         # CLI integration

License

Apache 2.0. See LICENSE.