A database for processing and storing datasets based on LevelDB
Pypeline DB is designed to simplify the creation and management of datasets. It has a friendly and easy-to-master API backed by the power of LevelDB. This allows it to manage datasets too large to fit in RAM without sacrificing data access performance.
Pypeline is great for:
- Exploring data without eating all your RAM
- Transforming data with maps, filters and reductions
- Stopping you from losing or overwriting your data (unless you explicitly ask it to)
It’s also super easy to export a dataset from Pypeline to Pandas for further analysis.