A NoSQL Database Implementation using Amazon S3
Project description
S3NoSQLDB
A NoSQL Database Implementation using Amazon S3 for storage.
Features
- Create and manage collections in S3 buckets
- Batch document insertion with automatic ID generation
- Concurrent processing using ThreadPoolExecutor
- Query documents using custom filter functions
- Parquet file format with gzip compression
Installation
pip install s3nosqldb
Quick Start
from s3nosqldb import S3NoSQLDB
# Initialize the database
db = S3NoSQLDB(
bucket_name="your-bucket-name",
base_path="optional/path",
batch_size=50000,
max_workers=4
)
# Create a collection
db.create_collection("users", id_field="user_id", auto_generate_id=True)
# Insert documents
users = [
{"name": "John Doe", "age": 30},
{"name": "Jane Smith", "age": 25}
]
db.insert_batch("users", users)
# Query documents
def age_filter(doc):
return doc["age"] > 25
results = db.query("users", filter_fn=age_filter)
Development
- Clone the repository:
git clone https://github.com/yourusername/s3nosqldb.git
cd s3nosqldb
- Create a virtual environment and install dependencies:
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -e .[dev]
- Run tests:
pytest
License
This project is licensed under the MIT License - see the LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
s3nosqldb-0.1.0.tar.gz
(5.6 kB
view hashes)
Built Distribution
Close
Hashes for s3nosqldb-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97c10f5db61d52c6aff81c1eaaed7c07f1e9a58de1d1fc182600304c43668dbb |
|
MD5 | 042b798e0239dd531984b386b9bcc3c1 |
|
BLAKE2b-256 | 9fe4c2c66dd9fc4d5c46878a6186dacb68dc37d84dc81f1aa44c1e24508b00d8 |