Skip to main content

An AWS S3-hosted SQLite database backend for Django.

Project description

django-s3-sqlite

PyPI

This project was inspired and started with Zappa. Thanks to Rich Jones for all of his amazing work.

Installation

Install via pip:

$ pip install django-s3-sqlite

Add to your installed apps:

INSTALLED_APPS += ["django_s3_sqlite"]

Using an S3-Backed Database Engine

django-s3-sqlite allows use of an S3-synced SQLite database as a Django database engine.

This will cause problems for applications with concurrent writes**, but it scales very well for high-read applications that don't have concurrent writes (like a CMS for your blog), and it's orders of magnitude cheaper than AWS RDS or Aurora (pennies per month instead of many dollars per month).

** Concurrent writes will often be lost and not show up in concurrent readers. This is because the database is transferred between S3 storage and the Lambda instance for each request.

Django Settings & Commands

DATABASES = {
    "default": {
        "ENGINE": "django_s3_sqlite",
        "NAME": "sqlite.db",
        "BUCKET": "your-db-bucket",
    }
}

Newer versions of Django (v2.1+) require a newer version of SQLite (3.8.3+) than is available on AWS Lambda instances (3.7.17).

Because of this, you will need to download the provided _sqlite3.so for your Python version (available in the shared-objects directory of this repository) and put it at the root of your Django project. Note that only the Python 3.6 and 3.8 runtimes are supported at this time. These shared object files contain a compiled binary static build of SQLite 3.30.1 that can be used with the corresponding version of Python. We hope this will soon be available via Lambda Packages, but for now, you will also need to add this line to your Zappa JSON settings file in each environment:

"use_precompiled_packages": false,

Since SQLite keeps the database in a single file, you will want to keep it as small and defragmented as possible. It is good to occasionally perform a database vacuum, especially after deleting or updating data. There's a command to vacuum your database:

zappa manage [instance] s3_sqlite_vacuum

Creating a Default Admin User

You'll probably need a default user to manage your application with, so you can now:

$ zappa manage <stage> create_admin_user

Or you can pass some arguments:

$ zappa manage <stage> create_admin_user one two three

This will internally make this call:

User.objects.create_superuser('one', 'two', 'three')

Release Notes

On GitHub: https://github.com/FlipperPA/django-s3-sqlite/releases

Maintainers and Creator

This package is largely maintained by the staff of Wharton Research Data Services. We are thrilled that The Wharton School allows us a certain amount of time to contribute to open-source projects. We add features as they are necessary for our projects, and try to keep up with Issues and Pull Requests as best we can. Due to time constraints (our full time jobs!), Feature Requests without a Pull Request may not be implemented, but we are always open to new ideas and grateful for contributions and our package users.

Contributors - Thank You!

Build Instructions for _sqlite3.so

If you'd like to use a different version of Python or SQLite than what is provided in this repo, you will need to build the static binary yourself. These instructions show you how to build the file: https://charlesleifer.com/blog/compiling-sqlite-for-use-with-python-applications/

After you've downloaded SQLite, follow the instructions to install it into a virtual environment. You must perform the installation on Amazon Linux or CentOS 7 (which Amazon Linux is based on).

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for django-s3-sqlite, version 0.0.4
Filename, size File type Python version Upload date Hashes
Filename, size django_s3_sqlite-0.0.4-py3-none-any.whl (8.3 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size django-s3-sqlite-0.0.4.tar.gz (4.7 MB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page