datasette-copy-to-memory

Copy database files into an in-memory database on startup

This plugin is highly experimental. It currently exists to support Datasette performance research, and is not designed for actual production usage.

Installation

Install this plugin in the same environment as Datasette.

datasette install datasette-copy-to-memory

Usage

On startup, Datasette will create an in-memory named database for each attached database. This database will have the same name but with _memory at the end.

So running this:

datasette fixtures.db

Will serve two databases: the original at /fixtures and the in-memory copy at /fixtures_memory.

Demo

A demo is running on latest-with-plugins.datasette.io - the /fixtures_memory table there is provided by this plugin.

Configuration

By default every attached database file will be loaded into a _memory copy.

You can use plugin configuration to specify just a subset of the database. For example, to create github_memory but not fixtures_memory you would use the following metadata.yml file:

plugins:
  datasette-copy-to-memory:
    databases:
    - github

Then start Datasette like this:

datasette github.db fixtures.db -m metadata.yml

If you don't want to have a fixtures and fixtures_memory database, you can use replace: true to have the plugin replace the file-backed database with the new in-memory one, reusing the same database name:

plugins:
  datasette-copy-to-memory:
    replace: true

Then:

datasette github.db fixtures.db -m metadata.yml

This will result in both /github and /fixtures but no /github_memory or /fixtures_memory.

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd datasette-copy-to-memory
python3 -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

pip install -e '.[test]'

To run the tests:

pytest