Skip to main content

Create and read fast constant databases

Project description

Introduction

speedyb lets you create and read constant databases.

  • fast lookups
  • database cannot be modified
  • database can be shared by multiple processes without using more memory
  • fast startup time

Let’s say you have a dict like this:

d = {
  'Alice': '26 Pine Street',
  'Bob':   '54 Oak Drive'
}

You can write this to a database file: speedyb_writer.dict_to_file(d, ‘my.db’)

Questions

Q: Why not store the dict in a JSON or pickle file and read into a python dict?

A: First, each process that loads the dict would allocate separate memory, while speedyb shares the memory. Second, for a large db the load time can be significant, while with speedyb it is essentially instant.

Q: How does speedyb compare to Berkeley DB, LevelDB, etc?

A: It is significantly faster because it’s constant.

Installation

Checkout the source and run python setup.py install.

Using speedyb

Creating a DB

import speedyb_writer
d = { ... } # data to store
speedyb_writer.dict_to_file(d, 'new.db')

This creates a new database file containing d. It may be slow. File creation time goes up linearly with number of keys in d.

Reading a DB

from speedyb_reader import speedyb_reader
db = speedyb_reader('tiny.tmp') # open file
res = db.get('Alice') # lookup a key

If the key is not present, res is None.

$Id: README.rst,v 1.2 2012-06-04 04:59:31 asher Exp $

Project details


Release history Release notifications

This version
History Node

0.1

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page