Skip to main content

About

BSDict is a dictionary for Python that supports the following objects as keys:

  • None,
  • bool, int, float (including math.nan), complex,
  • str, bytes, bytearray,
  • list, tuple, dict, set, frozenset,
  • numpy.ndarray (bool, int, float, complex).

Internally, a total lexicographical order is defined over all supported objects. A sorted array is then used for storage, and binary search is used for element lookup.

BSDict can be initialized in-memory or on-disc (persistent storage). If BSDict is initialized on-disc, then keys and values are stored in memory only as long as the client application has any references to them. So, the dictionary can be larger than available RAM.

Basic Usage

In-memory dictionary:

from bsdict import bsdict
import numpy as np
data = bsdict()
key = {'s1': np.array([1.0, 10.0, np.nan])}
data[key] = 5
data

On-disc dictionary:

from bsdict import bsdict
data = bsdict(datadir = 'cache')
# ...
data.clear()

Memoization

BSDict has been originally written to help memoize functions that accept complex data structures, including floating-point data, as arguments. (Such functions are common in data analysis.) This package includes a simple memoization wrapper that uses BSDict for caching the results.

from time import sleep
from bsdict import memoizer

cached = memoizer(verbose = True)

# Persistent memoization
#cached = memoizer(datadir = 'cache', verbose = True)

@cached
def mysum(x, y):
    print("Computing a challenging math problem...")
    sleep(1)
    return x + y

z = mysum(1, 2)
z = mysum(1, 2)

Warning

This is the first official release. Bug reports are welcome. (There are extensive test suits in the package, however.)

Technical Notes

A simpler way to support arbitrary keys would be to pickle them and store their binary representation in a dictionary. There are two minor issues with approach. Firstly, identical dictionaries and sets might be serialized differently depending on the order in which they were composed (So they need to be recursively sorted). Secondly, numerical algorithms than run on multiple processes or on multiple machines might break floating-point determinism. If it desirable to consider nearly identical numbers as the same number, then with binary search that would possible, while with serialization that won't work. Then again, lexicographical ordering of vectors in inconsistent with near-match lookups, meaning that it would work fine often but not always. If you need near-match lookups, let me know, I'll add them as an option then.

Metadata

Release files for bsdict 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for bsdict 0.1.0
File Size Uploaded Provenance Actions
bsdict-0.1.0.tar.gz 7.7 kB Without provenance Details for bsdict-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for bsdict 0.1.0
File Interpreter ABI Platform Actions
bsdict-0.1.0-py3-none-any.whl Python 3 none any Details for bsdict-0.1.0-py3-none-any.whl

Total release size: 17.8 kB

Release files / bsdict-0.1.0.tar.gz

Download URL bsdict-0.1.0.tar.gz
Size 7.7 kB
Tags Source
SHA-256 checksum
How to use checksums
d6898c8983b494cb1e33bac0140503e8cbf40627fda3a6a23fc6772b2d724e98
BLAKE2b-256 checksum
How to use checksums
6a26b1230ebe7ec91fe548d28909bc194626bf1e91439daa1a29e627dfba2371
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.28.1 CPython/3.7.3

Release files / bsdict-0.1.0-py3-none-any.whl

Download URL bsdict-0.1.0-py3-none-any.whl
Size 10.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d0012aa5c2e766ebd1e9918978738cfc131d30ac7904101a86deecab2dcd17a4
BLAKE2b-256 checksum
How to use checksums
1f7998b51270cef2415347ad3bd32d61e78837c3d8ab483320f28c0357c6df33
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.28.1 CPython/3.7.3

Security for bsdict 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page