About
BSDict is a dictionary for Python that supports the following objects as keys:
- None,
- bool, int, float (including math.nan), complex,
- str, bytes, bytearray,
- list, tuple, dict, set, frozenset,
- numpy.ndarray (bool, int, float, complex).
Internally, a total lexicographical order is defined over all supported objects. A sorted array is then used for storage, and binary search is used for element lookup.
BSDict can be initialized in-memory or on-disc (persistent storage). If BSDict is initialized on-disc, then keys and values are stored in memory only as long as the client application has any references to them. So, the dictionary can be larger than available RAM.
Basic Usage
In-memory dictionary:
from bsdict import bsdict
import numpy as np
data = bsdict()
key = {'s1': np.array([1.0, 10.0, np.nan])}
data[key] = 5
data
On-disc dictionary:
from bsdict import bsdict
data = bsdict(datadir = 'cache')
# ...
data.clear()
Memoization
BSDict has been originally written to help memoize functions that accept complex data structures, including floating-point data, as arguments. (Such functions are common in data analysis.) This package includes a simple memoization wrapper that uses BSDict for caching the results.
from time import sleep
from bsdict import memoizer
cached = memoizer(verbose = True)
# Persistent memoization
#cached = memoizer(datadir = 'cache', verbose = True)
@cached
def mysum(x, y):
print("Computing a challenging math problem...")
sleep(1)
return x + y
z = mysum(1, 2)
z = mysum(1, 2)
Warning
This is the first official release. Bug reports are welcome. (There are extensive test suits in the package, however.)
Technical Notes
A simpler way to support arbitrary keys would be to pickle them and store their binary representation in a dictionary. There are two minor issues with approach. Firstly, identical dictionaries and sets might be serialized differently depending on the order in which they were composed (So they need to be recursively sorted). Secondly, numerical algorithms than run on multiple processes or on multiple machines might break floating-point determinism. If it desirable to consider nearly identical numbers as the same number, then with binary search that would possible, while with serialization that won't work. Then again, lexicographical ordering of vectors in inconsistent with near-match lookups, meaning that it would work fine often but not always. If you need near-match lookups, let me know, I'll add them as an option then.
Metadata
Release files for bsdict 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| bsdict-0.1.0.tar.gz | 7.7 kB | Without provenance | Details for bsdict-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| bsdict-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for bsdict-0.1.0-py3-none-any.whl |
Total release size: 17.8 kB
Release files / bsdict-0.1.0.tar.gz
| Download URL | bsdict-0.1.0.tar.gz |
|---|---|
| Size | 7.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d6898c8983b494cb1e33bac0140503e8cbf40627fda3a6a23fc6772b2d724e98
|
|
BLAKE2b-256 checksum How to use checksums |
6a26b1230ebe7ec91fe548d28909bc194626bf1e91439daa1a29e627dfba2371
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.28.1 CPython/3.7.3
|
Release files / bsdict-0.1.0-py3-none-any.whl
| Download URL | bsdict-0.1.0-py3-none-any.whl |
|---|---|
| Size | 10.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d0012aa5c2e766ebd1e9918978738cfc131d30ac7904101a86deecab2dcd17a4
|
|
BLAKE2b-256 checksum How to use checksums |
1f7998b51270cef2415347ad3bd32d61e78837c3d8ab483320f28c0357c6df33
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.28.1 CPython/3.7.3
|
Security for bsdict 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.