Lazy dict
Project description
ldict
A lazy dict.
Latest release | Current code | API documentation
See also
Overview
A ldict
is a dict
with str
keys.
Simple usage example
from ldict import ldict
a = ldict(x=3)
print(a)
"""
{
"x": 3
}
"""
b = ldict(y=5)
print(b)
"""
{
"y": 5
}
"""
print(a >> b)
"""
{
"x": 3,
"y": 5
}
"""
We consider that every value is generated by a process, starting from an empty
ldict. The process is a sequence of
transformation steps done through the operator >>
, which symbolizes a data flow. There are two types of steps:
- value insertion - represented by dict-like objects
- function application - represented by ordinary python functions
A ldict
is completely defined by its key-value pairs so that
it can be converted from/to a built-in dict.
Creating a ldict is not different from creating an ordinary dict. Optionally it can be created through the >>
operator
used after empty
:
Function application is done in the same way. The parameter names define the input fields, while the keys in the returned dict define the output fields:
Similarly, for anonymous functions:
Finally, the result is only evaluated at request:
Installation
...as a standalone lib
# Set up a virtualenv.
python3 -m venv venv
source venv/bin/activate
# Install from PyPI...
pip install --upgrade pip
pip install -U ldict
# ...or, install from updated source code.
pip install git+https://github.com/davips/ldict
...from source
git clone https://github.com/davips/ldict
cd ldict
poetry install
Examples
Merging two ldicts
from ldict import ldict
a = ldict(x=3)
print(a)
"""
{
"x": 3
}
"""
b = ldict(y=5)
print(b)
"""
{
"y": 5
}
"""
print(a >> b)
"""
{
"x": 3,
"y": 5
}
"""
Lazily applying functions to ldict
from ldict import ldict
a = ldict(x=3)
print(a)
"""
{
"x": 3
}
"""
a = a >> ldict(y=5) >> {"z": 7} >> (lambda x, y, z: {"r": x ** y // z})
print(a)
"""
{
"x": 3,
"y": 5,
"z": 7,
"r": "→(x y z)"
}
"""
print(a.r)
"""
34
"""
print(a)
"""
{
"x": 3,
"y": 5,
"z": 7,
"r": 34
}
"""
Parameterized functions and sampling
from random import Random
from ldict import empty, let
# A function provide input fields and, optionally, parameters.
# For instance:
# 'a' is sampled from an arithmetic progression
# 'b' is sampled from a geometric progression
# Here, the syntax for default parameter values is borrowed with a new meaning.
def fun(x, y, a=[-100, -99, -98, ..., 100], b=[0.0001, 0.001, 0.01, ..., 100000000]):
return {"z": a * x + b * y}
def simplefun(x, y):
return {"z": x * y}
# Creating an empty ldict. Alternatively: d = ldict().
d = empty >> {}
print(d)
"""
{}
"""
# Putting some values. Alternatively: d = ldict(x=5, y=7).
d["x"] = 5
d["y"] = 7
print(d)
"""
{
"x": 5,
"y": 7
}
"""
# Parameter values are uniformly sampled.
d1 = d >> simplefun
print(d1)
print(d1.z)
"""
{
"x": 5,
"y": 7,
"z": "→(x y)"
}
35
"""
d2 = d >> simplefun
print(d2)
print(d2.z)
"""
{
"x": 5,
"y": 7,
"z": "→(x y)"
}
35
"""
# Parameter values can also be manually set.
e = d >> let(fun, a=5, b=10)
print(e.z)
"""
95
"""
# Not all parameters need to be set.
e = d >> let(simplefun, a=5)
print(e.z)
"""
35
"""
# Each run will be a different sample for the missing parameters.
e = e >> let(simplefun, a=5)
print(e.z)
"""
35
"""
# We can define the initial state of the random sampler.
# It will be in effect from its location place onwards in the expression.
e = d >> Random(0) >> let(fun, a=5)
print(e.z)
"""
725.0
"""
# All runs will yield the same result,
# if starting from the same random number generator seed.
e = e >> Random(0) >> let(fun, a=5)
print(e.z)
"""
725.0
"""
# Reproducible different runs are achievable by using a single random number generator.
rnd = Random(0)
e = d >> rnd >> let(fun, a=5)
print(e.z)
e = d >> rnd >> let(fun, a=5) # Alternative syntax.
print(e.z)
"""
725.0
700000025.0
"""
Composition of sets of functions
from random import Random
from ldict import empty
# A multistep process can be defined without applying its functions
def g(x, y, a=[1, 2, 3, ..., 10], b=[0.00001, 0.0001, 0.001, ..., 100000]):
return {"z": a * x + b * y}
def h(z, c=[1, 2, 3]):
return {"z": c * z}
# In the ldict framework 'data is function',
# so the alias ø represents the 'empty data object' and the 'reflexive function' at the same time.
# In other words: 'inserting nothing' has the same effect as 'doing nothing'.
fun = empty >> g >> h # empty enable the cartesian product of the subsequent sets of functions within the expression.
print(fun)
"""
«<function g at 0x7fa606ce29d0> × <function h at 0x7fa6071c98b0>»
"""
# The difference between 'ø * g * h' and 'ldict(x=3) >> g >> h' is that the functions in the latter are already applied
# (resulting in an ldict object). The former still has its free parameters unsampled,
# and results in an ordered set of composite functions.
# It is a set because the parameter values of the functions are still undefined.
d = {"x": 5, "y": 7} >> (Random(0) >> fun)
print(d)
"""
{
"x": 5,
"y": 7,
"z": "→(c z→(a b x y))"
}
"""
print(d.z)
"""
105.0
"""
d = {"x": 5, "y": 7} >> (Random(0) >> fun)
print(d.z)
"""
105.0
"""
# Reproducible different runs by passing a stateful random number generator.
rnd = Random(0)
e = d >> rnd >> fun
print(e.z)
"""
105.0
"""
e = d >> rnd >> fun
print(e.z)
"""
14050.0
"""
# Repeating the same results.
rnd = Random(0)
e = d >> rnd >> fun
print(e.z)
"""
105.0
"""
e = d >> rnd >> fun
print(e.z)
"""
14050.0
"""
Concept
A ldict is like a common Python dict, with extra functionality and lazy. It is a mapping between string keys, called
fields, and any serializable (pickable) object. The ldict id
(identifier) and the field ids
are also part of the
mapping.
The user can provide a unique identifier (hosh) for each function or value object. Otherwise, they will be calculated through blake3 hashing of the content of data or bytecode of function. For this reason, such functions should be simple, i.e., with minimal external dependencies, to avoid the unfortunate situation where two functions with identical local code actually perform different calculations through calls to external code that implement different algorithms with the same name.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ldict-3.211017.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b39d96500b889de2eda2a27b6cb831ecf25f54085004c0d0b434ac55d59143f2 |
|
MD5 | 77f8b1359a85f351b1a8684443fa375b |
|
BLAKE2b-256 | 4e98dc2100b2917898c3d98476a451b907b0033db189de8418ed637e1e8898ea |