pydq
A python library for managing and persisting named Python Queues
Installation
pip install pydq
Some backends may require extra packages:
pip install pydq[aws]
Basic Usage
with dq_provider('name') as dq:
# Query the last 10 entries with qid='item_id'
dq(qid='item_id', start_time=datetime(1, 1, 1), end_time=datetime.utcnow(), limit=10)
item = dq.get()
# queue items should be dictionaries with 3 keys: qid, ts, and val
dq.put({'qid': 'item_id', 'ts': '2019-09-11T03:23:54Z', 'val': 'foo'})
note: specifying qid in the query is optional, if omitted all items between the start and end time are returned
Supported Backends:
Local Storage
- SQLite:
from pydq.local import SQLite as dq_provider
Cloud Storage
- DynamoDB
from pydq.cloud import DynamoDB as dq_provider
note: cloud storage is created lazily; the first call to one of these providers will create the resources in the cloud.
Additional API
Some queue methods accept a forget argument that will tell the queue object whether the operation should be persisted. This always defaults to False.
dq.get(forget=True) # the returned item will be deleted from persistence
dq.put(item, forget=True) # item will not be persisted
dq.get_all(forget=True) # Easy way to delete all items in a queue
dq.put_all(items, forget=False)
dq.get_to_dataframe() # install with pandas extra. Returned items are not deleted
dq.put_from_dataframe(df) # items will be persisted
dq.clear() # another way to delete all items in a queue
Configuration queues
dq is great for managing simple JSON configuration too. And we get change tracking for free!
from pydq.helpers import DqConfig
from pydq.local import SQLite
with DqConfig(name='ApplicationConfig', dq_provider=SQLite) as config:
app_id = config['app_id']
last_start = config['last_start']
config['app_status'] = '#1best'
1 maintainer
Metadata
1 maintainer
Release files for pydq 0.1.20
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pydq-0.1.20-py3-none-any.whl | Python 3 | none | any | Details |
Release files / pydq-0.1.20-py3-none-any.whl
| Download URL | pydq-0.1.20-py3-none-any.whl |
|---|---|
| Size | 11.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0f8ea5619fa2e58840cfcb0d47943954d368df0c06b6334f936d735fce582bc5
|
|
BLAKE2b-256 checksum How to use checksums |
6c9dc891d235d35abe63bd2362f8faddcd0e3a4fe3ead4702334e8b220ab29b5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.9.6
|
Security for pydq 0.1.20
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
