Skip to main content

SharedData

SharedData is an open-source memory-mapped store for records and time series — in the workload category of tick stores such as kdb+ — plus the scheduler and worker fleet that produce the data. Local processes map the same files through the operating system's page cache; indexed writes coordinate through a per-table process lock. Remote clients use an authenticated HTTP API; shared memory does not span hosts.

Website and manual: https://shareddata.net

Install

Python 3.11 or 3.12:

python -m pip install shareddata

The quick start uses illustrative records in isolated storage. See configuration for public environment names and Windows limitations for native requirements and rewrite behavior.

A record in one minute

Save as example.py after setting up isolated storage in the quick start, then run python example.py:

from SharedData.SharedData import SharedData
import pandas as pd

store = SharedData(__file__, user="example")
index = pd.MultiIndex.from_tuples(
    [(pd.Timestamp("2026-01-02"), "EXAMPLE")],
    names=["date", "symbol"],
)
rows = pd.DataFrame({"close": [100.0]}, index=index)
records = store.table(
    "MarketData", "D1", "DEMO", "PRICES",
    value=rows,
)
print(records.loc["2026-01-02", "EXAMPLE"])

Containers and execution

  • Tables: structured NumPy records and primary-key lookup.
  • Time series: timestamped float matrices.
  • Collections, Redis caches, Kafka streams and optional S3 integration.
  • HTTP API: authenticated remote access.
  • Measured latency: what the containers cost in practice.

A store and the fleet that fills it

A routine is any program the Scheduler owns — a feed collector, a parser, a consolidation job, an API server. A schedule lists the routines that should stay alive or run at their times. Batch routines wait for their batch dependencies to complete; realtime routines wait for their realtime dependencies to be running. Dependencies between the two kinds are not enforced at runtime. Commands travel through a per-host worker pool that validates the requested source revision before every launch; routines report heartbeats and lifecycle messages through the logging plane, and the Scheduler writes queryable state into the store.

Data by symbol

The semantic data API resolves a symbol or entity to a dataset and includes provenance. It is a convenience, not the fast path: producers keep using direct container interfaces for performance-critical work.

Client integration

Client integration covers authenticated API access, the MCP server for compatible agent clients, and worker access.

Development and license

Submit changes through the repository's review process. Read the development guide and run the checks relevant to your change; building and publishing explains how the public site is produced and audited. SharedData is released under the MIT license; see LICENSE in the source distribution.

Metadata

Release files for shareddata 6.86.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for shareddata 6.86.2
File Size Uploaded Provenance Actions
shareddata-6.86.2.tar.gz 778.2 kB Without provenance Details for shareddata-6.86.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for shareddata 6.86.2
File Interpreter ABI Platform Actions
shareddata-6.86.2-py3-none-any.whl Python 3 none any Details for shareddata-6.86.2-py3-none-any.whl

Total release size: 1.6 MB

Release files / shareddata-6.86.2.tar.gz

Download URL shareddata-6.86.2.tar.gz
Size 778.2 kB
Tags Source
SHA-256 checksum
How to use checksums
532c77e44e73756f7a0898a9eabc5cfdb6bfdd4ec21d8b4a64c30c4c73066985
BLAKE2b-256 checksum
How to use checksums
132f517413edf6976c07879e295acd985a2d984a120d63d317c5293152cd7bdf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.0.1 CPython/3.12.3

Release files / shareddata-6.86.2-py3-none-any.whl

Download URL shareddata-6.86.2-py3-none-any.whl
Size 857.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e698fed7bc7cfce3ee7e679f7c9cbb90772d8ba9794b03de96db28e30265f897
BLAKE2b-256 checksum
How to use checksums
e9b472db015334023caaf62a6452428ca3372c26f231676d7105b13010acbbb1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.0.1 CPython/3.12.3

Security for shareddata 6.86.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

6.86.2 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page