Skip to main content

SharedData

SharedData is an open-source memory-mapped store for records and time series — in the workload category of tick stores such as kdb+ — plus the scheduler and worker fleet that produce the data. Local processes map the same files through the operating system's page cache; indexed writes coordinate through a per-table process lock. Remote clients use an authenticated HTTP API; shared memory does not span hosts.

Website and manual: https://shareddata.net

Install

Python 3.11 or 3.12:

python -m pip install shareddata

The quick start uses illustrative records in isolated storage. See configuration for public environment names and Windows limitations for native requirements and rewrite behavior.

A record in one minute

Save as example.py after setting up isolated storage in the quick start, then run python example.py:

from SharedData.SharedData import SharedData
import pandas as pd

store = SharedData(__file__, user="example")
index = pd.MultiIndex.from_tuples(
    [(pd.Timestamp("2026-01-02"), "EXAMPLE")],
    names=["date", "symbol"],
)
rows = pd.DataFrame({"close": [100.0]}, index=index)
records = store.table(
    "MarketData", "D1", "DEMO", "PRICES",
    value=rows,
)
print(records.loc["2026-01-02", "EXAMPLE"])

Containers and execution

  • Tables: structured NumPy records and primary-key lookup.
  • Time series: timestamped float matrices.
  • Collections, Redis caches, Kafka streams and optional S3 integration.
  • HTTP API: authenticated remote access.
  • Measured latency: what the containers cost in practice.

A store and the fleet that fills it

A routine is any program the Scheduler owns — a feed collector, a parser, a consolidation job, an API server. A schedule lists the routines that should stay alive or run at their times. Batch routines wait for their batch dependencies to complete; realtime routines wait for their realtime dependencies to be running. Dependencies between the two kinds are not enforced at runtime. Commands travel through a per-host worker pool that validates the requested source revision before every launch; routines report heartbeats and lifecycle messages through the logging plane, and the Scheduler writes queryable state into the store.

Data by symbol

The semantic data API resolves a symbol or entity to a dataset and includes provenance. It is a convenience, not the fast path: producers keep using direct container interfaces for performance-critical work.

Client integration

Client integration covers authenticated API access, the MCP server for compatible agent clients, and worker access.

Development and license

Submit changes through the repository's review process. Read the development guide and run the checks relevant to your change; building and publishing explains how the public site is produced and audited. SharedData is released under the MIT license; see LICENSE in the source distribution.

Metadata

Release files for shareddata 6.101.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for shareddata 6.101.0
File Size Uploaded Provenance Actions
shareddata-6.101.0.tar.gz 978.0 kB Without provenance Details for shareddata-6.101.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for shareddata 6.101.0
File Interpreter ABI Platform Actions
shareddata-6.101.0-py3-none-any.whl Python 3 none any Details for shareddata-6.101.0-py3-none-any.whl

Total release size: 2.1 MB

Release files / shareddata-6.101.0.tar.gz

Download URL shareddata-6.101.0.tar.gz
Size 978.0 kB
Tags Source
SHA-256 checksum
How to use checksums
d00b1629726006f542ebbb2adf572b9319b164195893ac2b7acafdb769422a9e
BLAKE2b-256 checksum
How to use checksums
4c74d2af697457cabfcf2e6dbce5fbc0b78f527e133b8f371a197db5275f114b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.0.1 CPython/3.12.3

Release files / shareddata-6.101.0-py3-none-any.whl

Download URL shareddata-6.101.0-py3-none-any.whl
Size 1.1 MB
Tags Python 3
SHA-256 checksum
How to use checksums
d2438075994a3bfc9671ee76dd4bea64027cfd74e103dcf45efd26e779ffa3de
BLAKE2b-256 checksum
How to use checksums
a5ed052c176b67b2dc974da4f16ceba88f75926d468d3a25b0ceb7865cd36fa2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.0.1 CPython/3.12.3

Security for shareddata 6.101.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

6.101.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page