Skip to main content

SharedData

SharedData is an open-source memory-mapped store for records and time series — in the workload category of tick stores such as kdb+ — plus the scheduler and worker fleet that produce the data. Local processes map the same files through the operating system's page cache; indexed writes coordinate through a per-table process lock. Remote clients use an authenticated HTTP API; shared memory does not span hosts.

Website and manual: https://shareddata.net

Install

Python 3.11 or 3.12:

python -m pip install shareddata

The quick start uses illustrative records in isolated storage. See configuration for public environment names and Windows limitations for native requirements and rewrite behavior.

A record in one minute

Save as example.py after setting up isolated storage in the quick start, then run python example.py:

from SharedData.SharedData import SharedData
import pandas as pd

store = SharedData(__file__, user="example")
index = pd.MultiIndex.from_tuples(
    [(pd.Timestamp("2026-01-02"), "EXAMPLE")],
    names=["date", "symbol"],
)
rows = pd.DataFrame({"close": [100.0]}, index=index)
records = store.table(
    "MarketData", "D1", "DEMO", "PRICES",
    value=rows,
)
print(records.loc["2026-01-02", "EXAMPLE"])

Containers and execution

  • Tables: structured NumPy records and primary-key lookup.
  • Time series: timestamped float matrices.
  • Collections, Redis caches, Kafka streams and optional S3 integration.
  • HTTP API: authenticated remote access.
  • Measured latency: what the containers cost in practice.

A store and the fleet that fills it

A routine is any program the Scheduler owns — a feed collector, a parser, a consolidation job, an API server. A schedule lists the routines that should stay alive or run at their times. Batch routines wait for their batch dependencies to complete; realtime routines wait for their realtime dependencies to be running. Dependencies between the two kinds are not enforced at runtime. Commands travel through a per-host worker pool that validates the requested source revision before every launch; routines report heartbeats and lifecycle messages through the logging plane, and the Scheduler writes queryable state into the store.

Data by symbol

The semantic data API resolves a symbol or entity to a dataset and includes provenance. It is a convenience, not the fast path: producers keep using direct container interfaces for performance-critical work.

Client integration

Client integration covers authenticated API access, the MCP server for compatible agent clients, and worker access.

Development and license

Submit changes through the repository's review process. Read the development guide and run the checks relevant to your change; building and publishing explains how the public site is produced and audited. SharedData is released under the MIT license; see LICENSE in the source distribution.

Metadata

Release files for shareddata 6.100.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for shareddata 6.100.0
File Size Uploaded Provenance Actions
shareddata-6.100.0.tar.gz 951.5 kB Without provenance Details for shareddata-6.100.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for shareddata 6.100.0
File Interpreter ABI Platform Actions
shareddata-6.100.0-py3-none-any.whl Python 3 none any Details for shareddata-6.100.0-py3-none-any.whl

Total release size: 2.0 MB

Release files / shareddata-6.100.0.tar.gz

Download URL shareddata-6.100.0.tar.gz
Size 951.5 kB
Tags Source
SHA-256 checksum
How to use checksums
936337ac6542806d645668e687a4f6ca6cb9c265433dd4835a693b5d7c4becfe
BLAKE2b-256 checksum
How to use checksums
18896f103d808ff5d6e3211ec366cb4be0eb6aab5781a727cae75eca9632a1da
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Release files / shareddata-6.100.0-py3-none-any.whl

Download URL shareddata-6.100.0-py3-none-any.whl
Size 1.0 MB
Tags Python 3
SHA-256 checksum
How to use checksums
ab7f624d0536be0602616468bec811547ecbed418e10403d5ae38607577ba2a9
BLAKE2b-256 checksum
How to use checksums
f98742cec24273ccb2bed609750b5c555563b6366095bd7b11c51dcd295cce70
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Security for shareddata 6.100.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

6.100.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page