Skip to main content

SharedData

SharedData is an open-source memory-mapped store for records and time series — in the workload category of tick stores such as kdb+ — plus the scheduler and worker fleet that produce the data. Local processes map the same files through the operating system's page cache; indexed writes coordinate through a per-table process lock. Remote clients use an authenticated HTTP API; shared memory does not span hosts.

Website and manual: https://shareddata.net

Install

Python 3.11 or 3.12:

python -m pip install shareddata

The quick start uses illustrative records in isolated storage. See configuration for public environment names and Windows limitations for native requirements and rewrite behavior.

A record in one minute

Save as example.py after setting up isolated storage in the quick start, then run python example.py:

from SharedData.SharedData import SharedData
import pandas as pd

store = SharedData(__file__, user="example")
index = pd.MultiIndex.from_tuples(
    [(pd.Timestamp("2026-01-02"), "EXAMPLE")],
    names=["date", "symbol"],
)
rows = pd.DataFrame({"close": [100.0]}, index=index)
records = store.table(
    "MarketData", "D1", "DEMO", "PRICES",
    value=rows,
)
print(records.loc["2026-01-02", "EXAMPLE"])

Containers and execution

  • Tables: structured NumPy records and primary-key lookup.
  • Time series: timestamped float matrices.
  • Collections, Redis caches, Kafka streams and optional S3 integration.
  • HTTP API: authenticated remote access.
  • Measured latency: what the containers cost in practice.

A store and the fleet that fills it

A routine is any program the Scheduler owns — a feed collector, a parser, a consolidation job, an API server. A schedule lists the routines that should stay alive or run at their times. Batch routines wait for their batch dependencies to complete; realtime routines wait for their realtime dependencies to be running. Dependencies between the two kinds are not enforced at runtime. Commands travel through a per-host worker pool that validates the requested source revision before every launch; routines report heartbeats and lifecycle messages through the logging plane, and the Scheduler writes queryable state into the store.

Data by symbol

The semantic data API resolves a symbol or entity to a dataset and includes provenance. It is a convenience, not the fast path: producers keep using direct container interfaces for performance-critical work.

Client integration

Client integration covers authenticated API access, the MCP server for compatible agent clients, and worker access.

Development and license

Submit changes through the repository's review process. Read the development guide and run the checks relevant to your change; building and publishing explains how the public site is produced and audited. SharedData is released under the MIT license; see LICENSE in the source distribution.

Metadata

Release files for shareddata 6.101.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for shareddata 6.101.1
File Size Uploaded Provenance Actions
shareddata-6.101.1.tar.gz 979.4 kB Without provenance Details for shareddata-6.101.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for shareddata 6.101.1
File Interpreter ABI Platform Actions
shareddata-6.101.1-py3-none-any.whl Python 3 none any Details for shareddata-6.101.1-py3-none-any.whl

Total release size: 2.1 MB

Release files / shareddata-6.101.1.tar.gz

Download URL shareddata-6.101.1.tar.gz
Size 979.4 kB
Tags Source
SHA-256 checksum
How to use checksums
ceb3dfff2bd49e11e0c77ba3b4e6f7205f646ea2c5fa4ddcbc1b78c9b61d82d6
BLAKE2b-256 checksum
How to use checksums
9a49c2ea0a5e8f3e6626688303d5d9fcbb27294946fbd195356f5c3bf5f06ab3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Release files / shareddata-6.101.1-py3-none-any.whl

Download URL shareddata-6.101.1-py3-none-any.whl
Size 1.1 MB
Tags Python 3
SHA-256 checksum
How to use checksums
ed6d4c3e51dcef624947abed1eaf81637f4e5589e9c5d764fbe4353992b691d9
BLAKE2b-256 checksum
How to use checksums
97ab395b1adf56431c659c8371de76739f0f1273d1f503c930d6ddf74560b22b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.3

Security for shareddata 6.101.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

6.101.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page