Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

HoLcStore

HoLcStore is a Django app for creating a simple TimeSeries store in your database.

Table of Contents

Getting Started

  1. Add "holcstore" to your INSTALLED_APPS setting like this
INSTALLED_APPS = [
    ...,
    "holcstore",
]
  1. Start using the abstract model Store by importing it
from hostore.models import Store

class YourStore(Store):
    # add new fields

    class Meta(Store.Meta):
        abstract = False
        # add your meta

Choose the appropriate store

Store class

This class is used to store timeseries, using a key:value pattern. A prm key is used to reference the saved series.

Handle update and replace features.

TimeseriesStore class

This class is used to store timeseries, using a user provided pattern through its model.

TimeseriesChunkStore class

This class is used to store timeseries, using a user provided pattern through its model.

Handle update and replace features.

Store timeseries by chunk for a better performance with large timeseries.

User friendly API to perform a client-server sync.

Basic Usage: Store class

This store is appropriate if you want to store using a "key - value" pattern.

Saving a timeserie to database

from path.to.YourStore import YourStore
import pandas as pd

key = "3000014324556"
client_id = 0
idx = pd.date_range('2024-01-01 00:00:00+00:00', '2024-01-02 00:00:00+00:00', freq='30min')
ds = pd.Series(data=[1]*len(idx), index=idx)
# Put the load curve to the store without versionning
YourStore.set_lc(key, ds, client_id)
# If you want to activate the versionning 
YourStore.set_lc(key, ds, client_id, versionning=True)
# Each time you call set_lc with your id, a new version will be saved in the database

Saving multiple timeseries to database

from path.to.YourStore import YourStore
import pandas as pd

key = "3000014324556"
client_id = 0
idx = pd.date_range('2024-01-01 00:00:00+00:00', '2024-01-02 00:00:00+00:00', freq='30min')
df = pd.Series(data={'key1': [1]*len(idx), 'key2': [2]*len(idx), }, index=idx)
# Put the load curve to the store without versionning
YourStore.set_many_lc(df, client_id)
# If you want to activate the versionning 
YourStore.set_many_lc(df, client_id, versionning=True)

Getting a load curve from the database

from path.to.YourStore import YourStore

key = "3000014324556"
client_id = 0
# Get the load curve from the database
# I multiple versions exists, they will be combined beginning with the version 0 and using 
# https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.combine_first.html
datas = YourStore.get_lc(key, client_id)

if not datas:
    my_timeserie = datas[0]['data']
    last_updated = datas[0]['last_modified']

# If you want to retreive all versions 
datas = YourStore.get_lc(key, client_id, combined_versions=False)
# datas contains all timeseries linked to key

# If you want to retreive a specific version
datas = YourStore.get_lc(key, client_id, version=1)

Basic Usage: TimeseriesStore class

This store is the easiest to use, but has less features than TimeseriesChunkStore

Define your class in models.py

class MyTimeseriesStore(TimeseriesStore):
    year = models.IntegerField()
    kind = models.CharField(max_length=100)

    class Meta(TimeseriesStore.Meta):
        unique_together = ('year', 'kind')

Usage samples

# specify attributes to set
ts_attrs = dict(year=2020, kind='a')

# build a timeserie 
ds_ts = pd.Series(...)  

# set timeserie to db
MyTimeseriesStore.set_ts(ts_attrs, ds_ts)

# update existing timeserie in db (will combine ds_ts + ds_ts_v2, giving priority to ds_ts_v2 datas)
MyTimeseriesStore.set_ts(ts_attrs, ds_ts_v2, update=True)

# get timeserie from db if unique match
ds_ts = MyTimeseriesStore.get_ts(ts_attrs, flat=True)

# get timeseries from db if multiple match
datas = MyTimeseriesStore.get_ts(ts_attrs)
ds_ts1 = datas[0]['data']

Basic usage: TimeseriesChunkStore class

Compact time-series storage for Django + PostgreSQL

TimeseriesChunkStore is an abstract Django model that lets you persist high-resolution time-series efficiently as compressed LZ4 blobs, while still querying them through the ORM.

Main features

Feature Description
Chunking Split each series by ('year',) or ('year','month') so blobs stay small.
Compression Data saved as LZ4-compressed NumPy buffers; index is rebuilt on the fly.
Dense layout Each chunk is re-indexed on a regular grid (STORE_FREQ, STORE_TZ).
Smart upsert set_ts(..., update=True) merges with existing data via combine_first.
Bulk helpers set_many_ts() / yield_many_ts() for mass insert / streaming read.
Sync ready list_updates, export_chunks, import_chunks enable cheap client ↔ server replication.
REST scaffolding TimeseriesChunkStoreSyncViewSet + TimeseriesChunkStoreSyncClient give you plug-and-play API endpoints and a Python client.

Quick start

1/ Define your store class

# models.py
class MyChunkedStore(TimeseriesChunkStore):
    # Custom fields (can be seen as "axis keys")
    version = models.IntegerField()
    kind    = models.CharField(max_length=20)

    # Store settings
    CHUNK_AXIS = ('year', 'month')   # Chunking axis for timeseries storage. Configs : ('year',) / ('year', 'month')
    STORE_TZ   = 'Europe/Paris' # Chunking timezone (also timeseries output tz)
    STORE_FREQ   = '1h' # Timeseries storage frequency. (the store reindex input series but never resample)
    ALLOW_CLIENT_SERVER_SYNC = False # if True, enable the sync features

The Custom fields are strictly indexation axis : you must not use them to store metadata or redundant data.

During django's setup, any class that inherits from TimeseriesChunkStore will have its Meta properties unique_together and indexes automatically edited, such as it contains all your keys and the mandatory keys from the abstract store.

Do not :

  • define your own Meta.unique_together or Meta.indexes.
  • define those fields : start_ts, data, dtype, updated_at, chunk_index, is_deleted.
  • edit "Store settings" : CHUNK_AXIS, STORE_TZ, STORE_FREQ, ALLOW_CLIENT_SERVER_SYNC once the table has been created through migration. This will lead to data corruption.

2/ Use your store class

This summarize use cases with legal and illegal usages.

# my_timeseries_usage.py
# Set one
import pandas as pd

attrs = {"version": 1, "kind": "kind1"}
MyChunkedStore.set_ts(attrs, my_series1)  # first write
MyChunkedStore.set_ts(attrs, my_series2, update=True)  # update (combine_first)
MyChunkedStore.set_ts(attrs, my_series3, replace=True)  # replace
MyChunkedStore.set_ts(attrs, my_series4)  # FAIL : attrs exists

# Get one
full = MyChunkedStore.get_ts(attrs)
window = MyChunkedStore.get_ts(attrs, start=pd.Timestamp("2024-01-01 00:00:00+01:00"), end=pd.Timestamp("2024-06-01 02:00:00+02:00"))
fail = MyChunkedStore.get_ts({"version": 1})  # FAIL : must specify all attrs
none = MyChunkedStore.get_ts({"version": 1, "kind": "nonexisting"})  # returns None : does not exists

# Set many
MyChunkedStore.set_many_ts(
  mapping={
    (5, "kind1"): my_series1,
    (5, "kind2"): my_series2,
  },
  keys=("version", "kind")
)

# Yield many
series_generator = MyChunkedStore.yield_many_ts({"version": 5})  # contains the 2 (serie, key_dict) from mapping
max_horodate = MyChunkedStore.get_max_horodate({"version": 5})  # maximum horodate of the 2 (serie, key_dict) from mapping

# CANNOT set many over existing
MyChunkedStore.set_many_ts(mapping, keys=("version", "kind"))  # FAIL : at least one value from mapping already exists

3/ Setup sync tools

Two TimeseriesChunkStore can be easily synchronized between a server and a client. Client and server models must be the same.

You must set ALLOW_CLIENT_SERVER_SYNC=True if you want to use those features. In that case please note some particular behaviour :

  • If you need to delete objects on server side, you must pass keep_tracking=True to delete method. Eg : ServerStore.objects.filter(version=1).delete(keep_tracking=True). Otherwise a ValueError will be raised.
  • You cannot use the set_ts(replace=False, update=False) or set_many_ts. Otherwise a ValueError will be raised.

3.1/ Define the ViewSet (server side : serve data)

You can set throttle, auth etc. through as_factory kwargs.

# views.py
from hostore.utils.ts_sync import TimeseriesChunkStoreSyncViewSet

YearSync = TimeseriesChunkStoreSyncViewSet.as_factory(MyChunkStoreServerSide, throttle_classes=[], **kwargs_views)  # pass ViewSet kwargs
router.register("ts/myendpoint", YearSync, basename="ts-myendpoint")

3.2/ Define the API (client side : pull new data)

# my_client_sync_module.py
from hostore.utils.ts_sync import TimeseriesChunkStoreSyncClient

client = TimeseriesChunkStoreSyncClient(
    endpoint="https://api.example.com/ts/myendpoint",
    store_model=MyChunkStoreClientSide,
)
client.pull(batch=100)      # fetch new / updated chunks limiting one batch request to 100 items

(back to top)

Metadata

Release files for holcstore 0.5.3a4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for holcstore 0.5.3a4
File Size Uploaded Provenance Actions
holcstore-0.5.3a4.tar.gz 43.3 kB With provenance Details for holcstore-0.5.3a4.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for holcstore 0.5.3a4
File Interpreter ABI Platform Actions
holcstore-0.5.3a4-py3-none-any.whl Python 3 none any Details for holcstore-0.5.3a4-py3-none-any.whl

Total release size: 100.3 kB

Release files / holcstore-0.5.3a4.tar.gz

Download URL holcstore-0.5.3a4.tar.gz
Size 43.3 kB
Tags Source
SHA-256 checksum
How to use checksums
069a42f3071c1cd7d32c7bd10177dca9b46da74a47ddfa0b6fc78a48cfd96e95
BLAKE2b-256 checksum
How to use checksums
b5fb99f6c219cb3b881272310f0f5180fef3eb189a7f1305eb895a2877eeddff
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.12.9

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 22, 2025.

Transparency log

Release files / holcstore-0.5.3a4-py3-none-any.whl

Download URL holcstore-0.5.3a4-py3-none-any.whl
Size 57.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
35df99325453195fb67661664e5e4beb2f66ed0122bfbee12792d2e22ff7fec0
BLAKE2b-256 checksum
How to use checksums
69bf468d303304f4c6a265f559297603991f3a8da40ed1152c660f3ca29d367e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.12.9

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 22, 2025.

Transparency log

Security for holcstore 0.5.3a4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for holcstore-0.5.3a4.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 22, 2025.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page