mic-datastore

Briques de persistance & messaging du framework MIC. Dépend de mic-core (DAG platform → data → core — jamais l'inverse).

Modules

Module	Rôle
`mic.datastorage`	Backends de données : `SqlDataStorage`, `DocumentDataStorage` (MongoDB), `GraphDataStorage` (Neo4j), variantes in-memory
`mic.cache`	Cache (in-memory / Redis) + `RevokeBlacklist` (révocation JWT)
`mic.shard`	Sélection de shard / session factory
`mic.locking`	Verrous distribués (in-memory / Redis)
`mic.outbox`	Pattern transactional outbox + dispatcher
`mic.eventbus`	Pub/sub synchrone (in-memory / Redis Streams)
`mic.queue`	Event bus async durable (NATS JetStream / Kafka)
`mic.idempotency`	Store d'idempotence
`mic.response_cache`	Middleware de cache de réponses HTTP
`mic.read_models`	Read-models (compteurs, leaderboards, timelines, HLL)
`mic.realtime`	WebSocket realtime (backend Redis Streams)
`mic.scheduling`	Tâches périodiques : verrou, checkpoint, suivi et alerte de silence

Namespace PEP 420 (pas de mic/__init__.py).

`mic.realtime` — WebSocket depuis un navigateur

Un navigateur ne peut pas poser d'en-tête Authorization sur l'upgrade WebSocket, et un jeton en query string finit dans les journaux d'accès : le client passe son jeton en sous-protocole et propose en plus un sous-protocole applicatif, que le serveur doit renvoyer dans l'accept (sinon le navigateur ferme la connexion).

new WebSocket(url, ["app.v1", `bearer.${jwt}`]);

from mic.realtime import (
    AsyncWebSocketAuthenticator, WebSocketAuthError, extract_bearer_token,
    make_websocket_endpoint,
)

class OidcAuthenticator(AsyncWebSocketAuthenticator):  # vérif. async (JWKS)
    async def authenticate(self, scope):
        token = extract_bearer_token(scope)
        if token is None or not await verify_with_jwks(token):
            raise WebSocketAuthError("unauthorized")  # → close 4401

endpoint = make_websocket_endpoint(
    room=room,
    authenticator=OidcAuthenticator(),  # ou ServiceAuthWebSocketAuthenticator (sync)
    subprotocol="app.v1",  # renvoyé seulement si proposé ; jamais `bearer.<jwt>`
)

RedisStreamsBackend(client=redis, maxlen=1_000) plafonne chaque stream de room via XADD MAXLEN ~ (défaut DEFAULT_MAXLEN = 1000, trim approximatif ; maxlen=None = pas de plafond).

`mic.scheduling` — tâches périodiques observables

Le planificateur de la plateforme (CronJob Kubernetes, systemd timer, boucle du service) appelle run_due() ; MIC porte les garanties qui doivent rester identiques quel que soit ce planificateur : exclusion distribuée, reprise, état d'exécution et détection du silence.

from mic.locking import RedisDistributedLock
from mic.scheduling import (
    PeriodicTaskRunner,
    RedisSchedulingStore,
    SilenceMonitor,
    TaskContext,
    TaskResult,
    periodic,
)

@periodic(
    name="collect:k8s",
    every="15m",
    lock="collect:k8s",
    timeout="10m",
    silence_after="35m",
)
def collect(ctx: TaskContext) -> TaskResult:
    cursor = ctx.checkpoint
    processed, next_cursor = collect_batch(cursor)
    ctx.raise_if_timed_out()  # point d'arrêt sûr entre deux lots
    return TaskResult(processed=processed, checkpoint=next_cursor)

store = RedisSchedulingStore(redis_url="redis://redis:6379/0")
runner = PeriodicTaskRunner(
    lock_backend=RedisDistributedLock(redis_url="redis://redis:6379/0"),
    store=store,
    metrics=metrics_backend,  # MetricsBackend optionnel
)
runner.run_due(collect)

alerts = SilenceMonitor(store=store, metrics=metrics_backend).check([collect])

Une tâche en échec ne fait jamais avancer son checkpoint et son exception est propagée au superviseur. Le timeout est une échéance coopérative : le runner mesure tout dépassement au retour et TaskContext.raise_if_timed_out() permet de l'interrompre entre deux lots. Il ne tue pas un thread Python arbitraire ; un arrêt forcé reste la responsabilité du superviseur de processus ou du activeDeadlineSeconds d'un CronJob. SilenceMonitor doit vivre dans un processus de supervision distinct si l'on veut détecter la disparition totale du worker.

Métriques émises via MetricsBackend : mic_scheduling_runs_total (task, status), mic_scheduling_duration_seconds, mic_scheduling_last_processed, mic_scheduling_last_success_timestamp_seconds et mic_scheduling_silent.

Installation

pip install "mic-datastore[postgres,redis]"

Extras

Extra	Tire
`sql`	sqlmodel + alembic
`postgres` / `mysql` / `oracle`	`sql` + driver (psycopg / PyMySQL / oracledb)
`mongo` (alias `document`)	pymongo
`neo4j` (alias `graph`)	neo4j
`redis`	redis
`nats` / `kafka`	nats-py / aiokafka
`messaging`	`nats` + `kafka`
`ws`	`redis` + `mic-core[auth]`

Versionnage

Brique volatile — reste en 0.x plus longtemps que mic-core. Contraindre avec >=0.1,<1. Release : make release-pkg PKG=data VERSION=x.y.z → tag data-vX.Y.Z (cf. ../../RELEASING.md).

Metadata

Release files for mic-datastore 0.1.24

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mic-datastore 0.1.24
File	Size	Uploaded	Provenance
mic_datastore-0.1.24.tar.gz	110.7 kB	Sep 30, 2026	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for mic-datastore 0.1.24
File	Interpreter	ABI	Platform	Reset
mic_datastore-0.1.24-py3-none-any.whl 155.7 kB Sep 30, 2026 Without provenance	Python 3	none	any	Details

Total release size: 266.4 kB

Release files / mic_datastore-0.1.24.tar.gz

Download URL	mic_datastore-0.1.24.tar.gz
Size	110.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`44621b340b397e098195d1a181ac0f71e96f3a59e7b22a1365434ee533f55aaf`
BLAKE2b-256 checksum How to use checksums	`db84ea09aa99403543434c96b2a83fde26f26440298e1946c6b4ba8a7ddc4d17`
Upload date	Sep 30, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.14.7`

Release files / mic_datastore-0.1.24-py3-none-any.whl

Download URL	mic_datastore-0.1.24-py3-none-any.whl
Size	155.7 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`b0c8b63c286b29fc68327d4f1ccc3204bcf347d48d53c5038c9240e838aa893b`
BLAKE2b-256 checksum How to use checksums	`04b000a022d1bee4f48848bf5126a019a019feabedc8107102f1e5f2ba445728`
Upload date	Sep 30, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.14.7`

Security for mic-datastore 0.1.24

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.