Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Rimsky

Rimsky - an orchestrator for automated online parameter estimation

pipeline coverage

Rimsky streams detector strain, estimates PSDs, keeps rolling caches of frames and PSDs, listens for GraceDB alerts, collects the data around each significant trigger and prepares (and optionally submits) bilby_pipe analyses, then watches for their results, reduces the posterior samples for upload to GraceDB, and publishes summary pages. See the documentation for an overview of the pipeline stages.

Usage

Run the pipeline with a YAML configuration file:

cd src/rimsky/examples
uv run python -m rimsky example.yaml

The example configurations are in src/rimsky/examples (also linked as examples/) and are installed with the package:

  • example.yaml streams simulated noise and writes the frame and PSD caches, the strain part of the pipeline.
  • pipeline_test.yaml runs the whole pipeline offline in under a minute: simulated noise, mock alerts (alert_stream), strain collection and bilby_pipe configuration (event_sink), mock results picked up by the result watcher (result_source, sample_sink) and the rescue watcher (rescue_source). Use it as the reference for the event-handling options.
  • devshm_backfill.yaml reads live strain from shared memory, see below.
  • arrakis_backfill.yaml reads replayed strain from an Arrakis server, see below.

The options of every section are documented in the API reference of rimsky.settings.

Testing

Run the test suite with

uv run python -m pytest src/rimsky/tests

This includes an end-to-end run of pipeline_test.yaml through the rimsky entry point (src/rimsky/tests/test_pipeline_run.py, marked pipeline), which takes about 30 s as the simulated noise is streamed in real time. Run only it with -m pipeline, or skip it with -m "not pipeline". It needs no network access: alerts and results are mocked, nothing is uploaded to GraceDB and bilby_pipe only writes its configuration. To run the example by hand from the repository,

mkdir -p pipeline-test && cd pipeline-test
uv run python -m rimsky ../src/rimsky/examples/pipeline_test.yaml

which stops itself after run_duration (40 s) and leaves the frames, PSDs, event directories, pages and rimsky.log in output/.

Configuration validation

Unknown configuration keys, e.g., a misspelt fft_lenght, are rejected when the configuration is loaded, with the dotted path of each key. Set unknown_keys: warn to log a warning and ignore them instead, or unknown_keys: ignore to ignore them silently. Top-level keys starting with x- or _ are ignored and can hold YAML anchors, see pipeline_test.yaml. The keys of event_sink.bilby_pipe_defaults and of the settings of event_sink.trigger_dependent_settings must be bilby_pipe options, and prior_bounds keys must set a prior of their prior_template.

The configuration is also checked for consistency, e.g., the event_sink.strain_buffer must cover the data each analysis reads (duration - post_trigger_duration seconds before the trigger and post_trigger_duration, 2 s by default, after it), and the frame cache (frame_sink.duration * frame_sink.max_files) must hold the strain buffer plus one frame.

Live data with startup backfill

The devshm-backfill data source reads live frames from shared memory (e.g. /kafka/H1_O4LLPIC) like devshm, but first replays frame files already in those directories, so the frame and PSD caches are full shortly after startup instead of after several minutes. backfill_duration sets the maximum number of seconds to replay; less is used if the directories hold less. See devshm_backfill.yaml.

The arrakis-backfill data source does the same for data streamed from an Arrakis server (set by the ARRAKIS_SERVER environment variable): it requests the stream from backfill_duration seconds before startup, which the server delivers as fast as it can before continuing with live data. Set replay_id to read one of the server's replays. Because the request has a start time, a dropped connection resumes from the last sample received rather than filling the outage with gaps. See arrakis_backfill.yaml.

Needed functionality

  • Streaming simulated data
  • Real time PSD estimation and periodic writes to an on-disk cache
  • Periodic writes to an on-disk cache of frame data
  • Stream real data
    • This can use the DevShmSource from sgnligo
    • Handled by generic sgnligo data source.
  • Listening to GW alerts
  • Read calibration files from specified directories
    • This should keep the latest available calibration file for each detector and watch a regex pattern.
  • Writing bilby_pipe config files
  • Submitting bilby_pipe jobs
  • Adding events to asimov project

Desired functionality

  • Better CI testing
  • Stream live injections
    • This should mirror streaming the real data and output frames containing the raw data + injections. Given the artificially higher signal rate, the PSD should not include the injected strain.
    • Handled by generic sgnligo data source.

Avatar created by GPT-5.6 Sol.

Metadata

Release files for rimsky 0.2.0rc1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for rimsky 0.2.0rc1
File Size Uploaded Provenance Actions
rimsky-0.2.0rc1.tar.gz 1.6 MB Without provenance Details for rimsky-0.2.0rc1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for rimsky 0.2.0rc1
File Interpreter ABI Platform Actions
rimsky-0.2.0rc1-py3-none-any.whl Python 3 none any Details for rimsky-0.2.0rc1-py3-none-any.whl

Total release size: 2.0 MB

Release files / rimsky-0.2.0rc1.tar.gz

Download URL rimsky-0.2.0rc1.tar.gz
Size 1.6 MB
Tags Source
SHA-256 checksum
How to use checksums
5fab15ad72a191dba058b0264e557ec9acc2f792bfaa5e3f54f265366882532b
BLAKE2b-256 checksum
How to use checksums
628663c862297f0187fc1dffb92d9d0bb0e9801dbc0f303f452367bb06898641
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.15.2

Release files / rimsky-0.2.0rc1-py3-none-any.whl

Download URL rimsky-0.2.0rc1-py3-none-any.whl
Size 422.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d4341bb61b9985cb8977c52f19993e87200378cf5d57099800830f182ed26413
BLAKE2b-256 checksum
How to use checksums
e670bee7d79c90b703ed8573eaa73406401e465ab699f97051b69b9687494d0e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.15.2

Security for rimsky 0.2.0rc1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page