This release is a pre-release and may not be stable for production use.
Rimsky - an orchestrator for automated online parameter estimation
Rimsky streams detector strain, estimates PSDs, keeps rolling caches of
frames and PSDs, listens for GraceDB alerts, collects the data around each
significant trigger and prepares (and optionally submits) bilby_pipe
analyses, then watches for their results, reduces the posterior samples for
upload to GraceDB, and publishes summary pages. See the
documentation for an overview of the pipeline stages.
Usage
Run the pipeline with a YAML configuration file:
cd src/rimsky/examples
uv run python -m rimsky example.yaml
The example configurations are in src/rimsky/examples
(also linked as examples/) and are installed with the package:
- example.yaml streams simulated noise and writes the frame and PSD caches, the strain part of the pipeline.
- pipeline_test.yaml runs the
whole pipeline offline in under a minute: simulated noise, mock alerts
(
alert_stream), strain collection andbilby_pipeconfiguration (event_sink), mock results picked up by the result watcher (result_source,sample_sink) and the rescue watcher (rescue_source). Use it as the reference for the event-handling options. - devshm_backfill.yaml reads live strain from shared memory, see below.
- arrakis_backfill.yaml reads replayed strain from an Arrakis server, see below.
The options of every section are documented in the API reference of
rimsky.settings.
Testing
Run the test suite with
uv run python -m pytest src/rimsky/tests
This includes an end-to-end run of
pipeline_test.yaml through the
rimsky entry point (src/rimsky/tests/test_pipeline_run.py, marked
pipeline), which takes about 30 s as the simulated noise is streamed in
real time. Run only it with -m pipeline, or skip it with
-m "not pipeline". It needs no network access: alerts and results are
mocked, nothing is uploaded to GraceDB and bilby_pipe only writes its
configuration. To run the example by hand from the repository,
mkdir -p pipeline-test && cd pipeline-test
uv run python -m rimsky ../src/rimsky/examples/pipeline_test.yaml
which stops itself after run_duration (40 s) and leaves the frames, PSDs,
event directories, pages and rimsky.log in output/.
Configuration validation
Unknown configuration keys, e.g., a misspelt fft_lenght, are rejected when
the configuration is loaded, with the dotted path of each key. Set
unknown_keys: warn to log a warning and ignore them instead, or
unknown_keys: ignore to ignore them silently. Top-level keys starting with
x- or _ are ignored and can hold YAML anchors, see
pipeline_test.yaml. The keys of
event_sink.bilby_pipe_defaults and of the settings of
event_sink.trigger_dependent_settings must be bilby_pipe options, and
prior_bounds keys must set a prior of their prior_template.
The configuration is also checked for consistency, e.g., the
event_sink.strain_buffer must cover the data each analysis reads
(duration - post_trigger_duration seconds before the trigger and
post_trigger_duration, 2 s by default, after it), and the frame cache
(frame_sink.duration * frame_sink.max_files) must hold the strain buffer
plus one frame.
Live data with startup backfill
The devshm-backfill data source reads live frames from shared memory (e.g.
/kafka/H1_O4LLPIC) like devshm, but first replays frame files already in
those directories, so the frame and PSD caches are full shortly after startup
instead of after several minutes. backfill_duration sets the maximum number
of seconds to replay; less is used if the directories hold less. See
devshm_backfill.yaml.
The arrakis-backfill data source does the same for data streamed from an
Arrakis server (set by the
ARRAKIS_SERVER environment variable): it requests the stream from
backfill_duration seconds before startup, which the server delivers as fast
as it can before continuing with live data. Set replay_id to read one of the
server's replays. Because the request has a start time, a dropped connection
resumes from the last sample received rather than filling the outage with
gaps. See arrakis_backfill.yaml.
Needed functionality
- Streaming simulated data
- Real time PSD estimation and periodic writes to an on-disk cache
- Periodic writes to an on-disk cache of frame data
- Stream real data
- This can use the
DevShmSourcefromsgnligo - Handled by generic sgnligo data source.
- This can use the
- Listening to GW alerts
- Read calibration files from specified directories
- This should keep the latest available calibration file for each detector and watch a regex pattern.
- Writing bilby_pipe config files
- Submitting bilby_pipe jobs
- Adding events to asimov project
Desired functionality
- Better CI testing
- Stream live injections
- This should mirror streaming the real data and output frames containing the raw data + injections. Given the artificially higher signal rate, the PSD should not include the injected strain.
- Handled by generic sgnligo data source.
Avatar created by GPT-5.6 Sol.
Metadata
Release files for rimsky 0.2.0rc1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| rimsky-0.2.0rc1.tar.gz | 1.6 MB | Without provenance | Details for rimsky-0.2.0rc1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| rimsky-0.2.0rc1-py3-none-any.whl | Python 3 | none | any | Details for rimsky-0.2.0rc1-py3-none-any.whl |
Total release size: 2.0 MB
Release files / rimsky-0.2.0rc1.tar.gz
| Download URL | rimsky-0.2.0rc1.tar.gz |
|---|---|
| Size | 1.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5fab15ad72a191dba058b0264e557ec9acc2f792bfaa5e3f54f265366882532b
|
|
BLAKE2b-256 checksum How to use checksums |
628663c862297f0187fc1dffb92d9d0bb0e9801dbc0f303f452367bb06898641
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.15.2
|
Release files / rimsky-0.2.0rc1-py3-none-any.whl
| Download URL | rimsky-0.2.0rc1-py3-none-any.whl |
|---|---|
| Size | 422.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d4341bb61b9985cb8977c52f19993e87200378cf5d57099800830f182ed26413
|
|
BLAKE2b-256 checksum How to use checksums |
e670bee7d79c90b703ed8573eaa73406401e465ab699f97051b69b9687494d0e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.15.2
|
Security for rimsky 0.2.0rc1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.