This release is a pre-release and may not be stable for production use.
ST4SD Runtime Core
This repository contains the runtime-core of the Simulation Toolkit for Scientific Discovery (ST4SD). The ST4SD-Runtime is a python framework, and associated services, for creating and deploying virtual-experiments - data-flows which embody the measurement of properties of systems.
A data-flow is a workflow which allows consumers to run concurrently with their producers if desired.
Developers describe their data-flows using a YAML configuration file, which is interpreted and executed by the ST4SD-Runtime.
ST4SD-Runtime supports multiple execution-backends including Kubernetes and LSF and a single YAML file can support multiple-archs and multiple run-modes via overlays.
The ST4SD-Runtime also interacts with the ST4SD-Datastore, a database which allows querying of executed virtual-experiments and retrieval of their data.
There are three parts to the ST4SD-Runtime
st4sd-runtime-core: The core python framework (this git repository) for describing and executing virtual experimentsst4sd-runtime-k8s: Extensions which enable to running and managing virtual-experiments on k8s clustersst4sd-runtime-service: A RESTapi based service allowing users to add, start, stop and query virtual-experiments
Features
- Cross-platform data-flows
- Supports multiple backends (LSF, OpenShift/Kubernetes, local)
- Abstracts differences between backends allowing a single component description to be used
- Variables can be used to encapsulate platform specific options
- Can define component and platform specific environments
- Co-processing model
- Consumers can be configured to run repeatedly while their producers are alive
- Simple to replicate workflow sub-graphs over sets of inputs
- Supports
do-whileconstructs - Handles task persistence across backend allocation windows and allows user customisable restarts
- Deploy workflows directly from github (Kubernetes stack)
- Store and retrieve data and metadata from st4sd-datastore
Lightning Start
If you have
python3withvirtualenv- Have
sshaccess to GitHub set up
The following snippet will install st4sd-runtime-core and run a toy-workflow on your laptop
virtualenv -p python3 $HOME/st4sd-runtime-test
source $HOME/st4sd-runtime-test/bin/activate
pip install st4sd-runtime-core[deploy]
git clone git@github.com:st4sd/sum-numbers.git
elaunch.py --nostamp -l40 sum-numbers
This will create a new virtualenv called st4sd-runtime-test at $HOME/st4sd-runtime-test and install st4sd-runtime-core into it. It will also clone a repository into a directory called sum-numbers in whatever directory you run the above commands in. It will then run a toy-workflow that takes a couple of minutes to run. The toy workflow output will be in a directory called sum-numbers.instance
You can learn more about the toy-workflow, and workflow specification, here.
More Information
Our documentation website contains detailed information on installing ST4SD, writing and running virtual-experiments, along with much more.
4 maintainers
Release files for st4sd-runtime-core 2.0.0a9.dev5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| st4sd-runtime-core-2.0.0a9.dev5.tar.gz | 688.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| st4sd_runtime_core-2.0.0a9.dev5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.3 MB
Release files / st4sd-runtime-core-2.0.0a9.dev5.tar.gz
| Download URL | st4sd-runtime-core-2.0.0a9.dev5.tar.gz |
|---|---|
| Size | 688.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f90d96f92b146a7427f1392ee3adc6df0e5864b01c7618df50ac38796e58f0eb
|
|
BLAKE2b-256 checksum How to use checksums |
e1059952018f7625dac5ef1f40aff5f2274b8760c822eb194a3a2558b3409ffd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.7.5
|
Release files / st4sd_runtime_core-2.0.0a9.dev5-py3-none-any.whl
| Download URL | st4sd_runtime_core-2.0.0a9.dev5-py3-none-any.whl |
|---|---|
| Size | 660.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
940e39f4eeb03e51041e74a9b783fb0f1c9e3d268f70ad2e36373866ab5a8e16
|
|
BLAKE2b-256 checksum How to use checksums |
81acf6b5eb081cec21e7d63a624c9a080d703ff601970c334318e7c8a91c6e6a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.7.5
|
