Efficient, lightweight, and reliable distributed computation.
Documentation: https://finos.github.io/opengris-scaler/
Start there for installation options, command reference, worker manager guides, scaling policies, and examples.
What Is Scaler?
OpenGRIS Scaler is a distributed computing framework for running Python tasks across local machines or remote infrastructure.
It provides:
- A Python client API similar to
multiprocessingpatterns such assubmit(),map(), andstarmap(). - A centralized scheduler that dispatches work and balances load across workers.
- Worker managers for local execution and cloud-backed capacity.
- Support for graph/DAG execution, monitoring, and task recovery.
Architecture
- Clients submit tasks to a scheduler.
- The scheduler tracks state, applies scaling/allocation policies, and dispatches work.
- Worker managers provision workers locally or on external infrastructure.
- Workers execute tasks and return results.
- An object storage service stores task inputs and outputs used by the cluster.
Local Quickstart
Install the package:
pip install opengris-scaler
Create config.toml:
[object_storage_server]
bind_address = "tcp://127.0.0.1:8517"
[scheduler]
bind_address = "tcp://127.0.0.1:8516"
object_storage_address = "tcp://127.0.0.1:8517"
[[worker_manager]]
type = "baremetal_native"
scheduler_address = "tcp://127.0.0.1:8516"
worker_manager_id = "wm-native"
Start a fully local stack:
scaler config.toml
Submit tasks from Python:
from scaler import Client
def square(value: int) -> int:
return value * value
with Client(address="tcp://127.0.0.1:8516") as client:
results = client.map(square, range(10))
print(results)
Learn More
1 maintainer
Metadata
1 maintainer
Release files for scaler-dev 2.8.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
Total release size: 135.1 MB
Release files / scaler_dev-2.8.0-cp315-cp315t-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp315-cp315t-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.15 CPython 3.15 free-threading Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
5baa1a87b70084c7470f0f133f3bb407d54f4357f6286927f73a8c9606c893db
|
|
BLAKE2b-256 checksum How to use checksums |
5ff9298e2f399580ebf00f5f6edefb14d6565f11c350d39278eeaa6e13bb2861
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp315-cp315t-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp315-cp315t-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.15 CPython 3.15 free-threading Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
1354e38253ac58b22427b960532e1a55d1e5d6d6872aa5477faf9440bbfa8415
|
|
BLAKE2b-256 checksum How to use checksums |
65499c9cc48bbb6c6d8afee61641d2f47cc6096705e688bbe1ee897aa8640d73
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp315-cp315-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp315-cp315-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.15 Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
15a523f6aa21cb14e30ff8967169fe7958acb4b76d3552fc910973cf3eaaa6b1
|
|
BLAKE2b-256 checksum How to use checksums |
1919737f8c1ed7ad9fdf5f5856b860652ec2d7912fd9e1e27ba23cbfe7ba94e9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp315-cp315-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp315-cp315-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.15 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
66de392455a4081fb61b4ae8a64ba4c0d7f535465247696c47dbe3392f6cb2f6
|
|
BLAKE2b-256 checksum How to use checksums |
11cd6437014c35450a3801fe2a71d7c153b236a354390801091ebc6e33163693
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp314-cp314t-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp314-cp314t-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
eae4a7dbff9fb3ed48b3cd24b15a944ebadb5c8822b18607c9ca40cd0789b5a2
|
|
BLAKE2b-256 checksum How to use checksums |
c99b1e47c9ab47c574ceb75ffadbf5be2251596d306865ef8ef564ff9320a49c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp314-cp314t-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp314-cp314t-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
0235abacd58178e3014b0e7aaded16ec29541cdbb3fb6d0ef3c10016950f2b66
|
|
BLAKE2b-256 checksum How to use checksums |
65ad98ea07594d2961ab8ccbc1c0f76b432efb670d72df553e5ceeddfbfc0606
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp314-cp314-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp314-cp314-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.14 Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
1cff1cecfef131853ef86dec056b54c275ee901834221ee52ca4b83666d5a603
|
|
BLAKE2b-256 checksum How to use checksums |
91eda33b31c8327ebb3d23aa3922cda65d4d2e5647d2bb0f0ea81f24c8f3a57f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp314-cp314-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp314-cp314-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.14 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
5d34aa219fa9c257290209f31479b6529c1e6e1bceb359a30aeffc6ba577e0c9
|
|
BLAKE2b-256 checksum How to use checksums |
01c61dd6c6289166ba7972a1c953e6735748a4053bb3ada762d1587456dbae73
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp313-cp313-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp313-cp313-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.13 Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
61c86e01e7df7b7b0a9b68b359e6d507663cb53362036d12848ad492a447cacb
|
|
BLAKE2b-256 checksum How to use checksums |
884ef736b768b8fc8dc906a5b9b8fa64c3e9e9a403e7402c6be938bab2773f33
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp313-cp313-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp313-cp313-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.13 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
cee78407117e47c41f6dabe81c1046532345b461b954ae7675af33415adffac2
|
|
BLAKE2b-256 checksum How to use checksums |
7b6da47675aaf439a0e4a2493fd2a38a20560f4f5ae8c7282c2b533e6c0ffa52
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp312-cp312-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp312-cp312-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.12 Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
ef7de11d2810c9f12089deebfb55ba768f854816b1dccb72b3940a9eabc7d0c3
|
|
BLAKE2b-256 checksum How to use checksums |
5b0f170b2436a28ac61362761bcbf7fd4c1444ba68d1d5e7707d3d7dec61012b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp312-cp312-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp312-cp312-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.12 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
decf3b4ea8f0dd38dda60855c48082a9d7d89dbd4319b273732ddc70457d30c8
|
|
BLAKE2b-256 checksum How to use checksums |
c379edfa63be60a115c761666e5f399d6c452b65ebaad6aebb51c5d2371ea12e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp311-cp311-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp311-cp311-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.11 Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
c6f8d5f07f5c0570e653e19b965e9bcd9bc707fb9dc4276af5bbf0cc6f704bea
|
|
BLAKE2b-256 checksum How to use checksums |
dd1ff1a173e587e90b4b46b9e3be6e62c207a53f8b8eacc84ad7ffa8ee3e8827
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp311-cp311-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp311-cp311-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.11 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
d27d13112d41b295c1e916b96c0bd577eb556d04a57b8c1a126f303eb101c94a
|
|
BLAKE2b-256 checksum How to use checksums |
068887377a2de2d044984dc6835ad06bb0e9d990978871fa893919cacd99c6d3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp310-cp310-musllinux_1_2_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp310-cp310-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 8.3 MB |
| Tags | CPython 3.10 Linux musl 1.2+ x86-64 |
|
SHA-256 checksum How to use checksums |
a343eafcb9b2dc10f89f00b686e6033dd78fc69d52cb11ac59796f50f8d694d4
|
|
BLAKE2b-256 checksum How to use checksums |
4ed012cf69904af8442bd97f635f84f7004e7759815ac0834644216986740090
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / scaler_dev-2.8.0-cp310-cp310-manylinux_2_28_x86_64.whl
| Download URL | scaler_dev-2.8.0-cp310-cp310-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 8.6 MB |
| Tags | CPython 3.10 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
e8ec5cbb8741b82bb242ddf426708373d0c2701210e2742f391e3580c692a2eb
|
|
BLAKE2b-256 checksum How to use checksums |
7464184c60431900738c844b2eadcaa625290b595103b80dae2039d3c51e3dfd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
