OpenGRIS Scaler

Efficient, lightweight, and reliable distributed computation.

FINOS Incubating Documentation License PyPI OpenSSF Scorecard

Documentation: https://finos.github.io/opengris-scaler/

Start there for installation options, command reference, worker manager guides, scaling policies, and examples.

What Is Scaler?

OpenGRIS Scaler is a distributed computing framework for running Python tasks across local machines or remote infrastructure.

It provides:

  • A Python client API similar to multiprocessing patterns such as submit(), map(), and starmap().
  • A centralized scheduler that dispatches work and balances load across workers.
  • Worker managers for local execution and cloud-backed capacity.
  • Support for graph/DAG execution, monitoring, and task recovery.

Architecture

Scaler architecture

  • Clients submit tasks to a scheduler.
  • The scheduler tracks state, applies scaling/allocation policies, and dispatches work.
  • Worker managers provision workers locally or on external infrastructure.
  • Workers execute tasks and return results.
  • An object storage service stores task inputs and outputs used by the cluster.

Local Quickstart

Install the package:

pip install opengris-scaler

Create config.toml:

[object_storage_server]
bind_address = "tcp://127.0.0.1:8517"

[scheduler]
bind_address = "tcp://127.0.0.1:8516"
object_storage_address = "tcp://127.0.0.1:8517"

[[worker_manager]]
type = "baremetal_native"
scheduler_address = "tcp://127.0.0.1:8516"
worker_manager_id = "wm-native"

Start a fully local stack:

scaler config.toml

Submit tasks from Python:

from scaler import Client


def square(value: int) -> int:
    return value * value


with Client(address="tcp://127.0.0.1:8516") as client:
    results = client.map(square, range(10))

print(results)

Learn More

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for raphaelj from gravatar.com raphaelj

Credits

Author: Citi

License

Apache Software License (Apache 2.0)

Requires

Python >=3.10

Provides Extra

zmq uvloop gui graphblas aws orb oci all

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

scaler_dev-2.5.12-cp314-cp314t-musllinux_1_2_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ x86-64

scaler_dev-2.5.12-cp314-cp314t-manylinux_2_28_x86_64.whl (9.1 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.28+ x86-64

scaler_dev-2.5.12-cp314-cp314-musllinux_1_2_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.14musllinux: musl 1.2+ x86-64

scaler_dev-2.5.12-cp314-cp314-manylinux_2_28_x86_64.whl (9.1 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.28+ x86-64

scaler_dev-2.5.12-cp313-cp313-musllinux_1_2_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.13musllinux: musl 1.2+ x86-64

scaler_dev-2.5.12-cp313-cp313-manylinux_2_28_x86_64.whl (9.1 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.28+ x86-64

scaler_dev-2.5.12-cp312-cp312-musllinux_1_2_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.12musllinux: musl 1.2+ x86-64

scaler_dev-2.5.12-cp312-cp312-manylinux_2_28_x86_64.whl (9.1 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

scaler_dev-2.5.12-cp311-cp311-musllinux_1_2_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.11musllinux: musl 1.2+ x86-64

scaler_dev-2.5.12-cp311-cp311-manylinux_2_28_x86_64.whl (9.1 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

scaler_dev-2.5.12-cp310-cp310-musllinux_1_2_x86_64.whl (8.7 MB view details)

Uploaded CPython 3.10musllinux: musl 1.2+ x86-64

scaler_dev-2.5.12-cp310-cp310-manylinux_2_28_x86_64.whl (9.1 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

File details

Details for the file scaler_dev-2.5.12-cp314-cp314t-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp314-cp314t-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 afc04c6b14ee1a1148c791a853b2521fa95e77d4ee0ad66cf87f868513af3569
MD5 eaa8f69b3994062ee34b54cae1d7b47f
BLAKE2b-256 0281790d41e5cd02a189878b6767a356d5d27d799beb9297ef9463067bbacaee

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp314-cp314t-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp314-cp314t-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 25650c6f9b959ea102b197b9162e1282268bc534ff786e6e9519b64bfae1a487
MD5 9a3cd046170b6b7a50db6f9c4f463612
BLAKE2b-256 a0f289f10e95f13a25210808108a338e01068d331073b52c929af3ff0a7265ed

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp314-cp314-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp314-cp314-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 22432e0c9a4134e777689671db1e5ef69a402d50d5f2b57fa25849b2a5bcb9fd
MD5 5c9ec2f8b5e2112f23ea1984d542f641
BLAKE2b-256 ec4a5d9d0e12ec15930f357a7e155a9646ebf6b412a4c913bd66ea409025de6b

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp314-cp314-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp314-cp314-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 11a30c5bf9510c67ba29eb24ca6bba682cf7fb969c46c5efca06620e6bb8ece7
MD5 5c595c9a925c0bd56ce9066524b4598e
BLAKE2b-256 e62a700e986c6dce1a4d373e9cfa3bad9029fd30af430938823582192134d683

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp313-cp313-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp313-cp313-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 1005d88a4ebcfc02798991eed50b0efd3294775903a8ed45401116fe1000fdfc
MD5 d85e36bd18b670080792ed80a63ad454
BLAKE2b-256 46c06bec8622a8d7a86ffcbe036beff1b0428c22779f475fda860029c2bd52c9

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp313-cp313-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp313-cp313-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 cbc0865d01f06bcf5687710b044adf15196ddf7b0cfa2f7982f6186afde2104a
MD5 7a59bb97eb4c128acc0a7946f35ec7fe
BLAKE2b-256 67bc1de76c6a72c8b84a03e96356ac746bb6cfcd1b514f31394ccd7a55650c1b

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp312-cp312-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp312-cp312-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 5ff843d1a9ec0c8b5e4400a654f81e4fe6ff84e0217abf1893f6ba435a51f35a
MD5 d5b7d56827bb6f04f154e7736f041368
BLAKE2b-256 d31ec6fdc369917ca8ff8d1941c61d94d71333e282b1dc373dce768e25e55b34

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 6db4ecd08ae8841d05bc84c39b56611c4fee066b7ebc40a109a25de928ef800a
MD5 9dbedfa9c11a91721a621aa8f3515168
BLAKE2b-256 9a54fb08a293e7aba8618bc9b97b7f8fa48764eaf8dda253cb0346462000c390

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp311-cp311-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp311-cp311-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 f6be18e64fc0639f125854178af1e11d06c2f6c3e2407f4d730e555f114e793f
MD5 aba46627c0641a3bc7cfdd83f99f9b5b
BLAKE2b-256 1a11337d0982f3f4d2613cb59f67f287d1aaff48d693fcaaaecc531b9cd9e8ac

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 640ffde066601c125128a77f2cb1c4c6dbef7e87d013d04a1341b52859e806ca
MD5 c75e916da4813fb344291cd7acf6334f
BLAKE2b-256 d5e4890c749de2718bc7ecefa8af2002abdb978302739cdf7754e77f1128292e

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp310-cp310-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp310-cp310-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 d8510e772a1ac48597520331f4d6544e2e837f0c4375733b10c979e5990b05d7
MD5 3f9934926b0cf2e0e0460fc4f5aefaf0
BLAKE2b-256 ff40da4a42394975825a9f693f88faac92885f69c91c9c09584ff1ba12915b34

See more details on using hashes here.

File details

Details for the file scaler_dev-2.5.12-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for scaler_dev-2.5.12-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 eaa0a87f5011b6f46975056bb2c451e4ee805c185eb95287262bdc0d5196d0dc
MD5 8ea37c9219706f2406e5caef006ba388
BLAKE2b-256 0537475ee6463565140d15f259fce5dd2b0eb055c7fbdad9737cde99710d43c7

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

2.5.12 This release

12 files

2.3.6

12 files

1.27.1

14 files

1.26.2

1 file

1.24.13

1 file

1.24.12

1 file

1.24.11

1 file

1.24.10

1 file

1.24.9

1 file

1.24.8

14 files

1.24.7

14 files

1.24.6

14 files

1.21.0

14 files

1.17.1

14 files

1.17.0

14 files

1.10.6

4 files

1.10.5

4 files

1.10.4

4 files

1.10.3

4 files

1.10.2

4 files

1.10.1

4 files

1.10.0

4 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page