Utility library for easily distributing code execution on clusters
Project description
# Cluster Tools
[![CircleCI](https://circleci.com/gh/scalableminds/cluster_tools/tree/master.svg?style=svg)](https://circleci.com/gh/scalableminds/cluster_tools/tree/master)
This package provides python `Executor` classes for distributing tasks on a slurm cluster or via multi processing.
## Example
```python
import cluster_tools
def square(n):
return n * n
if __name__ == '__main__':
strategy = "slurm" # other valid values are "multiprocessing" and "sequential"
with cluster_tools.get_executor(strategy) as executor:
result = list(executor.map(square, [2, 3, 4]))
assert result == [4, 9, 16]
```
## Credits
Thanks to [sampsyo/clusterfutures](https://github.com/sampsyo/clusterfutures) for providing the slurm core abstraction and [giovtorres/slurm-docker-cluster](https://github.com/giovtorres/slurm-docker-cluster) for providing the slurm docker environment which we use for CI based testing.
[![CircleCI](https://circleci.com/gh/scalableminds/cluster_tools/tree/master.svg?style=svg)](https://circleci.com/gh/scalableminds/cluster_tools/tree/master)
This package provides python `Executor` classes for distributing tasks on a slurm cluster or via multi processing.
## Example
```python
import cluster_tools
def square(n):
return n * n
if __name__ == '__main__':
strategy = "slurm" # other valid values are "multiprocessing" and "sequential"
with cluster_tools.get_executor(strategy) as executor:
result = list(executor.map(square, [2, 3, 4]))
assert result == [4, 9, 16]
```
## Credits
Thanks to [sampsyo/clusterfutures](https://github.com/sampsyo/clusterfutures) for providing the slurm core abstraction and [giovtorres/slurm-docker-cluster](https://github.com/giovtorres/slurm-docker-cluster) for providing the slurm docker environment which we use for CI based testing.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
cluster_tools-1.15.tar.gz
(7.4 kB
view hashes)