spacy-ray: Parallel and distributed training with spaCy and Ray
⚠️ This repo is still a work in progress and requires the new spaCy v3.0.
Ray is a fast and simple framework for building and running distributed applications. This very lightweight extension package lets you use Ray for parallel and distributed training with spaCy. If spacy-ray is installed in the same environment as spaCy, it will automatically add spacy ray commands to your spaCy CLI.
The main command is spacy ray train for
parallel and distributed training, but we expect to add spacy ray pretrain and spacy ray parse as well.
🚀 Quickstart
You can install spacy-ray from pip:
pip install spacy-ray
To check if the command has been registered successfully:
python -m spacy ray --help
Train a model using the same API as spacy train:
python -m spacy ray train config.cfg --n-workers 2
1 maintainer
Metadata
1 maintainer
Release files for spacy-ray 0.1.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| spacy_ray-0.1.4.tar.gz | 12.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| spacy_ray-0.1.4-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details |
Total release size: 25.6 kB
Release files / spacy_ray-0.1.4.tar.gz
| Download URL | spacy_ray-0.1.4.tar.gz |
|---|---|
| Size | 12.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8aace86e08dfb8169b14e2a8ab18c7da5b6977d0c6ba286403bb5a443327b8dd
|
|
BLAKE2b-256 checksum How to use checksums |
d309034a71f86e7d0880272a667e7f56df25dde0124afb0d4ed748fb5da01cb9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.7.9
|
Release files / spacy_ray-0.1.4-py2.py3-none-any.whl
| Download URL | spacy_ray-0.1.4-py2.py3-none-any.whl |
|---|---|
| Size | 13.3 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
ebbe8268ee642eb460a18dc243269bb340c31214d32409c5b9c21beffecf0bce
|
|
BLAKE2b-256 checksum How to use checksums |
2f6526795a2901c38466b64e0a17447d987ec31b50145ef4d3fc16c94f9480ea
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.7.9
|
