Skip to main content

Parallel and distributed training with spaCy and Ray

Project description

spacy-ray: Parallel and distributed training with spaCy and Ray

⚠️ This repo is still a work in progress and requires the new spaCy v3.0.

Ray is a fast and simple framework for building and running distributed applications. This very lightweight extension package lets you use Ray for parallel and distributed training with spaCy. If spacy-ray is installed in the same environment as spaCy, it will automatically add spacy ray commands to your spaCy CLI.

The main command is spacy ray train for parallel and distributed training, but we expect to add spacy ray pretrain and spacy ray parse as well.

Azure Pipelines Current Release Version PyPi Version

🚀 Quickstart

You can install spacy-ray from pip:

pip install spacy-ray

To check if the command has been registered successfully:

python -m spacy ray --help

Train a model using the same API as spacy train:

python -m spacy ray train config.cfg --n-workers 2

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spacy_ray-0.1.4.tar.gz (12.3 kB view details)

Uploaded Source

Built Distribution

spacy_ray-0.1.4-py2.py3-none-any.whl (13.3 kB view details)

Uploaded Python 2Python 3

File details

Details for the file spacy_ray-0.1.4.tar.gz.

File metadata

  • Download URL: spacy_ray-0.1.4.tar.gz
  • Upload date:
  • Size: 12.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.7.9

File hashes

Hashes for spacy_ray-0.1.4.tar.gz
Algorithm Hash digest
SHA256 8aace86e08dfb8169b14e2a8ab18c7da5b6977d0c6ba286403bb5a443327b8dd
MD5 5de24d5b149d385151e2aaf0aaae0d8d
BLAKE2b-256 d309034a71f86e7d0880272a667e7f56df25dde0124afb0d4ed748fb5da01cb9

See more details on using hashes here.

File details

Details for the file spacy_ray-0.1.4-py2.py3-none-any.whl.

File metadata

  • Download URL: spacy_ray-0.1.4-py2.py3-none-any.whl
  • Upload date:
  • Size: 13.3 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.7.9

File hashes

Hashes for spacy_ray-0.1.4-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 ebbe8268ee642eb460a18dc243269bb340c31214d32409c5b9c21beffecf0bce
MD5 8634b8732970c4c690cee6315c808b33
BLAKE2b-256 2f6526795a2901c38466b64e0a17447d987ec31b50145ef4d3fc16c94f9480ea

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page