covalent 0.220.0.post1

Covalent Workflow Tool

 

🤔 What is Covalent?

Covalent is a Pythonic workflow tool for computational scientists, AI/ML software engineers, and anyone who needs to run experiments on limited or expensive computing resources including quantum computers, HPC clusters, GPU arrays, and cloud services.

Covalent enables a researcher to run computation tasks on an advanced hardware platform – such as a quantum computer or serverless HPC cluster – using a single line of code.

💭 Why Covalent?

Covalent overcomes computational and operational challenges inherent in AI/ML experimentation.

Computational challenges	Operational challenges
Advanced compute hardware is expensive, and access is often limited – shared with other researchers, for example. You'd like to iterate quickly, but running large models takes time. Parallel computation speeds execution, but requires careful attention to data relationships.	Proliferation of models, datasets, and hardware trials. Switching between development tools, including notebooks, scripts, and submission queues. Tracking, repeating, and sharing results.

With Covalent, you:

• Assign functions to appropriate resources: Use advanced hardware (quantum computers, HPC clusters) for the heavy lifting and commodity hardware for bookkeeping.
• Test functions on local servers before shipping them to advanced hardware.
• Let Covalent's services analyze functions for data independence and automatically parallelize them.
• Run experiments from a Jupyter notebook (or whatever your preferred interactive Python environment is).
• Track workflows and examine results in a browser-based GUI.

📦 Installation

Covalent is developed using Python version 3.8 on Linux and macOS. The easiest way to install Covalent is using the PyPI package manager:

pip install covalent

Refer to the Quick Start guide for quick setup instructions, or to the First Experiment guide for a more thorough approach. For a full list of supported platforms, see the Covalent compatibility matrix.

🚀 Getting started

Ready to try it? Go to the First Experiment guide in the documentation.

For a more in-depth description of Covalent's features and how they work, see the Concepts page in the documentation.

📚 Know more !	✍️ Tutorials and Examples
What is Covalent? Covalent in the era of cloud-HPC Basic Concepts of Covalent How does Covalent work?	Covalent with pytorch for classical machine learning Covalent with pennylane for quantum machine learning Covalent with Qiskit for quantum-SVM Covalent with Dask for scaling Ensemble classification Covalent for Deep Neural Network on AWS

How Does It Work?

Covalent has three main components:

• A Python module containing an API that you use to build manageable workflows out of new or existing Python functions.
• A set of services that run locally or on a server to dispatch and execute workflow tasks.
• A browser-based UI from which to manage workflows and view results.

You compose workflows using the Covalent API by simply adding a single line of python decorator and submit them to the Covalent server. The server analyzes the workflow to determine dependencies between tasks, then dispatches each task to its specified execution backend. Independent tasks are executed concurrently if resources are available.

The Covalent UI displays the progress of each workflow at the level of individual tasks.

The Covalent API

The Covalent API is a Python module containing a small collection of classes that implement server-based workflow management. The key elements are two decorators that wrap functions to create managed *tasks* and *workflows*.

The task decorator is called an electron. The electron decorator simply turns the function into a dispatchable task.

The workflow decorator is called a lattice. The lattice decorator turns a function composed of electrons into a manageable workflow.

Covalent Services

The Covalent server is a lightweight service that runs on your local machine or a server. A dispatcher analyzes workflows (lattices) and hands its component functions (electrons) off to executors. Each executor is an adaptor to a backend hardware resource. Covalent has a growing list of turn-key executors for common compute backends. If no executor exists yet for your compute platform, Covalent supports writing your own.

The Covalent GUI

The Covalent user interface runs as a web server on the machine where the Covalent server is running. The GUI dashboard shows a list of dispatched workflows. From there, you can drill down to workflow details or a graphical view of the workflow. You can also view logs, settings, and result sets.

📚 Documentation

The official documentation includes tips on getting started, high-level concepts, tutorials, and the API documentation, and more. To learn more, see the Covalent documentation.

Troubleshooting

Solutions to common issues can be found in the Troubleshooting Guide.

✔️ Contributing

To contribute to Covalent, refer to the Contribution Guidelines. We use GitHub's issue tracking to manage known issues, bugs, and pull requests. Get started by forking the develop branch and submitting a pull request with your contributions. Improvements to the documentation, including tutorials and how-to guides, are also welcome from the community. For more more information on adding tutorials, check the Tutorial Guidelines Participation in the Covalent community is governed by the Code of Conduct.

⚓ Citation

Please use the following citation in any publications:

https://doi.org/10.5281/zenodo.5903364

📃 License

Covalent is licensed under the Apache 2.0 License. See the LICENSE file or contact the support team for more details.

For a detailed history of changes and new features, see the Changelog.