Codeflare-SDK

Python application coverage badge

An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute resources, either in the cloud or on-prem.

For guided demos and basics walkthroughs, check out the following links:

  • Guided demo notebooks available here, and copies of the notebooks with expected output also available
  • Note that these notebooks will work with the latest codeflare-sdk PyPI release. For testing and experimentation with main branch, please use the preview notebooks
  • Additionally, we have a video walkthrough of these basic demos from June, 2023

Full documentation can be found here

Installation

Can be installed via pip: pip install codeflare-sdk

Development

Prerequisites

We recommend using Python 3.9 for development, along with Poetry. Create a Poetry virtual environment with the required Python version 3.9, and run all commands within this environment.

  • run: poetry shell

Install dependencies:

  • run: poetry install

This will install standard requirements as specified in the poetry.lock file. Test and docs dependencies are optional.

  • To include test dependencies run: poetry install --with test

  • To include docs dependencies run: poetry install --with docs

  • To include test and docs dependencies run: poetry install --with test,docs

If you require a requirements.txt file you can run:

poetry export -f requirements.txt --output requirements.txt --without-hashes

Pre-commit

We use pre-commit to make sure the code is consistently formatted. To make sure that pre-commit is run every time you commit changes, simply run pre-commit install

To build the codeflare-sdk pre-commit image run podman build -f .github/build/Containerfile . from the root directory.

Testing

  • To install codeflare-sdk in editable mode, run pip install -e . from the repo root.
  • Any new test functions/scripts can be added into the tests folder
  • NOTE: Functional tests coming soon, will live in tests/func_test.py

Unit Testing

  • To run the unit tests, run pytest -v tests/unit_test.py

Local e2e Testing

Code Coverage

  • Run tests with the following command: coverage run -m --source=src pytest tests/unit_test.py
  • To then view a code coverage report w/ missing lines, run coverage report -m

Code Formatting

  • To check file formatting, in top-level dir run black --check .
  • To auto-reformat all files, remove the --check flag
  • To reformat an individual file, run black <filename>

Package Build

To build the python package: $ poetry build

Release Instructions

Automated Releases

It is possible to use the Release Github workflow to do the release. This is generally the process we follow for releases

Manual Releases

The following instructions apply when doing release manually. This may be required in instances where the automation is failing.

  • Check and update the version in "pyproject.toml" file.
  • Generate new documentation. pdoc --html -o docs src/codeflare_sdk && pushd docs && rm -rf cluster job utils && mv codeflare_sdk/* . && rm -rf codeflare_sdk && popd && find docs -type f -name "*.html" -exec bash -c "echo '' >> {}" \; (it is possible to install pdoc using the following command poetry install --with docs)
  • Commit all the changes to the repository.
  • Create Github release (https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release).
  • Build the Python package. poetry build
  • If not present already, add the API token to Poetry. poetry config pypi-token.pypi API_TOKEN
  • Publish the Python package. poetry publish
  • Change directory to custom-nb-image. cd custom-nb-image
  • Set tag export tag=TAG
  • Build the container image. podman build --build-arg SDK_VERSION=<version> -t quay.io/project-codeflare/notebook:${tag} .
  • Login to quay.io. podman login quay.io
  • Push the image. podman push quay.io/project-codeflare/notebook:${tag}
  • Push the stable image tag podman push quay.io/project-codeflare/notebook:${tag} quay.io/project-codeflare/notebook:stable

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.39.1)

10 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for arceofrancisco from gravatar.com arceofrancisco Avatar for bobbins228 from gravatar.com bobbins228 Avatar for chipspeak from gravatar.com chipspeak
+7 more Avatar for eoinfennessy from gravatar.com eoinfennessy Avatar for kpostlet from gravatar.com kpostlet Avatar for kryanbeane from gravatar.com kryanbeane Avatar for laurafitzgerald from gravatar.com laurafitzgerald Avatar for pmccarthy from gravatar.com pmccarthy Avatar for ppaszki from gravatar.com ppaszki Avatar for szaher from gravatar.com szaher

Credits

Author: Michael Clifford

License

Apache Software License (Apache-2.0)

Requires

Python >=3.8, <4.0

Tags

codeflare python sdk client batch scale

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

codeflare_sdk-0.14.0.tar.gz (35.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

codeflare_sdk-0.14.0-py3-none-any.whl (43.3 kB view details)

Uploaded Python 3

File details

Details for the file codeflare_sdk-0.14.0.tar.gz.

File metadata

  • Download URL: codeflare_sdk-0.14.0.tar.gz
  • Upload date:
  • Size: 35.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.8

File hashes

Hashes for codeflare_sdk-0.14.0.tar.gz
Algorithm Hash digest
SHA256 3ea28cc2bd2c6c1fd205429fe8c4e111320b2183843b7f45f6a892f399c02f0f
MD5 4bd391e0c1cb0c8254d66d9cf3072dc8
BLAKE2b-256 f16a4cc0b2c2beddf9ee297de4edaee8f6ce46cd3db20e439327de502b1ea683

See more details on using hashes here.

File details

Details for the file codeflare_sdk-0.14.0-py3-none-any.whl.

File metadata

  • Download URL: codeflare_sdk-0.14.0-py3-none-any.whl
  • Upload date:
  • Size: 43.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.8

File hashes

Hashes for codeflare_sdk-0.14.0-py3-none-any.whl
Algorithm Hash digest
SHA256 3fc387381cfa05b1558388fd6bf575284b72e7c6472676dd445826fe1a0db402
MD5 df298ac059c0da9ddca17c2bfb936794
BLAKE2b-256 5ac5defe6e94d685a347fae238e054fca9f10c47f6266a864c0f53ee43af1cbb

See more details on using hashes here.

Release history Release notifications | RSS feed

0.39.1

2 files

0.39.0

2 files

0.38.2

2 files

0.38.1

2 files

0.38.0

2 files

0.37.0

2 files

0.36.0

2 files

0.35.0

2 files

0.34.0

2 files

0.33.1

2 files

0.33.0

2 files

0.32.2

2 files

0.32.1

2 files

0.32.0

2 files

0.31.5

2 files

0.31.3

2 files

0.31.2

2 files

0.31.1

2 files

0.31.0

2 files

0.30.0

2 files

0.29.0

2 files

0.28.1

2 files

0.28.0

2 files

0.27.0

2 files

0.26.0

2 files

0.25.0

2 files

0.24.3

2 files

0.24.2

2 files

0.24.1

2 files

0.24.0

2 files

0.23.1

2 files

0.23.0

2 files

0.22.0

2 files

0.21.1

2 files

0.21.0

2 files

0.20.2

2 files

0.20.0

2 files

0.19.1

2 files

0.19.0

2 files

0.18.0

2 files

0.17.0

2 files

0.16.4

2 files

0.16.3

2 files

0.16.2

2 files

0.16.1

2 files

0.16.0

2 files

0.15.1

2 files

0.15.0

2 files

0.14.1

2 files

This release

0.14.0 This release

2 files

0.13.0

2 files

0.12.1

2 files

0.12.0

2 files

0.11.0

2 files

0.10.1

2 files

0.10.0

2 files

0.9.0

2 files

0.8.0

2 files

0.7.1

2 files

0.7.0

2 files

0.6.1

2 files

0.5.0

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

0.1.0

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page