Codeflare-SDK

Python application coverage badge

An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute resources, either in the cloud or on-prem.

For guided demos and basics walkthroughs, check out the following links:

  • Guided demo notebooks available here, and copies of the notebooks with expected output also available
  • Note that these notebooks will work with the latest codeflare-sdk PyPI release. For testing and experimentation with main branch, please use the preview notebooks
  • Additionally, we have a video walkthrough of these basic demos from June, 2023

Full documentation can be found here

Installation

Can be installed via pip: pip install codeflare-sdk

Development

Prerequisites

We recommend using Python 3.9 for development, along with Poetry. Create a Poetry virtual environment with the required Python version 3.9, and run all commands within this environment.

  • run: poetry shell

Install dependencies:

  • run: poetry install

This will install standard requirements as specified in the poetry.lock file. Test and docs dependencies are optional.

  • To include test dependencies run: poetry install --with test

  • To include docs dependencies run: poetry install --with docs

  • To include test and docs dependencies run: poetry install --with test,docs

If you require a requirements.txt file you can run:

poetry export -f requirements.txt --output requirements.txt --without-hashes

Pre-commit

We use pre-commit to make sure the code is consistently formatted. To make sure that pre-commit is run every time you commit changes, simply run pre-commit install

To build the codeflare-sdk pre-commit image run podman build -f .github/build/Containerfile . from the root directory.

Testing

  • To install codeflare-sdk in editable mode, run pip install -e . from the repo root.
  • Any new test functions/scripts can be added into the tests folder
  • NOTE: Functional tests coming soon, will live in tests/func_test.py

Unit Testing

  • To run the unit tests, run pytest -v tests/unit_test.py

Local e2e Testing

Code Coverage

  • Run tests with the following command: coverage run -m --source=src pytest tests/unit_test.py
  • To then view a code coverage report w/ missing lines, run coverage report -m

Code Formatting

  • To check file formatting, in top-level dir run black --check .
  • To auto-reformat all files, remove the --check flag
  • To reformat an individual file, run black <filename>

Package Build

To build the python package: $ poetry build

Release Instructions

Automated Releases

It is possible to use the Release Github workflow to do the release. This is generally the process we follow for releases

Manual Releases

The following instructions apply when doing release manually. This may be required in instances where the automation is failing.

  • Check and update the version in "pyproject.toml" file.
  • Generate new documentation. pdoc --html -o docs src/codeflare_sdk && pushd docs && rm -rf cluster job utils && mv codeflare_sdk/* . && rm -rf codeflare_sdk && popd && find docs -type f -name "*.html" -exec bash -c "echo '' >> {}" \; (it is possible to install pdoc using the following command poetry install --with docs)
  • Commit all the changes to the repository.
  • Create Github release (https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release).
  • Build the Python package. poetry build
  • If not present already, add the API token to Poetry. poetry config pypi-token.pypi API_TOKEN
  • Publish the Python package. poetry publish
  • Change directory to custom-nb-image. cd custom-nb-image
  • Set tag export tag=TAG
  • Build the container image. podman build --build-arg SDK_VERSION=<version> -t quay.io/project-codeflare/notebook:${tag} .
  • Login to quay.io. podman login quay.io
  • Push the image. podman push quay.io/project-codeflare/notebook:${tag}
  • Push the stable image tag podman push quay.io/project-codeflare/notebook:${tag} quay.io/project-codeflare/notebook:stable

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.39.0)

10 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for arceofrancisco from gravatar.com arceofrancisco Avatar for bobbins228 from gravatar.com bobbins228 Avatar for chipspeak from gravatar.com chipspeak
+7 more Avatar for eoinfennessy from gravatar.com eoinfennessy Avatar for kpostlet from gravatar.com kpostlet Avatar for kryanbeane from gravatar.com kryanbeane Avatar for laurafitzgerald from gravatar.com laurafitzgerald Avatar for pmccarthy from gravatar.com pmccarthy Avatar for ppaszki from gravatar.com ppaszki Avatar for szaher from gravatar.com szaher

Credits

Author: Michael Clifford

License

Apache Software License (Apache-2.0)

Requires

Python >=3.8, <4.0

Tags

codeflare python sdk client batch scale

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

codeflare_sdk-0.14.1.tar.gz (36.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

codeflare_sdk-0.14.1-py3-none-any.whl (43.6 kB view details)

Uploaded Python 3

File details

Details for the file codeflare_sdk-0.14.1.tar.gz.

File metadata

  • Download URL: codeflare_sdk-0.14.1.tar.gz
  • Upload date:
  • Size: 36.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.8

File hashes

Hashes for codeflare_sdk-0.14.1.tar.gz
Algorithm Hash digest
SHA256 e9c3c9fb0d34df1a5a06038ab00fe0480c44f2f20686d54df6c5e1ef067a5a47
MD5 babfbcee3db364eb439ef2d0c302f450
BLAKE2b-256 39cf1867c657f0480c72825e3314c9506ab99b326d46973500bad0f5f2545991

See more details on using hashes here.

File details

Details for the file codeflare_sdk-0.14.1-py3-none-any.whl.

File metadata

  • Download URL: codeflare_sdk-0.14.1-py3-none-any.whl
  • Upload date:
  • Size: 43.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.8

File hashes

Hashes for codeflare_sdk-0.14.1-py3-none-any.whl
Algorithm Hash digest
SHA256 93e9f3e0324eb3d2f787da05db4875b23dcc59834a8950c2f79d29758d07835f
MD5 fdb9e6928aa1f1489545bbeca2e838ef
BLAKE2b-256 bdb66f407cee49a3dfd91ca266dcb868e2001e2f6e6242f7c34cffef751c54ba

See more details on using hashes here.

Release history Release notifications | RSS feed

0.39.0

2 files

0.38.2

2 files

0.38.1

2 files

0.38.0

2 files

0.37.0

2 files

0.36.0

2 files

0.35.0

2 files

0.34.0

2 files

0.33.1

2 files

0.33.0

2 files

0.32.2

2 files

0.32.1

2 files

0.32.0

2 files

0.31.5

2 files

0.31.3

2 files

0.31.2

2 files

0.31.1

2 files

0.31.0

2 files

0.30.0

2 files

0.29.0

2 files

0.28.1

2 files

0.28.0

2 files

0.27.0

2 files

0.26.0

2 files

0.25.0

2 files

0.24.3

2 files

0.24.2

2 files

0.24.1

2 files

0.24.0

2 files

0.23.1

2 files

0.23.0

2 files

0.22.0

2 files

0.21.1

2 files

0.21.0

2 files

0.20.2

2 files

0.20.0

2 files

0.19.1

2 files

0.19.0

2 files

0.18.0

2 files

0.17.0

2 files

0.16.4

2 files

0.16.3

2 files

0.16.2

2 files

0.16.1

2 files

0.16.0

2 files

0.15.1

2 files

0.15.0

2 files

This release

0.14.1 This release

2 files

0.14.0

2 files

0.13.0

2 files

0.12.1

2 files

0.12.0

2 files

0.11.0

2 files

0.10.1

2 files

0.10.0

2 files

0.9.0

2 files

0.8.0

2 files

0.7.1

2 files

0.7.0

2 files

0.6.1

2 files

0.5.0

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

0.1.0

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page