Codeflare-SDK

Python application coverage badge

An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute resources, either in the cloud or on-prem.

For guided demos and basics walkthroughs, check out the following links:

  • Guided demo notebooks available here, and copies of the notebooks with expected output also available
  • Note that these notebooks will work with the latest codeflare-sdk PyPI release. For testing and experimentation with main branch, please use the preview notebooks
  • Additionally, we have a video walkthrough of these basic demos from June, 2023

Full documentation can be found here

Installation

Can be installed via pip: pip install codeflare-sdk

Development

Prerequisites

We recommend using Python 3.9 for development, along with Poetry. Create a Poetry virtual environment with the required Python version 3.9, and run all commands within this environment.

  • run: poetry shell

Install dependencies:

  • run: poetry install

This will install standard requirements as specified in the poetry.lock file. Test and docs dependencies are optional.

  • To include test dependencies run: poetry install --with test

  • To include docs dependencies run: poetry install --with docs

  • To include test and docs dependencies run: poetry install --with test,docs

If you require a requirements.txt file you can run:

poetry export -f requirements.txt --output requirements.txt --without-hashes

Pre-commit

We use pre-commit to make sure the code is consistently formatted. To make sure that pre-commit is run every time you commit changes, simply run pre-commit install

To build the codeflare-sdk pre-commit image run podman build -f .github/build/Containerfile . from the root directory.

Testing

  • To install codeflare-sdk in editable mode, run pip install -e . from the repo root.
  • Any new test functions/scripts can be added into the tests folder
  • NOTE: Functional tests coming soon, will live in tests/func_test.py

Unit Testing

  • To run the unit tests, run pytest -v tests/unit_test.py

Local e2e Testing

Code Coverage

  • Run tests with the following command: coverage run -m --source=src pytest tests/unit_test.py
  • To then view a code coverage report w/ missing lines, run coverage report -m

Code Formatting

  • To check file formatting, in top-level dir run black --check .
  • To auto-reformat all files, remove the --check flag
  • To reformat an individual file, run black <filename>

Package Build

To build the python package: $ poetry build

Release Instructions

Automated Releases

It is possible to use the Release Github workflow to do the release. This is generally the process we follow for releases

Manual Releases

The following instructions apply when doing release manually. This may be required in instances where the automation is failing.

  • Check and update the version in "pyproject.toml" file.
  • Generate new documentation. pdoc --html -o docs src/codeflare_sdk && pushd docs && rm -rf cluster job utils && mv codeflare_sdk/* . && rm -rf codeflare_sdk && popd && find docs -type f -name "*.html" -exec bash -c "echo '' >> {}" \; (it is possible to install pdoc using the following command poetry install --with docs)
  • Commit all the changes to the repository.
  • Create Github release (https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release).
  • Build the Python package. poetry build
  • If not present already, add the API token to Poetry. poetry config pypi-token.pypi API_TOKEN
  • Publish the Python package. poetry publish
  • Change directory to custom-nb-image. cd custom-nb-image
  • Set tag export tag=TAG
  • Build the container image. podman build --build-arg SDK_VERSION=<version> -t quay.io/project-codeflare/notebook:${tag} .
  • Login to quay.io. podman login quay.io
  • Push the image. podman push quay.io/project-codeflare/notebook:${tag}
  • Push the stable image tag podman push quay.io/project-codeflare/notebook:${tag} quay.io/project-codeflare/notebook:stable

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.38.2)

10 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for arceofrancisco from gravatar.com arceofrancisco Avatar for bobbins228 from gravatar.com bobbins228 Avatar for chipspeak from gravatar.com chipspeak
+7 more Avatar for eoinfennessy from gravatar.com eoinfennessy Avatar for kpostlet from gravatar.com kpostlet Avatar for kryanbeane from gravatar.com kryanbeane Avatar for laurafitzgerald from gravatar.com laurafitzgerald Avatar for pmccarthy from gravatar.com pmccarthy Avatar for ppaszki from gravatar.com ppaszki Avatar for szaher from gravatar.com szaher

Credits

Author: Michael Clifford

License

Apache Software License (Apache-2.0)

Requires

Python <4.0, >=3.8

Tags

codeflare python sdk client batch scale

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

codeflare_sdk-0.15.0.tar.gz (32.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

codeflare_sdk-0.15.0-py3-none-any.whl (38.6 kB view details)

Uploaded Python 3

File details

Details for the file codeflare_sdk-0.15.0.tar.gz.

File metadata

  • Download URL: codeflare_sdk-0.15.0.tar.gz
  • Upload date:
  • Size: 32.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for codeflare_sdk-0.15.0.tar.gz
Algorithm Hash digest
SHA256 9efaa7f50397a862126cac4678fb5ef24667346f90085c7d8d151d37850185fd
MD5 3d755fba22751a378408e97a124ec71c
BLAKE2b-256 fd42139a7f1c23357174d18105bc87b74fcbe38a2ba8807c3c60ae6c0ec6331b

See more details on using hashes here.

File details

Details for the file codeflare_sdk-0.15.0-py3-none-any.whl.

File metadata

  • Download URL: codeflare_sdk-0.15.0-py3-none-any.whl
  • Upload date:
  • Size: 38.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for codeflare_sdk-0.15.0-py3-none-any.whl
Algorithm Hash digest
SHA256 4379412625fff9c58ff521a0036b4f98a07db641cee752cdfb031153b4ece3d3
MD5 46bfc794f99e39d19540bed269e11712
BLAKE2b-256 808edfb52d36be31a513bd77191de7353cb302ed3dbed6332336a3fce2771c9c

See more details on using hashes here.

Release history Release notifications | RSS feed

0.38.2

2 files

0.38.1

2 files

0.38.0

2 files

0.37.0

2 files

0.36.0

2 files

0.35.0

2 files

0.34.0

2 files

0.33.1

2 files

0.33.0

2 files

0.32.2

2 files

0.32.1

2 files

0.32.0

2 files

0.31.5

2 files

0.31.3

2 files

0.31.2

2 files

0.31.1

2 files

0.31.0

2 files

0.30.0

2 files

0.29.0

2 files

0.28.1

2 files

0.28.0

2 files

0.27.0

2 files

0.26.0

2 files

0.25.0

2 files

0.24.3

2 files

0.24.2

2 files

0.24.1

2 files

0.24.0

2 files

0.23.1

2 files

0.23.0

2 files

0.22.0

2 files

0.21.1

2 files

0.21.0

2 files

0.20.2

2 files

0.20.0

2 files

0.19.1

2 files

0.19.0

2 files

0.18.0

2 files

0.17.0

2 files

0.16.4

2 files

0.16.3

2 files

0.16.2

2 files

0.16.1

2 files

0.16.0

2 files

0.15.1

2 files

This release

0.15.0 This release

2 files

0.14.1

2 files

0.14.0

2 files

0.13.0

2 files

0.12.1

2 files

0.12.0

2 files

0.11.0

2 files

0.10.1

2 files

0.10.0

2 files

0.9.0

2 files

0.8.0

2 files

0.7.1

2 files

0.7.0

2 files

0.6.1

2 files

0.5.0

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

0.1.0

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page