Codeflare-SDK

Python application coverage badge

An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute resources, either in the cloud or on-prem.

For guided demos and basics walkthroughs, check out the following links:

  • Guided demo notebooks available here, and copies of the notebooks with expected output also available
  • Note that these notebooks will work with the latest codeflare-sdk PyPI release. For testing and experimentation with main branch, please use the preview notebooks
  • Additionally, we have a video walkthrough of these basic demos from June, 2023

Full documentation can be found here

Installation

Can be installed via pip: pip install codeflare-sdk

Development

Prerequisites

We recommend using Python 3.9 for development, along with Poetry. Create a Poetry virtual environment with the required Python version 3.9, and run all commands within this environment.

  • run: poetry shell

Install dependencies:

  • run: poetry install

This will install standard requirements as specified in the poetry.lock file. Test and docs dependencies are optional.

  • To include test dependencies run: poetry install --with test

  • To include docs dependencies run: poetry install --with docs

  • To include test and docs dependencies run: poetry install --with test,docs

If you require a requirements.txt file you can run:

poetry export -f requirements.txt --output requirements.txt --without-hashes

Pre-commit

We use pre-commit to make sure the code is consistently formatted. To make sure that pre-commit is run every time you commit changes, simply run pre-commit install

To build the codeflare-sdk pre-commit image run podman build -f .github/build/Containerfile . from the root directory.

Testing

  • To install codeflare-sdk in editable mode, run pip install -e . from the repo root.
  • Any new test functions/scripts can be added into the tests folder
  • NOTE: Functional tests coming soon, will live in tests/func_test.py

Unit Testing

  • To run the unit tests, run pytest -v tests/unit_test.py

Local e2e Testing

Code Coverage

  • Run tests with the following command: coverage run -m --source=src pytest tests/unit_test.py
  • To then view a code coverage report w/ missing lines, run coverage report -m

Code Formatting

  • To check file formatting, in top-level dir run black --check .
  • To auto-reformat all files, remove the --check flag
  • To reformat an individual file, run black <filename>

Package Build

To build the python package: $ poetry build

Release Instructions

Automated Releases

It is possible to use the Release Github workflow to do the release. This is generally the process we follow for releases

Manual Releases

The following instructions apply when doing release manually. This may be required in instances where the automation is failing.

  • Check and update the version in "pyproject.toml" file.
  • Generate new documentation. pdoc --html -o docs src/codeflare_sdk && pushd docs && rm -rf cluster job utils && mv codeflare_sdk/* . && rm -rf codeflare_sdk && popd && find docs -type f -name "*.html" -exec bash -c "echo '' >> {}" \; (it is possible to install pdoc using the following command poetry install --with docs)
  • Commit all the changes to the repository.
  • Create Github release (https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release).
  • Build the Python package. poetry build
  • If not present already, add the API token to Poetry. poetry config pypi-token.pypi API_TOKEN
  • Publish the Python package. poetry publish
  • Change directory to custom-nb-image. cd custom-nb-image
  • Set tag export tag=TAG
  • Build the container image. podman build --build-arg SDK_VERSION=<version> -t quay.io/project-codeflare/notebook:${tag} .
  • Login to quay.io. podman login quay.io
  • Push the image. podman push quay.io/project-codeflare/notebook:${tag}
  • Push the stable image tag podman push quay.io/project-codeflare/notebook:${tag} quay.io/project-codeflare/notebook:stable

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.39.0)

10 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for arceofrancisco from gravatar.com arceofrancisco Avatar for bobbins228 from gravatar.com bobbins228 Avatar for chipspeak from gravatar.com chipspeak
+7 more Avatar for eoinfennessy from gravatar.com eoinfennessy Avatar for kpostlet from gravatar.com kpostlet Avatar for kryanbeane from gravatar.com kryanbeane Avatar for laurafitzgerald from gravatar.com laurafitzgerald Avatar for pmccarthy from gravatar.com pmccarthy Avatar for ppaszki from gravatar.com ppaszki Avatar for szaher from gravatar.com szaher

Credits

Author: Michael Clifford

License

Apache Software License (Apache-2.0)

Requires

Python <4.0, >=3.8

Tags

codeflare python sdk client batch scale

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

codeflare_sdk-0.15.1.tar.gz (32.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

codeflare_sdk-0.15.1-py3-none-any.whl (38.6 kB view details)

Uploaded Python 3

File details

Details for the file codeflare_sdk-0.15.1.tar.gz.

File metadata

  • Download URL: codeflare_sdk-0.15.1.tar.gz
  • Upload date:
  • Size: 32.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for codeflare_sdk-0.15.1.tar.gz
Algorithm Hash digest
SHA256 1b4d54c550354e612cad8439b1e7e332cd44767e9d9ab5670f6a4952bf3303d6
MD5 ef9d01d5f149d87ba1e38111eca3e824
BLAKE2b-256 a9d496e8e3d30f79c7d0a4695665afaa0cd0be0e6dea0633802f04447bf4e058

See more details on using hashes here.

File details

Details for the file codeflare_sdk-0.15.1-py3-none-any.whl.

File metadata

  • Download URL: codeflare_sdk-0.15.1-py3-none-any.whl
  • Upload date:
  • Size: 38.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for codeflare_sdk-0.15.1-py3-none-any.whl
Algorithm Hash digest
SHA256 b56c8d07d96db6e4a8ce795e2002fcd31f34e783025f5ae7881603ecb88f44a7
MD5 5812486de0807923ad65bd66470f2650
BLAKE2b-256 761855eb5ea791d8052a283cb5aa9f1a73593c8dc53abad29972fa455c0403b0

See more details on using hashes here.

Release history Release notifications | RSS feed

0.39.0

2 files

0.38.2

2 files

0.38.1

2 files

0.38.0

2 files

0.37.0

2 files

0.36.0

2 files

0.35.0

2 files

0.34.0

2 files

0.33.1

2 files

0.33.0

2 files

0.32.2

2 files

0.32.1

2 files

0.32.0

2 files

0.31.5

2 files

0.31.3

2 files

0.31.2

2 files

0.31.1

2 files

0.31.0

2 files

0.30.0

2 files

0.29.0

2 files

0.28.1

2 files

0.28.0

2 files

0.27.0

2 files

0.26.0

2 files

0.25.0

2 files

0.24.3

2 files

0.24.2

2 files

0.24.1

2 files

0.24.0

2 files

0.23.1

2 files

0.23.0

2 files

0.22.0

2 files

0.21.1

2 files

0.21.0

2 files

0.20.2

2 files

0.20.0

2 files

0.19.1

2 files

0.19.0

2 files

0.18.0

2 files

0.17.0

2 files

0.16.4

2 files

0.16.3

2 files

0.16.2

2 files

0.16.1

2 files

0.16.0

2 files

This release

0.15.1 This release

2 files

0.15.0

2 files

0.14.1

2 files

0.14.0

2 files

0.13.0

2 files

0.12.1

2 files

0.12.0

2 files

0.11.0

2 files

0.10.1

2 files

0.10.0

2 files

0.9.0

2 files

0.8.0

2 files

0.7.1

2 files

0.7.0

2 files

0.6.1

2 files

0.5.0

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.1

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

0.1.0

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page