datacrunch 1.15.0

Official Python SDK for DataCrunch Public API

DataCrunch Python SDK

The official DataCrunch.io Python SDK.

The SDK's documentation is available on ReadTheDocs

DataCrunch's Public API documentation is available here.

Getting Started - Using the SDK:

• Install via pip:

  pip3 install datacrunch
  

• Generate your client credentials - instructions in the public API docs.

• Add your client id and client secret to an environment variable (don't want it to be hardcoded):

  Linux (bash):

  export DATACRUNCH_CLIENT_ID=YOUR_ID_HERE
  export DATACRUNCH_CLIENT_SECRET=YOUR_SECRET_HERE
  

• To enable sending inference requests from SDK you must generate an inference key - Instructions on inference authorization

• Add your inference key to an environment variable

  Linux (bash):

  export DATACRUNCH_INFERENCE_KEY=YOUR_API_KEY_HERE
  

  Other platforms: https://en.wikipedia.org/wiki/Environment_variable

• Example for creating a new instance:

  import os
  from datacrunch import DataCrunchClient
  
  # Get credentials from environment variables
  CLIENT_ID = os.environ.get('DATACRUNCH_CLIENT_ID')
  CLIENT_SECRET = os.environ['DATACRUNCH_CLIENT_SECRET']
  
  # Create datcrunch client
  datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET)
  
  # Get all SSH keys
  ssh_keys = datacrunch.ssh_keys.get()
  ssh_keys = list(map(lambda key: key.id, ssh_keys))
  
  # Create a new instance
  instance = datacrunch.instances.create(instance_type='1V100.6V',
                                        image='ubuntu-24.04-cuda-12.8-open-docker',
                                        ssh_key_ids=ssh_keys,
                                        hostname='example',
                                        description='example instance')
  
  # Delete instance
  datacrunch.instances.action(instance.id, datacrunch.constants.instance_actions.DELETE)
  

  More examples can be found in the /examples folder or in the documentation.

Development

Setting up the local development environment

• Clone the repository:

  git clone
  

• Create local virtual environment:

  python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate
  

  or if using fish shell:

  python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate.fish
  

• Install Dependencies:

  pip3 install -e .[test]
  pip3 install -U pytest
  

Running Tests

We use pytest for testing.

• To execute all tests

  pytest
  

• To execute a single test file

  pytest ./tests/unit_tests/test_file.py
  

Local Manual Testing

Create this file in the root directory of the project:

from datacrunch.datacrunch import DataCrunchClient

CLIENT_SECRET = 'secret'
CLIENT_ID = 'your-id'

# Create datcrunch client
datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET, base_url='http://localhost:3001/v1')

Generating the documentation

If added a new service, create a documentation template under api/services for that service.

cd docs
make html

Style Guide

Use autopep8 for auto code formatting:

# Install
pip3 install autopep8

# Apply to an entire directory
autopep8 directory_name --recursive --in-place --pep8-passes 2000 --verbose

# Or a single file
autopep8 file.py --in-place

Contact

You can contact us here, or open an issue in the repo.