Skip to main content

Official Python SDK for DataCrunch Public API

Project description

DataCrunch Python SDK

The official DataCrunch.io Python SDK.

The SDK's documentation is available on ReadTheDocs

DataCrunch's Public API documentation is available here.

Getting Started - Using the SDK:

  • Install via pip:

    pip3 install datacrunch
    
  • Generate your client credentials - instructions in the public API docs.

  • Add the client secret to an environment variable (don't want it to be hardcoded):

    Linux (bash):

    export DATACRUNCH_CLIENT_SECRET=Z4CZq02rdwdB7ISV0k4Z2gtwAFKiyvr2U1l0KDIeYi
    

    Other platforms: https://en.wikipedia.org/wiki/Environment_variable

  • Example for creating a new instance:

    import os
    from datacrunch import DataCrunchClient
    
    # Get client secret from environment variable
    CLIENT_SECRET = os.environ['DATACRUNCH_CLIENT_SECRET']
    CLIENT_ID = 'Ibk5bdxV64lKAWOqYnvSi'
    
    # Create datcrunch client
    datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET)
    
    # Get all SSH keys
    ssh_keys = datacrunch.ssh_keys.get()
    ssh_keys = list(map(lambda key: key.id, ssh_keys))
    
    # Create a new instance
    instance = datacrunch.instances.create(instance_type='1V100.6V',
                                          image='fastai',
                                          ssh_key_ids=ssh_keys,
                                          hostname='example',
                                          description='example instance')
    
    # Delete instance
    datacrunch.instances.action(instance.id, datacrunch.constants.instance_actions.DELETE)
    

    More examples can be found in the /examples folder or in the documentation.

Development

Setting up the local development environment

  • Clone the repository:

    git clone
    
  • Create local virtual environment:

    python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate
    

    or if using fish shell:

    python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate.fish
    
  • Install Dependencies:

    pip3 install -e .[test]
    pip3 install -U pytest
    

Running Tests

We use pytest for testing.

  • To execute all tests

    pytest
    
  • To execute a single test file

    pytest ./tests/unit_tests/test_file.py
    

Local Manual Testing

Create this file in the root directory of the project:

from datacrunch.datacrunch import DataCrunchClient

CLIENT_SECRET = 'secret'
CLIENT_ID = 'Ibk5bdxV64lKAWOqYnvSi'

# Create datcrunch client
datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET, base_url='http://localhost:3001/v1')

Generating the documentation

If added a new service, create a documentation template under api/services for that service.

cd docs
make html

Style Guide

Use autopep8 for auto code formatting:

# Install
pip3 install autopep8

# Apply to an entire directory
autopep8 directory_name --recursive --in-place --pep8-passes 2000 --verbose

# Or a single file
autopep8 file.py --in-place

Contact

You can contact us here, or send a message / open an issue in the repo.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datacrunch-1.6.0.tar.gz (28.6 kB view details)

Uploaded Source

Built Distribution

datacrunch-1.6.0-py3-none-any.whl (42.3 kB view details)

Uploaded Python 3

File details

Details for the file datacrunch-1.6.0.tar.gz.

File metadata

  • Download URL: datacrunch-1.6.0.tar.gz
  • Upload date:
  • Size: 28.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for datacrunch-1.6.0.tar.gz
Algorithm Hash digest
SHA256 1793569421ad1dea0dd12ab97ea41ed1055d47914947dfcfaea257767e22cbec
MD5 74da17f0832f62ddeadf12fffcf02206
BLAKE2b-256 a002b0803877d08b91fa840fcf176c402922a74b71e661f0022cf4fa70c5c3be

See more details on using hashes here.

File details

Details for the file datacrunch-1.6.0-py3-none-any.whl.

File metadata

  • Download URL: datacrunch-1.6.0-py3-none-any.whl
  • Upload date:
  • Size: 42.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for datacrunch-1.6.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b62881a48f99c7f98c27ffa210f1fa1f47e83034a759153bad3e02d43e89f953
MD5 ccb1f679c2b4d69fe775442c4ee84d08
BLAKE2b-256 6ced9b89e3de91df9f7ddd06849afbf41a06fcab1ec7b0d57859678c1ba0d130

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page