Skip to main content

The Oshepherd guiding the Ollama(s) inference orchestration.

Project description

oshepherd

The Oshepherd guiding the Ollama(s) inference orchestration.

A centralized Flask API service, using Celery (RabbitMQ + Redis) to orchestrate multiple Ollama workers.

Install

pip install oshepherd

Usage

  1. Setup RabbitMQ and Redis:

    Create instances for free for both: * cloudamqp.com * redislabs.com

  2. Setup Flask API Server:

    # define configuration env file
    #   use credentials for redis and rabbitmq
    cp .api.env.template .api.env
    
    # start api
    oshepherd start-api --env-file .api.env
    
  3. Setup Celery/Ollama Worker(s):

    # install ollama https://ollama.com/download
    ollama run mistral
    
    # define configuration env file
    #   use credentials for redis and rabbitmq
    cp .worker.env.template .worker.env
    
    # start worker
    oshepherd start-worker --env-file .worker.env
    

Words of advice 🚨

This package is in alpha, its architecture and api might change in the near future. Currently this is getting tested in a closed environment by real users, but haven't been audited, nor tested thorugly. Use it at your own risk.

Disclaimer on Support

As this is an alpha version, support and responses might be limited. We'll do our best to address questions and issues as quickly as possible.

Contribution Guidelines

We welcome contributions! If you find a bug or have suggestions for improvements, please open an issue or submit a pull request.

Conda Support

To run and build locally you can use conda:

conda create -n oshepherd python=3.8
conda activate oshepherd
pip install -r requirements.txt

# install oshepherd
pip install -e .
Tests

Follow usage instructions to start api server and celery worker using a local ollama, and then run the tests:

pytest -s tests/

Reporting Issues

Please report any issues you encounter on the GitHub issues page. Before creating a new issue, take a moment to search through the existing issues to avoid duplicates.

Author

Currently, mnemonica.ai is sponsoring the development of this tool.

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

oshepherd-0.0.4.tar.gz (8.3 kB view details)

Uploaded Source

Built Distribution

oshepherd-0.0.4-py3-none-any.whl (9.1 kB view details)

Uploaded Python 3

File details

Details for the file oshepherd-0.0.4.tar.gz.

File metadata

  • Download URL: oshepherd-0.0.4.tar.gz
  • Upload date:
  • Size: 8.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.8.19

File hashes

Hashes for oshepherd-0.0.4.tar.gz
Algorithm Hash digest
SHA256 dd585c916a705d2c0f9c52b85b455bc546e9b71aeb058395332558da2ecb88af
MD5 fc0dd94297cfa9103ebf1a0e524bbe81
BLAKE2b-256 0c51ff278da9806885c259c8594622900b836bd6b09fb1ccbe0dc4aef01d9ba0

See more details on using hashes here.

File details

Details for the file oshepherd-0.0.4-py3-none-any.whl.

File metadata

  • Download URL: oshepherd-0.0.4-py3-none-any.whl
  • Upload date:
  • Size: 9.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.8.19

File hashes

Hashes for oshepherd-0.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 4fdbf51c787d68927a0f2bb03042945ae7736afc19d5cdd263a23862364166f8
MD5 0adbd8b757d8d6c636596196229bf1b4
BLAKE2b-256 c1d61816ab606a57189920394c77af58183aecc8860f39465021eb73cf0ebf75

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page