Skip to main content

NotDiamond Python library for easy OpenAI integration.

Project description

Getting started with Not Diamond

Not Diamond helps you leverage multiple LLMs dynamically, sending queries to the best-suited model in real-time.

Key features

  • Not a proxy: When we receive a query, our backend service returns a recommendation of the model you should call and the actual call goes out client-side. So we never see your response outputs.
  • Privacy preserving: To protect your privacy, we use fuzzy hashing to conceal your query strings from us, and our model recommendation engine works entirely using the fuzzy hashed queries. This means we also never see the raw query strings you send to an LLM.
  • Maximize performance: By dynamically routing to the best-suited LLM for each query, we improve your overall LLM output quality and enhance product quality. Our recommendations are based on millions of data points from rigorous evaluation benchmarks and real-world data.
  • Reduce cost and latency: Most of the time, a small and specialized model can optimally handle a given query. We can help you determine this in real-time, cutting your LLM inference costs while reducing latency.

👍 Free to use!

Not Diamond is free up to 100K monthly requests. Beyond this pay just $10 per 10K requests.

Installation

Requires Python 3.9+

pip install notdiamond

API keys

Sign up and get a Not Diamond API key.

Create a .env file with your Not Diamond API key, and the API keys of the models you want to route between.

OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
GOOGLE_API_KEY="YOUR_GOOGLE_API_KEY"
ANTHROPIC_API_KEY="YOUR_ANTHROPIC_API_KEY"
NOTDIAMOND_API_KEY="YOUR_NOTDIAMOND_API_KEY"

Alternatively, you can also set API keys programmatically as described further below.

📘 API keys

The notdiamond library uses your API keys client-side to call the LLM we recommend. We never pass your keys to our servers.

🚧 Model access

Since notdiamond calls the LLMs client-side using your keys, we will only call models you have access to. You can also use our router to determine the best model to call regardless of whether you have access or not (see example). Our router supports most of the popular open and proprietary models (see full list).

Drop me a line if you have a specific model requirement and we're happy to work with you to support it.

Example

If you already have existing projects in either OpenAI SDK or LangChain, check out our OpenAI and Langchain integration guides. Otherwise, continue reading.

Create a main.py file in the same folder as the .env file you created earlier, or try it in Colab.

from notdiamond.llms.llm import NDLLM
from notdiamond.prompts.prompt import NDPrompt, NDContext, NDQuery, NDPromptTemplate


# Define your prompt and query
prompt = NDPrompt("You are a world class software developer.") # The system prompt, defines the LLM's role
query = NDQuery("Write a merge sort in Python.") # The specific query written by an end-user

# Define the prompt template to combine prompt and query into a single string
prompt_template = NDPromptTemplate("About you: {prompt}\n{query}",
                                   partial_variables={"prompt": prompt, "query": query})

# Define the available LLMs you'd like to route between
llm_providers = ['openai/gpt-3.5-turbo', 'openai/gpt-4','openai/gpt-4-1106-preview', 'openai/gpt-4-turbo-preview',
                 'anthropic/claude-2.1', 'anthropic/claude-3-sonnet-20240229', 'anthropic/claude-3-opus-20240229',
                 'google/gemini-pro']

# Create the NDLLM object -> like a 'meta-LLM' combining all of the specified models
nd_llm = NDLLM(llm_providers=llm_providers)

# After fuzzy hashing the inputs, the best LLM is determined by the ND API and the LLM is called client-side
result, session_id, provider = nd_llm.invoke(prompt_template=prompt_template)


print("ND session ID: ", session_id)  # A unique ID of the invoke. Important for future references back to ND API
print("LLM called: ", provider.model)  # The LLM routed to
print("LLM output: ", result.content)  # The LLM response

👍 Run it!

python main.py

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

notdiamond_openai-0.1.4.tar.gz (145.5 kB view details)

Uploaded Source

Built Distribution

notdiamond_openai-0.1.4-py3-none-any.whl (265.4 kB view details)

Uploaded Python 3

File details

Details for the file notdiamond_openai-0.1.4.tar.gz.

File metadata

  • Download URL: notdiamond_openai-0.1.4.tar.gz
  • Upload date:
  • Size: 145.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.2

File hashes

Hashes for notdiamond_openai-0.1.4.tar.gz
Algorithm Hash digest
SHA256 babb3539cc98ad254c71753691c31e809bb95b65d8ece529116a05e1464feb5a
MD5 f28c18d1283e3e70ad3f05965cb258d2
BLAKE2b-256 fda46737f193c81984cd97e7bb152f6f7f165d1c52e471082e8f714418f4f01c

See more details on using hashes here.

File details

Details for the file notdiamond_openai-0.1.4-py3-none-any.whl.

File metadata

File hashes

Hashes for notdiamond_openai-0.1.4-py3-none-any.whl
Algorithm Hash digest
SHA256 be21395fe0fdb0220e9812bbcd9bd631bd397d4b6f45ced6a48861c7055753fa
MD5 faadb2c6ec1e988833cd2a39995aa228
BLAKE2b-256 ae5ac8418409166608d6360c9869bf4d3c677c555d666c2e901af681e4561880

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page