Skip to main content

Call LLM as easily as calling a taxi.

Project description

llm-taxi

Installation

pip install llm-taxi

Usage

Use as a library

import asyncio

from llm_taxi.conversation import Message, Role
from llm_taxi.factory import embedding, llm


async def main():
    # Non-streaming response
    client = llm(model="openai:gpt-3.5-turbo")
    messages = [
        Message(role=Role.User, content="What is the capital of France?"),
    ]

    response = await client.response(messages)
    print(response)

    # Streaming response
    client = llm(model="mistral:mistral-small")
    messages = [
        Message(role=Role.User, content="Tell me a joke."),
    ]
    response = await client.streaming_response(messages)
    async for chunk in response:
        print(chunk, end="", flush=True)
    print()

    # Embed text
    embedder = embedding("openai:text-embedding-ada-002")
    embeddings = await embedder.embed_text("Hello, world!")
    print(embeddings[:10])

    # Embed texts
    embedder = embedding("mistral:mistral-embed")
    embeddings = await embedder.embed_texts(["Hello, world!"])
    print(embeddings[0][:10])



if __name__ == "__main__":
    asyncio.run(main())

Common parameters

  • temperature
  • max_tokens
  • top_k
  • top_p
  • stop
  • seed
  • presence_penalty
  • frequency_penalty
  • response_format
  • tools
  • tool_choice

Command line interface

llm-taxi --model openai:gpt-3.5-turbo-16k

See all supported arguments

llm-taxi --help

Supported Providers

Provider LLM Embedding
Anthropic
DashScope
DeepInfra
DeepSeek
Google
Groq
Mistral
OpenAI
OpenRouter
Perplexity
Together
BigModel

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llm_taxi-0.7.0.tar.gz (1.1 MB view hashes)

Uploaded Source

Built Distribution

llm_taxi-0.7.0-py3-none-any.whl (24.9 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page