api4all 0.2.6

Easy-to-use LLM API from a state-of-the-art provider and comparison

api4all

Easy-to-use LLM API from a state-of-the-art provider and comparison.

Installation

1. Clone the repository

git clone https://github.com/api4all/api4all.git

2. Create and activate a virtual environment

• Unix / macOS

python3 -m venv venv
source venv/bin/activate

• Windows

python3 -m venv venv
.\venv\Scripts\activate

3. Install the requirements

pip install -r requirements.txt

Providers and Pricing

Providers

Provider	Free Credit	Rate Limit	API Key name	Provider string name
Groq	Unlimited	30 Requests / Minute	GROQ_API_KEY	"groq"
Anyscale	$10	30 Requests / Second	ANYSCALE_API_KEY	"anyscale"
Together AI	$25	1 Requests / Second	TOGETHER_API_KEY	"together"
Replicate	Free to try	50 Requests / Second	REPLICATE_API_KEY	"replicate"
Fireworks	$1	600 Requests / Minute	FIREWORKS_API_KEY	"fireworks"
Deepinfra	Free to try	200 Concurrent request	DEEPINFRA_API_KEY	"deepinfra"
Google AI (Vertex AI)	Unlimited	60 Requests / Minute	GOOGLE_API_KEY	"google"
OpenAI	✕	60 Requests / Minute	OPENAI_API_KEY	"openai"
Mistral AI	✕	5 Requests / Second	MISTRAL_API_KEY	"mistral"
Anthropic	✕	5 Requests / Minute	ANTHROPIC_API_KEY	"anthropic"

• Free to try: Free to try, no credit card required but limited to a certain number of tokens.

Open-source models

1. Mixtral-8x7b-Instruct-v0.1

Mixtral 8x7B is a high-performing sparse mixture of experts model (SMoE) with open weights, licensed under Apache 2.0. It surpasses Llama 2 70B in most benchmarks, offering 6x faster inference. It's the best open-weight model considering cost/performance trade-offs, matching or exceeding GPT3.5 on most standard benchmarks.

• Developer: Mistral AI
• Context Length: 32,768

Provider	Input Pricing ($/1M Tokens)	Output Pricing ($/1M Tokens)
Groq	$0	$0
Anyscale	$0.5	$0.5
Together AI	$0.6	$0.6
Replicate	$0.3	$1
Fireworks	$0.5	$0.5
Deepinfra	$0.27	$0.27

2. Gemma 7B it

• Developer: Google AI
• Context Length: 8,192

Provider	Input Pricing ($/1M Tokens)	Output Pricing ($/1M Tokens)
Groq	$0	$0
Anyscale	$0.15	$0.15
Together AI	$0.2	$0.2
Replicate	✕	✕
Fireworks	$0.2	$0.2
Deepinfra	✕	✕

3. LLaMA2-70b

• Developer: Meta AI
• Context Length: 4,096

Provider	Input Pricing ($/1M Tokens)	Output Pricing ($/1M Tokens)
Groq	$0	$0
Anyscale	$1.0	$1.0
Together AI	$0.9	$0.9
Replicate	$0.65	$2.75
Fireworks	$0.9	$0.9
Deepinfra	$0.7	$0.9

4. Mistral-7B-Instruct-v0.1

• Developer: Mistral AI
• Context Length: 4,096

Provider	Input Pricing ($/1M Tokens)	Output Pricing ($/1M Tokens)
Groq	✕	✕
Anyscale	$0.15	$0.15
Together AI	$0.2	$0.2
Replicate	$0.05	$0.25
Fireworks	$0.2	$0.2
Deepinfra	✕	✕