embedbase 1.0.1

API, SDK & dashboard to easily create, store, and retrieve machine learning embeddings

Embedbase

The end to end platform to help you ship embeddings-powered apps.

Open-source API, SDK & dashboard to easily create, store, and retrieve machine learning embeddings

Used by AVA and serving 100k request a day

Try Embedbase Cloud now · Request Feature · Report Bug

Check out the docs for more info.

Table of Contents

• Getting started
• Javascript SDK
• Docs and support
• Contributing

The 3 ways to use Embedbase

• Embedbase Cloud: Build embeddings-powered apps in minutes | npm i embedbase-js
• Embedbase.py: Choose your own db, embeddings models, and get started with a simple | pip install embedbase
• Embedbase self-hosted: Get Embedbase Cloud on your infra | docker-compose up

Examples

Please refer to examples in the documentation.

What are people building

• Creating a recommendation engine: AVA uses Embedbase to help their users find related notes
• Creating chat with your data experiences: Solpilot uses Embedbase to put smart contract integration on autopilot
• Creating a talk with your docs experience: ChatGPT-powered search for markdown documentation

Getting started

Let's install Python dependencies:

pip install embedbase uvicorn

And write a basic boilerplate code in main.py:

import uvicorn
from embedbase import get_app

from embedbase.database.postgres_db import Postgres
from embedbase.embedding.openai import OpenAI
 
async def custom_middleware(request, call_next):
    # customise as you prefer :)
    start_time = time.time()
    response = await call_next(request)
    process_time = time.time() - start_time
    response.headers["X-Process-Time"] = str(process_time)
    
    return response
 
app = (
    get_app()
    .use_middleware(custom_middleware)
    .use_embedder(OpenAI("<your key>"))
    .use_db(Postgres())
).run()

if __name__ == "__main__":
    uvicorn.run("main:app")

Let's use Postgres as a database, you can start it with Docker:

docker run -d -p 8080:8080 -p 5432:5432 -e POSTGRES_DB=embedbase -e POSTGRES_PASSWORD=localdb ankane/pgvector

Now start embedbase using the boilerplate code we just wrote in main.py:

python3 main.py

🔥 Embedbase now runs! Time to ship your product

Managed Instance

The fastest way to get started with Embedbase is signing up for free to Embedbase Cloud.

How to use

SDK

npm i embedbase-js

import { createClient } from 'embedbase-js'

const question = 'What can I do with Embedbase API?'

const embedbase = createClient(
  'https://api.embedbase.xyz',
  'api-key')

const context = await embedbase
.dataset('embedbase-docs')
.createContext('What can I do with Embedbase API?', { limit: 3 });

console.log(context) 
[
  "Embedbase API allows to store unstructured data...",
  "Embedbase API has 3 main functions a) provides a plug and play solution to store embeddings b) makes it easy to connect to get the right data into llms c)..",
  "Embedabase API is self-hostable...",
]

// refer to https://github.com/openai/openai-node for the exact api
openai.createCompletion(
  `Write a response to question: ${question} 
  based on the follwing context ${context.toString()}`
)
// answer:
// You can use the Embedbase API to store unstructured data and then use the data to connect it to LLMs

Inserting data

const URL = 'http://localhost:8000'
const VAULT_ID = 'people'
// if using the hosted version
const API_KEY = '<https://app.embedbase.xyz/signup>'
fetch(`${URL}/v1/${VAULT_ID}`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      // if using the hosted version, uncomment
      // 'Authorization': `Bearer ${API_KEY}`
    },
    body: JSON.stringify({
      documents: [{
        data: 'Elon is sipping a tea on Mars',
      }],
    }),
  });

Searching

fetch(`${URL}/v1/${VAULT_ID}/search`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      // 'Authorization': `Bearer ${API_KEY}`
    },
    body: JSON.stringify({
      query: 'Something about a red planet',
    }),
  });

Result:

{
  "query": "Something about a red planet",
  "similarities": [
    {
      "score": 0.828773,
      "id": "ABCU75FEBE",
      "data": "Elon is sipping a tea on Mars",
    }
  ]
}

Docs and support

Check out our tutorials for step-by-step guides, how-to's, and best practices, our documentation is powered by GPT-4, so you can ask question directly.

Ask a question in our Discord community to get support.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.