embedbase 0.9.0

The open source database for ChatGPT

Embedbase

The open source database for ChatGPT

Open-source sdk & api to easily connect data to ChatGPT

Used by AVA and serving 100k request a day

Try the sandbox playground now · Request Feature · Report Bug · Join our Discord

Check out the docs for more info.

Table of Contents

• Javascript SDK
• What are people building with Embedbase?
• Docs and support
• Contributing
• Open-source vs hosted

What are people building

• ChatGPT-powered search for markdown documentation
• AVA uses Embedbase to help their users find related notes

Embedbase Flavours

As a library

from embedbase import get_app

from embedbase.settings import Settings
from embedbase.supabase_db import Supabase

settings = Settings(
    # your config goes here, or use "get_settings" helper to use a config.yaml
)

app = (
    get_app(settings)
    # we use supabase.com as db here, but pick your favourite one
    .use(Supabase(settings.supabase_url, settings.supabase_key))
).run()

Docker

Deploy an instance in one line with Docker:

docker-compose up

Managed Instance

The fastest way to get started with Embedbase is signing up for free to Embedbase Cloud.

How to use

SDK

npm i embedbase-js

import { createClient } from 'embedbase-js'

const question = 'What can I do with Embedbase API?'

const embedbase = createClient(
  'https://api.embedbase.xyz',
  'api-key')

const context = await embedbase
.dataset('embedbase-docs')
.createContext('What can I do with Embedbase API?', { limit: 3 });

console.log(context) 
[
  "Embedbase API allows to store unstructured data...",
  "Embedbase API has 3 main functions a) provides a plug and play solution to store embeddings b) makes it easy to connect to get the right data into llms c)..",
  "Embedabase API is self-hostable...",
]

// refer to https://github.com/openai/openai-node for the exact api
openai.createCompletion(
  `Write a response to question: ${question} 
  based on the follwing context ${context.toString()}`
)
// answer:
// You can use the Embedbase API to store unstructured data and then use the data to connect it to LLMs

Inserting data

const URL = 'http://localhost:8000'
const VAULT_ID = 'people'
// if using the hosted version
const API_KEY = '<https://app.embedbase.xyz/signup>'
fetch(`${URL}/v1/${VAULT_ID}`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      // if using the hosted version, uncomment
      // 'Authorization': `Bearer ${API_KEY}`
    },
    body: JSON.stringify({
      documents: [{
        data: 'Elon is sipping a tea on Mars',
      }],
    }),
  });

Searching

fetch(`${URL}/v1/${VAULT_ID}/search`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      // 'Authorization': `Bearer ${API_KEY}`
    },
    body: JSON.stringify({
      query: 'Something about a red planet',
    }),
  });

Result:

{
  "query": "Something about a red planet",
  "similarities": [
    {
      "score": 0.828773,
      "id": "ABCU75FEBE",
      "data": "Elon is sipping a tea on Mars",
    }
  ]
}

Docs and support

Check out our tutorials for step-by-step guides, how-to's, and best practices, our documentation is powered by ChatGPT, so you can ask question directly.

Ask a question in our Discord community to get support.

Contributing

We recommend using Gitpod for development.

Make sure to add a variable in your Gitpod dashboard EMBEDBASE_CONFIGas a JSON value.

To create a json from yaml: yq -o=json eval config.yaml or cat config.yaml | yq depending on your yq version

Current Stack

• Embeddings
  • openai embeddings
  • cohere embeddings
  • Google PaLM embeddings
  • local (BERT, etc.)
• Vector database
  • supabase
  • pinecone
  • local (sqlite, etc.)
• fastapi
• Authentication (optional)
  • firebase
  • supabase

Configuration

Prerequisite

• Supabase account & database configured
• Openai account

minimal config.yaml (see config.example.yaml)

vector_database: supabase # or pinecone

supabase_url: <get me here https://supabase.com>
supabase_key: <get me here https://supabase.com>

# https://platform.openai.com/account/api-keys
openai_api_key: "sk-xxxxxxx"
# https://platform.openai.com/account/org-settings
openai_organization: "org-xxxxx"

Python

To run a uvicorn server that automatically reloads on code changes:

make run

Docker

docker-compose up

Open-source vs hosted

This repo is available under the MIT license.

To learn more, book a demo.