goodai-ltm 0.2.2

A text memory meant to be used with conversational language models.

GoodAI-LTM

GoodAI-LTM brings together all the components necessary for equipping agents with text-based long term memory. This includes text embedding models, reranking, vector databases, chunking, metadata such as time stamps and document information, memory and query rewriting (expansion and disambiguation), storage and retrieval.

The package is especially adapted to provide a dialog-centric memory stream for social agents.

• Embedding models: Use OpenAI, Hugging Face Sentence Transformers, or our own locally trainable embeddings. The trainable embeddings allow multiple embeddings for a query or passage, which can capture different aspects of the text for more accurate retrieval.

• Query-passage match ranking: In addition to similarity-based retrieval, we support models for estimating query-passage matching after retrieval.

• Vector databases: We currently provide a light-weight local vector database as well as support for FAISS.

Installation

pip install goodai-ltm

Short example

The following code snippet creates an instance of LTM, loads in some text and then retrieves the most relevant text chunks given a query:

from goodai.ltm.mem.auto import AutoTextMemory
mem = AutoTextMemory.create()
mem.add_text("Lorem ipsum dolor sit amet, consectetur adipiscing elit\n")
mem.add_text("Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore\n",
             metadata={'timestamp': time.time(), 'type': 'generic'})
r_memories = mem.retrieve(query='dolorem eum fugiat quo voluptas nulla pariatur?', k=3)

Additional information

Visit the Github page: https://github.com/GoodAI/goodai-ltm