liteindex 0.0.2.dev64

SQLite based queryable python indexes for dicts and lists

LiteIndex

Embedded, thread and process safe, disk backed, easy to use, query-able, fast Index implementations

pip install --upgrade liteindex

DefinedIndex

• Documentation | Detailed example | Benchmarks
• fixed schema index i.e: schema has to be defined before hand
• number, boolean, datetime, string, compressed_string, blob, json, normalized_embedding, and other types are supported and can be queried upon
• can store any python objects with varying levels of query-ability
• Query language is subset of mongodb's. All queries are documented
• seamless, very fast nearest neighbor search with filtering. a practical wrapper of faiss + sqlite. Approximate nearest neighbor search is not supported. Intended for few million vector search
• works across threads, processes seamlessly
• handy features like search and update in single query, batch operation support for update, search, del, pop etc ..
• in-place math operations support wherever applicable i.e: += etc can be done in single query optimally
• compression is supported natively and optional custom compression dictionaries can be built automatically

KVIndex

• Documentation | Detailed example | Benchmarks
• simple key value store, can store any python objects, can be queried for equality, sorting, max, min etc wherever applicable
• in-place math operations support wherever applicable i.e: += etc can be done in single query optimally
• has a python dict like interface
• batch operation support for update, search, del, pop etc ..
• ultra fast, works across threads, processes seamlessly
• Eviction policies supported: LRU, LFU, any and age based invalidation and size, count based eviction

function_cache

• Documentation | Detailed example | Benchmarks
• based on KVIndex, can be used to cache function calls of any type
• batch inference caching friendly