dynamodb-haystack

• Integration page
• Changelog

Requirements

DynamoDB's native vector search (SearchVectors, vector indexes) is only available in boto3 >= 1.43.66, which requires Python 3.10+. Both are enforced by this package's metadata.

Limitations

• Metadata filters are evaluated client-side: filter_documents, count_documents and the filter-based bulk operations scan the whole table, and embedding retrieval filters the at most 100 nearest candidates that DynamoDB's SearchVectors returns (top_k cannot exceed 100).
• Only cosine similarity is supported for now.
• A DynamoDB item is limited to 400 KB, which bounds a document's content, metadata and embedding.

Running the integration tests

The integration tests run against real AWS — there is no local DynamoDB emulator that supports vector indexes. They are skipped unless you opt in:

export AWS_DEFAULT_REGION=<your-region>
export HAYSTACK_DYNAMODB_INTEGRATION_TESTS=1
hatch run test:integration

You also need credentials (any standard boto3 credential source) with permission to CreateTable, DeleteTable, DescribeTable, ListTables, the item-level operations, and SearchVectors. Each test class creates one table named haystack_test_<run id>_<class> and deletes it afterwards; a session-scoped safety net sweeps any table of the same run whose delete was rejected while its index was still settling. Expect the suite to create and destroy real, billable tables.

Contributing

Refer to the general Contribution Guidelines.