langchain-aneforge

LangChain embeddings that run on the Apple Neural Engine, via ANEForge. The encoder runs on the ANE with no CoreML, matching the fp32 reference to cosine ~1.0 at a fraction of the GPU's energy.

pip install langchain-aneforge

Apple Silicon macOS only (ANEForge reaches the engine through a built dispatch shim; see the ANEForge install notes).

Use

from langchain_aneforge import ANEEmbeddings

emb = ANEEmbeddings(model="sentence-transformers/all-MiniLM-L6-v2")
vec = emb.embed_query("the Apple Neural Engine runs networks at low power")
docs = emb.embed_documents(["one passage", "another passage"])

ANEEmbeddings is a standard langchain_core.embeddings.Embeddings, so it drops into any LangChain retriever or vector store:

from langchain_community.vectorstores import FAISS

store = FAISS.from_texts(["...", "..."], ANEEmbeddings())
hits = store.similarity_search("query", k=3)

model is any BERT-family sentence encoder on the Hugging Face hub. Its pooling mode and normalize flag are read from the model's own sentence-transformers config, so mean-pooled (MiniLM, E5) and cls-pooled (BGE, GTE) models both produce correct vectors. Pass int8=True to stream int8 weights (half the size, cosine ~0.9999).

License

MIT. Not affiliated with or endorsed by Apple or LangChain.