Skip to main content

Add your description here

Project description

embedmr

embedmr is a cache-aware MapReduce embedding framework.

It helps users:

  • Convert text into deterministic chunks
  • Generate embeddings efficiently
  • Avoid duplicate embedding computation using cache
  • Scale embedding generation using MapReduce architecture

Install

pip install embedmr

Usage

embedmr make-chunks --input docs/ --output chunks.jsonl
embedmr validate chunks.jsonl

Purpose

embedmr ensures deterministic, reproducible, and scalable embedding pipelines.


Then publish update:

```powershell
# bump version
# in pyproject.toml change:
version = "0.1.1"

Remove-Item -Recurse -Force build, dist -ErrorAction SilentlyContinue
uv run python -m build
uv run python -m twine upload dist/*

Step 8: Verify install works

Test in new terminal:

pip install embedmr
embedmr --help

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

embedmr-0.1.1.tar.gz (10.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

embedmr-0.1.1-py3-none-any.whl (14.6 kB view details)

Uploaded Python 3

File details

Details for the file embedmr-0.1.1.tar.gz.

File metadata

  • Download URL: embedmr-0.1.1.tar.gz
  • Upload date:
  • Size: 10.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.5

File hashes

Hashes for embedmr-0.1.1.tar.gz
Algorithm Hash digest
SHA256 dc066e113e344b2a408a1382c34a4b5ebb6502718620307c6582c099fec487a5
MD5 4d26f3aa9041ef932a203b3babf87d96
BLAKE2b-256 36223e3790f174dad49592d980b7a4cf17f07d251e991220432a72e03d195f2f

See more details on using hashes here.

File details

Details for the file embedmr-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: embedmr-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 14.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.5

File hashes

Hashes for embedmr-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 3ddbd2dbac807bc1266346c97a9233bca083990e0350590faf91efc74f977069
MD5 8c23f054a631175371dfa4897128cf9f
BLAKE2b-256 a09d005e6548ccdc3e1e20a20d009735cd61a408ef07bbd115ef00bccbd96b9d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page