hel-elmuro 0.1.0

Helpers for Cosmos DB and Azure Storage operations

hel-elmuro

Python helpers for the ElMuro transportation analytics platform.

Features

• Persist pandas DataFrames into Azure Cosmos DB containers.
• Perform common Azure Blob Storage operations (count, copy, delete, move).

Installation

pip install hel-elmuro

Quick Start

from hel_elmuro import CosmosManager, FileManager

cosmos = CosmosManager(
    endpoint="https://<account>.documents.azure.com:443/",
    key="<cosmos-key>",
    database_name="elmuro",
    container_name="drivers",
)
inserted = cosmos.save_dataframe(df, partition_key_column="driver_id")

files = FileManager(
    connection_string="<storage-connection-string>",
    container_name="dap0gescab0con",
)
files.move("bronze/file.json", "silver/file.json")

File Organization

• Is not defined by the workspace convention.
• The organization is defined by the build process defined in the /pyproject.toml file; by default the build artifacts source code is collected from src/hel_elmuro.
• All library modules should live under src/hel_elmuro so editable installs and wheels include the package correctly.

Development

• All examples assume Visual Studio Code and a WSL terminal environment.
• Install the package in editable mode for iterative work: pip3 install -e "hel_elmuro_pkg[dev]".
• Development dependencies (pytest, black, flake8) are defined in pyproject.toml under the dev extra.

Testing

• Based on the pytest python module. defined in dev depedencies in the .toml file of the proyect. Test is in the tests folder and it will execute every function declared in that files. Organization of the files is something reletaed with the workspace conventions to organize per file the different unit testing of specific objects. (Modules/class in this cases).

Run the suite from the repository root so editable installs resolve correctly:

python3 -m pytest

When you need to run tests from the package directory with explicit logging parameters:

cd hel_elmuro_pkg
python3 -m pytest --log-cli-level=INFO -s

Release (PyPI)

1. Bump the project.version value in hel_elmuro_pkg/pyproject.toml to the desired semantic version (MAJOR.MINOR.PATCH), and update any information there.
2. Ensure the workspace has build and twine installed (pip3 install --upgrade build twine).
3. From the repository root execute hel_elmuro_pkg/scripts/publish_release.py (tests are executed again as part of this step):

   python3 hel_elmuro_pkg/scripts/publish_release.py
   

   • The script reads the package name and version from pyproject.toml.
   • It verifies the working tree is clean, checks the declared version is newer than the latest git tag and PyPI release, and runs python -m pytest before proceeding.
   • On passing tests it builds the distributions, uploads them with twine upload dist/*, and creates/pushes the git tag v<version>.