omgutils 0.0.4

Python utils for OMG processing

omgutils

Python utils for OMG processing

This omgutils repo is a WIP. One day it might have and do the following:

• *project and dependency management with uv

• *unit and integration tests with pytest

• *GitHub Actions workflows for tests (CI)

• *tagged releases on pypi

• *GitHub Actions workflow for publishing tags to pypi (CI)

• Machado lab code for TIFF and OMG processing (currently subpackages of stepsutils)

• nice-to-read Python docstrings

• do we need to publish separate readthedocs docs?

  • clicking through to real code is often more informative
  • published docs are often incomplete and/or stale
  • how far can we get with one good README?
  • publishing docs might be a good thing to do, anyway

• notebooks to demonstrate preprocessing, similar to omg-preprocess

• notebooks to demonstrate motion correction, similar to omg-motion-correction

• tests for the of the demonstrated functionality with pytest

  • can the tests just run the notebooks?

• installation, testing, getting started instructions

• links to small datasets / test fixtures

Here's an overview of the workflow that this repo might support.

editable diagram

Required and optional data inputs are in yellow boxes. Derived data products are in blue boxes. Diagnostic plots are in purple boxes.

The main processings steps (notebooks?) are in red boxes. Currently these mirror what we have in omg-preprocess and omg-motion-correction. Each of these contains many sub-steps, and the shape of these might want to evolve.

We should make our existing processing steps consume omgutils as a dependency so that we are not duplicating key code, and so that we are dogfooding.

We have oeg-preprocess code that also uses the AlignedTiff class in the stepsutils.tiff subpackage. OEG code could obtain the AlignedTiff dependency from omgutils. This should work find as a practical matter. But it would be slightly awkward to imply that OEG depends on OMG. To me this is the least bad option, though, as opposed to duplicating the common code, or creating ad managing a separate package like "tiffutils".

Dev bootstrapping notes

Here are some dev notes for reference!

uv and project setup

install uv locally

Init the project as a library

cd omgutils
uv init --lib

Reonfigure the build system in pyproject.toml because we'll want dynamic versioning on CI.

[build-system]
requires = ["hatchling", "uv-dynamic-versioning"]
build-backend = "hatchling.build"

Leave the src/omgutils/__init__.py there, but make it empty.

initial (placeholder) code to run and test

Add some hello world code so we can test something with pytest.

src/omgutils/hello/hello.py

import numpy as np


def hello(
    repetitions: int = 0
) -> list[str]:
    """Say "hello", repetitions times."""
    hellos = np.full((repetitions,), "hello", dtype=np.object_)
    return hellos

Add the numpy dependency

uv add 'numpy==2.4.6'

This declares the dependency in pyproject.toml. It also sets up a virtual environment for the project (.venv, git-ignored) and a uv.lock with explicit dependency versions.

Add a .gitignore

# Python-generated files
__pycache__/
*.py[oc]
build/
dist/
wheels/
*.egg-info

# Virtual environments
.venv

get the project version from git tags

Configure dynamic versioning

We want the project version numbers to correspond to Git tags, as described here.

Edit the pyproject.toml

[project]
name = "omgutils"
dynamic = ["version"]

...

[tool.hatch.version]
source = "uv-dynamic-versioning"

Create an initial tag.

git tag v0.0.1

Check the build results.

uv build
ls dist
# omgutils-0.0.1-py3-none-any.whl  omgutils-0.0.1.tar.gz

Make the version available at runtime, in src/omgutils/__init__.py

import importlib.metadata

try:
    __version__ = importlib.metadata.version(__name__)
except importlib.metadata.PackageNotFoundError:
    __version__ = "0.0.0"  # Fallback for development mode

Try printing the version at runtime.

uv run python -c "import omgutils; print(omgutils.__version__)"
# 0.0.1

testing with uv and pytest

Set up pytest and uv tooling, as described here

uv add --dev pytest

Create a few tests to run.

tests/omgutils/hello/test_hello.py

from omgutils.hello.hello import hello


def test_default_is_zero():
    hellos = hello()
    assert len(hellos) == 0

def test_ten():
    hellos = hello(10)
    assert len(hellos) == 10

Run the tests (uv run makes both pytest and omgutils available).

uv run pytest -v

Be able to run with coverage.

uv add --dev coverage

uv run coverage run -m pytest
uv run coverage report -m

run tests on CI

Set up CI testing and publishing to PyPI via GitHub Actions has described here.

Add .github/workflows/pytest.yml to set up uv and invoke pytest, similar to manual steps above. Run tests with python versions 3.11 - 3.14.

publish tags to PyPI on CI

PyPI, GitHub and uv are all friends, with just a bit of admin on GitHub and PyPI:

• https://docs.pypi.org/trusted-publishers/
• https://docs.astral.sh/uv/guides/integration/github/#publishing-to-pypi
• https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments

Created a pypi environment for this GitHub repo. No special environment configuration yet, but the environment exists with that name.

Add .github/workflows/pypi.yml to build the package, sanity check with tests/smoke_test.py, and publish to PyPi.

Create and push a repo tag to test out the publishing flow:

git tag -a v0.0.2 -m "Test of PyPI publishing."
git push --tags

The build succeeded.

This resulted in an omgutils package published to PyPI.