Skip to main content

Determine how to group together input files into batches for subsequent concatenation

Project description

Batchee logo

Project Status: Active – The project has reached a stable, usable state and is being actively developed Mypy checked Python Versions Package version Code coverage

Overview

Batchee is a Python package that intelligently groups filenames together, enabling efficient batch operations like concatenation.

What does it do?

Batchee analyzes filename patterns and groups related files together. For example (note that these are pseudo-real, not actual, TEMPO file names):

batchee TEMPO_NO2_L2_S006G01.nc TEMPO_NO2_L2_S006G02.nc TEMPO_NO2_L2_S007G08.nc TEMPO_NO2_L2_S007G09.nc

Output:

  • TEMPO_NO2_L2_S006G01.nc, TEMPO_NO2_L2_S006G02.nc → Group 1 (scan 6)
  • TEMPO_NO2_L2_S007G08.nc, TEMPO_NO2_L2_S007G09.nc → Group 2 (scan 7)

This enables batch processing operations on each group separately.

Key Features

  • Automatic filename grouping based on configurable patterns
  • Command-line interface and Python API for integration with NASA Harmony service orchestrator
  • Verbose logging for debugging

Installation

From PyPI (Recommended)

pip install batchee

From Source (Development)

For local development or the latest features:

git clone <Repository URL>
cd batchee

(Option A) using poetry (Recommended for development):

# Install poetry: https://python-poetry.org/docs/
poetry install

(Option B) using pip:

pip install .

Usage

Basic Usage

batchee [file_names ...]

With Poetry (if installed via poetry)

poetry run batchee [file_names ...]

Options

  • -h, --help - Show help message and exit
  • -v, --verbose - Enable verbose output to stdout; useful for debugging

Contributing

Issues and pull requests welcome on GitHub.

License & Attribution

Batchee is released under the Apache License 2.0.

This package is NASA Software Release Authorization (SRA) # LAR-20440-1

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

batchee-1.5.0.tar.gz (11.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

batchee-1.5.0-py3-none-any.whl (15.4 kB view details)

Uploaded Python 3

File details

Details for the file batchee-1.5.0.tar.gz.

File metadata

  • Download URL: batchee-1.5.0.tar.gz
  • Upload date:
  • Size: 11.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/2.1.3 CPython/3.12.3 Linux/6.11.0-1018-azure

File hashes

Hashes for batchee-1.5.0.tar.gz
Algorithm Hash digest
SHA256 5b6f4b7c1f540e1c37d1e68ec43dd73ebdb68d48061abdc47fec8e991d1e2863
MD5 28edcc60ae1ac448061d9dac6e681047
BLAKE2b-256 35bf55bcbdeb216fc6930e4dffe37d7c1b08541bd4a4560a313811ecc2a98c6a

See more details on using hashes here.

File details

Details for the file batchee-1.5.0-py3-none-any.whl.

File metadata

  • Download URL: batchee-1.5.0-py3-none-any.whl
  • Upload date:
  • Size: 15.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/2.1.3 CPython/3.12.3 Linux/6.11.0-1018-azure

File hashes

Hashes for batchee-1.5.0-py3-none-any.whl
Algorithm Hash digest
SHA256 3fd83632d30390345931b20f9aaae5240997e312729c8b3903834d6451e7b26e
MD5 4189a1f4ab46fa3f0edfc746a1708082
BLAKE2b-256 68d6c83c2c46d5cf2cf043935b7f7d953be25639625a59080a97c4b31f6f26bf

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page