imgread

A public beta Python library for decoding JPEG, PNG and TIFF into NumPy arrays, with a Rust decoder and a statically linked libjpeg-turbo backend.

python -m pip install --pre imgread

Use a wheel on a supported platform; it needs no Rust toolchain. To require a wheel: python -m pip install --pre --only-binary=:all: imgread. A source build requires Rust 1.88 or newer, CMake, a C compiler and NASM.

from pathlib import Path
import imgread

# Path and os.PathLike[str]
rgb = imgread.load_numpy(Path("photo.jpg"))

# bytes, bytearray, memoryview or a uint8 NumPy buffer (including strided views)
bgr = imgread.load_numpy_from_bytes(Path("photo.png").read_bytes(), color="bgr")

# Choose the portable Rust decoder explicitly
rgb = imgread.load_numpy("scan.tiff", backend="image")

Array and format contract

Every successful call returns a new writable, C-contiguous numpy.ndarray of shape (height, width, 3) and dtype uint8. The default order is RGB; color="bgr" reverses the channels. Only dtype="uint8" is supported. Color, dtype, backend and limit-profile names are case-insensitive. Bytes paths are rejected; use the buffer APIs for encoded bytes. Decoding releases the GIL. The standalone buffer functions and Loader.decode borrow immutable bytes for the duration of the call and copy other buffer types, including read-only views, before releasing the GIL.

• JPEG: baseline, progressive, grayscale and CMYK. CMYK may use the fallback decoder. Decoder rounding and chroma upsampling can produce small pixel differences between backends; bit-for-bit JPEG parity is not promised.
• PNG: grayscale, RGB, RGBA and 16-bit input. Alpha is discarded without compositing. Grayscale is replicated into three channels. Unsigned 16-bit values are rounded to 8 bits using (value + 128) // 257.
• TIFF: the first page only, without applying orientation. Common uncompressed, LZW and Deflate variants and little-/big-endian samples are supported. Supported encodings follow the image TIFF decoder; unsupported encodings raise errors.
• EXIF orientation, ICC color conversion and other metadata are not applied or returned. Pixels remain in the encoded color space. Animated PNG returns its default image, not an animation.

Other formats are rejected even though the underlying Rust image dependency retains its default features in this beta.

Persistent Loader

Requires imgread 0.2.1 or newer. See the changelog for release details.

loader = imgread.Loader(color="rgb", backend="auto")
rgb = loader("photo.jpg")
data = Path("photo.jpg").read_bytes()  # or an encoded image from another source
rgb = loader.decode(data)

indexed = imgread.Loader(["first.jpg", "second.jpg"])
rgb = indexed[0]

decode(data) accepts the same uint8-compatible buffers as load_numpy_from_bytes, including strided views, and uses the Loader's fixed options and resource limits. It reuses the same native JPEG state as path/index calls and returns an independent array. Input buffers are never retained. Mutable buffers and memoryviews, including read-only views, are copied in C order; immutable bytes require no input copy. max_buffer_bytes caps retained file-read storage only, not the accepted image size.

A Loader can be pickled and passed to DataLoader workers: each process initializes its own decoder lazily. Keep one Loader per worker; overlapping or reentrant calls on the same instance raise RuntimeError. Buffer data belongs to the caller and is not included in the Loader's pickle. Preloading an entire dataset is optional and has its own memory and worker-startup costs.

Preparing a JPEG cache with jpegli

Install the optional encoder with python -m pip install 'imgread[jpegli]'. encode_jpeg uses ajpegli, a binding to Google's jpegli. The training/reading environment needs only imgread: the cache contains ordinary JPEG bytes, decoded by the existing TurboJPEG backend. The optional encoder has its own wheel requirements: ajpegli 1.0.0 provides CPython 3.11–3.13 wheels for imgread's platforms, with glibc 2.27+ required on Linux. Python 3.14 or older Linux systems may require an ajpegli source build.

import imgread

# Prepare once, outside the training loop. Input is HWC uint8 RGB.
encoded = imgread.encode_jpeg(rgb, quality=90, subsampling="420")

# Keep immutable bytes in RAM, or save them in your existing cache storage.
loader = imgread.Loader()  # one per worker
restored = loader.decode(encoded)

encode_jpeg(image, *, quality=90, subsampling="420", progressive=False, color="rgb") accepts NumPy arrays of shape (height, width, 3) and dtype uint8. Both dimensions must be in 1..65500. Strided arrays are copied to contiguous storage; color="bgr" also swaps channels. The function returns immutable bytes. It rejects other dtypes, grayscale and alpha rather than implicitly changing the pixels. Do not mutate the source array until encoding returns.

Quality is an integer in 1..100; even 100 is lossy. Subsampling can be "420", "422", "440" or "444". Sequential encoding is the default for cache reads; progressive=True selects jpegli level 2, or use an integer level from 0 to 3. Progressive encoding can save space but adds decoding work; compare size, quality and read speed on your data. Equal quality numbers across encoders do not mean equal reconstructed quality. Adaptive quantization and optimized Huffman tables are enabled; no metadata or ICC transform is applied. Encoded bytes may change with the ajpegli/jpegli version, so pin the encoder when preparing reproducible caches.

Missing ajpegli raises ImportError with installation guidance only when encoding is requested; ordinary imports and decoding do not require it. Invalid input raises TypeError/ValueError, and native encoding failures raise RuntimeError.

Backends and the simple API

backend="auto" selects TurboJPEG for JPEG when compiled in and image otherwise. backend="image" always selects the Rust decoder. backend="turbojpeg" falls back to image when unavailable, incompatible with the format, or unable to decode it. A successful fallback emits RuntimeWarning. An unsuccessful decode raises an error. auto also warns if an attempted TurboJPEG decode falls back. Inspect the build with imgread.supported_backends(); official wheels include TurboJPEG.

load_numpy_simple(path, *, limits="safe") and load_numpy_simple_from_bytes(data, *, limits="safe") are JPEG-only RGB shortcuts. They use the same checked decoder and warning/fallback behavior. Non-JPEG input is always a ValueError. Python's normal warnings filters control every warning, including always, ignore and error; the library has no global deduplication.

Resource limits

All four decode functions accept the keyword-only argument limits="safe":

Input size is checked before copying a Python buffer or reading a regular file. The open file is read with a bound even if it grows. Dimensions and output sizes are checked before the pixel allocation. A resource-limit failure is terminal: no fallback can retry it with another decoder.

The decoder budget is passed to image::Limits and to libjpeg-turbo's intermediate buffer memory limit. The image JPEG backend also preflights a conservative working-set bound covering padded coefficient planes, decoded pixels and row/upsampling buffers, because its upstream JPEG decoder ignores max_alloc. This applies to fallback routes too, and may reject a large JPEG even when its final pixels fit the output cap (including baseline JPEGs that need less memory). These budgets do not measure total process RSS: encoded input, output, metadata, conversion buffers, allocator overhead and concurrent calls can add to it. It is a per-image policy, not a process sandbox.

For trusted large inputs only, use limits="unlimited". This removes policy caps; checked arithmetic, address-space checks and fallible application allocations remain enabled. It makes no speed promise. Ordinary allocation pressure and upstream codec behavior can still cause errors.

Errors

Format detection uses content; for path input an extension is a last resort for identifying a damaged image. Unknown bytes raise ValueError; a recognized but corrupt image raises RuntimeError.

Beta support and stability

The release wheel matrix targets CPython 3.11–3.14, Linux x86_64 manylinux2014 (glibc 2.17+) and macOS x86_64 (10.13+) / arm64 (11.0+). Windows, Linux aarch64, musllinux, PyPy, free-threaded CPython and Python 3.15 are not supported by this beta. The 3.15 compatibility CI job is informational.

Loader and the functions above form the beta API. Bug fixes can change rejection of malformed inputs, limits or decoder results. Intentional API changes will be recorded in release notes and beta versions; pin a version for reproducible work. The Rust crate is internal and is not published to crates.io.

Development

Use uv run for Python-facing commands. Normal builds and tests use the committed Cargo.lock and uv.lock without updating them.

uv sync --locked --no-install-project
uv run maturin develop --locked
uv run pytest python_tests -q
cargo test --locked
cargo test --locked --features turbojpeg
cargo fmt -- --check
cargo clippy --locked --all-targets --all-features -- -D warnings
uv run maturin build --release --locked --sdist
uv run twine check target/wheels/*

Keep maturin's configured features when building; passing --features extension-module alone overrides them and removes TurboJPEG.

Publishing

Releases are built and published by GitHub Actions. Run Build distributions (beta-build.yml) on a release branch to validate the full package matrix without publishing.

1. Update Cargo.toml, its root Cargo.lock entry and CHANGELOG.md, then merge the reviewed changes into main.
2. Permit the release tag in the GitHub pypi environment deployment policy. For this release, add the exact tag v0.3.0. Keep its required reviewer enabled.
3. Create and push an annotated tag matching the package version, for example git tag -a v0.3.0 -m 'Release imgread 0.3.0' followed by git push origin v0.3.0. These commands assume that tag does not exist yet.
4. Publish release checks the tag/version and runs CI. It builds and tests 12 wheels (CPython 3.11–3.14; Linux x86_64 and macOS x86_64/arm64), rebuilds the sdist, validates package metadata and saves SHA256SUMS.
5. Review the successful build and approve the pypi deployment in Actions. The job publishes the same artifacts using PyPI Trusted Publishing.
6. The workflow installs the published wheels on all 12 combinations, verifies PyPI file hashes, and publishes GitHub release notes with those artifacts.

Configure the PyPI project's GitHub Trusted Publisher with owner ayasyrev, repository imgread, workflow release.yml, and environment pypi. GitHub also requires the Actions variable PUBLIC_RELEASE_REPOSITORY=ayasyrev/imgread. See PyPI's setup guide.

uv build produces a local verification build. On Linux it may generate a linux_x86_64 wheel, which PyPI rejects. The workflow builds portable manylinux2014 wheels in the matching container. Do not rename a local wheel or publish the local dist/ directory as a release.

For interrupted releases, retain the original tag and Actions artifact set. Inspect any existing PyPI files before retrying an upload; a rebuild is not a replacement for already published files. Rerun only failed jobs when possible. If only the GitHub release job failed, rerun that job: it can finish an existing draft after verifying its input against PyPI, and refuses to overwrite a public GitHub release.

License

The project uses the MIT license. Redistributed dependency notices are in THIRD_PARTY_NOTICES.md and licenses/ and are included in both wheels and sdists.

This software is based in part on the work of the Independent JPEG Group.

Loader: paths and saved indices

Loader(
    paths=None, *, color="rgb", dtype="uint8", backend="auto",
    limits="safe", max_buffer_bytes=1048576,
)

The signature above describes the keyword-only options. For a working call:

loader = imgread.Loader(color="bgr")
array = loader("photo.jpg")

indexed = imgread.Loader(["first.jpg", "second.png", "first.jpg"])
first = indexed[0]
last = indexed[-1]
other = indexed("outside-the-snapshot.tiff")

paths is a finite iterable of str or os.PathLike[str]; a single path or bytes container is rejected. Construction converts each element once and preserves its text, order and duplicates in an immutable native snapshot. It opens no images, checks no image metadata and creates no native decoder. Changing the original list or PathLike objects does not change this snapshot. Files remain mutable: each call opens the current file, and relative paths use the working directory at load time.

len(indexed) counts snapshot entries. An empty snapshot is valid. Without a snapshot, length and indexing raise TypeError; path calls still work. bool(loader) is always true. Indexing accepts the integer __index__ protocol, including NumPy integers and negative indices. Out-of-range integers, even huge ones, raise IndexError. Python bool, slices and collections of indices raise TypeError. Settings cannot be changed after construction.

Loader returns the same pixels, exceptions and fallback warnings as load_numpy with the same build, backend and settings. Each array owns separate writable memory and remains valid after later calls, failures or Loader deletion. No output cache, batch API or internal prefetch is added.

max_buffer_bytes limits the capacity of compressed input retained after a call; its default is 1 MiB, and zero disables input retention. It accepts non-negative addressable integers through __index__. Larger allowed files use temporary input storage. This cap is separate from limits, native decoder allocations, the path snapshot and returned arrays; it is not a total RSS limit. Native JPEG state may still be reused with a zero input cap. A conservative JPEG marker check prevents a previous image's tables from affecting a later decode. Uncertain files use fresh native state and the usual fallback rules.

One Loader allows one active image call. Overlap and reentry raise RuntimeError("Loader is busy") immediately, including reentry from path/index protocols and warning handlers. Different instances operate independently. I/O and decoding release the GIL. Pickle stores only configuration and snapshot strings; restored objects start with no input/native state. Spawn, forkserver and an idle fork create process-owned decoding state. Fork while a Loader call is active is unsupported. Long-lived workers retain at most one workspace per Loader; process startup, manifest copies and pickle costs remain part of application preparation.

For PyTorch, ImageFolder(..., loader=Loader()) calls Loader with a path. Use a transform that accepts NumPy arrays and restrict the dataset to supported image formats. An indexed Dataset can snapshot paths in a Loader and call loader[index], keeping labels alongside the paths. Torch and torchvision are optional integrations, never imgread runtime dependencies.