libmagic-wheel

A specific, pinned libmagic version bundled as a wheel - for our own applications, not a general python-magic replacement.

For most cases, python-magic is the better choice - see Why not python-magic? below for the one narrow problem this package exists to solve instead.

.mgc databases are version-locked in their binary format - libmagic refuses to load a .mgc compiled by a newer release than itself. A distro that pins its libmagic package to an old version therefore can't load a database built with a current upstream release, no matter how it's loaded. This package builds a specific upstream release from source and ships it as a self-contained Python package - the library, the upstream Magdir definitions, and the tooling to compile a .mgc from them - so applications can use a current libmagic without depending on whatever version the host distro happens to provide.

Usage

A bundled, unmodified-Magdir .mgc ships with the package, so this works right away, no build step needed - Magic() with no arguments loads it:

from libmagic import Magic

m = Magic()

r = m.from_buffer(sample_bytes)
r.description, r.mime_type, r.extensions

Pass your own compiled database via the keyword-only magic_file:

m = Magic(magic_file="/etc/libmagic-wheel/combined.mgc")

uncompress=True alone never decompresses anything - see Magic.__init__'s docstring for why and when allow_compress_fork=True is safe to add.

CLI

libmagic-wheel ships three subcommands; --help on any of them lists the full set of options.

compile

Compiles a .mgc from the bundled upstream Magdir plus your own overrides/additions - a file matches a bundled fragment by name and replaces it, any other name is added as a new fragment:

libmagic-wheel compile --override-dir ./our-definitions --output-dir /etc/libmagic-wheel

Add --rpm to also wrap the result in a minimal RPM in the same step (the RPM's version defaults to the current UTC timestamp). By default, both the RPM package name and the path it installs to derive from --name (/etc/libmagic-wheel/<name>.mgc) - override either independently with --rpm-name or --install-path, e.g. to match an existing naming scheme without changing where the file ends up, or vice versa.

The same thing from Python:

from libmagic import compile_database

mgc = compile_database(output_dir="/etc/libmagic-wheel")

classify

file(1)-like: description and mime type together, one line per file.

libmagic-wheel classify --mgc /etc/libmagic-wheel/combined.mgc some-file.pdf

Falls back to the bundled default .mgc if --mgc is omitted (development/debugging only). --uncompress looks inside compressed files - see Magic.__init__'s docstring for why --allow-compress-fork is also needed before that actually decompresses anything.

diff

Side-by-side comparison of override files against the bundled Magdir fragments they'd replace, so you can review what a compile would actually change first:

libmagic-wheel diff --override-dir ./our-definitions

Uses the system diff binary (diff -y) - no extra dependency just for this.

Django / Celery

Magic.__init__ does no I/O, so a plain module-level instance is enough

• no lazy caching needed. settings.py is a natural place for it (Django only exposes uppercase names via django.conf.settings):

# settings.py
from libmagic import Magic

MAGIC = Magic(magic_file="/etc/libmagic-wheel/combined.mgc")

# a view, a utility function, a Celery task
from dataclasses import asdict
from django.conf import settings


def classify(data: bytes) -> dict:
    return asdict(settings.MAGIC.from_buffer(data))

If Celery workers load the same Django settings, tasks reuse this exact object - no separate initialization needed. asdict() is only needed for JSON-serializable task return values; a plain function can return the Classification object directly.

Why not python-magic?

python-magic remains the right general choice for classification - this package is narrower, only for the version-lock problem above. Its load_lib() resolves the library via ctypes.util.find_library() at import time, which depends on the OS search mechanism this package is built to avoid - import magic fails outright on a host with no system libmagic, before you'd even get a chance to redirect it. It also shares one cookie across all threads behind a global lock rather than one per thread, so concurrent use is serialized instead of genuinely parallel.

License

libmagic/file itself is BSD-2-Clause, Copyright Ian F. Darwin and contributors - see LICENSE.file-upstream (generated by scripts/build-lib.sh, gitignored like the other build output).