simplemagic

Simple file magic. We try to get file's mimetype using 'file-magic', 'command file' and 'puremagic'. On linux we need system package 'file-libs' which mostly already installed. On MacOS we need system package 'libimage' which can be installed by 'brew install libmagic'. On windows we need file command which can be install by 'pacman -S file' within msys2. If system package missing, we try to get the file's mimetype using 'puremagic' which is write in pure python without any extra depends.

Install

pip3 install simplemagic

System requirements

Linux

• file-libs

Mostly it is installed already, and you can installed it with command:

yum install file-libs

MacOS

• libmagic

You can installed it with command:

brew install libmagic

Windows

libmagic mostly not working on windows. Suggest you install msys2 on in system, and in msys2 you can install libmagic with command:

pacman -S file

Add msys2's bin path to your system's PATH env. We can call the external command file to get the mimetype of a file.

API Overview

simplemagic tries three engines to detect a file's mimetype from its content, in order:

1. libmagic (via the magic python binding, needs the system libmagic),
2. the external command file (from the file package),
3. puremagic (a pure-python implementation, needs no system package).

Once a mimetype is found by an engine, the following engines are skipped.

Public APIs:

API	Description
simplemagic.get_mimetype_by_filename(filename, ...)	Detect a file's mimetype by its path.
simplemagic.get_mimetype_by_stream(stream, filename, ...)	Detect a file's mimetype from an opened stream. filename is still required, it is only used for the puremagic engine hint and for extension compare.
simplemagic.guess_all_extensions(mimetype)	List all candidate file extensions of a mimetype (built-in + simplemagic's own mapping).
simplemagic.file_content_matches_with_file_extension_test(filename, stream=None, ...)	Detect by content, then test whether the filename's extension is a candidate extension of the detected mimetype. Returns (matched, ext, mimetype).
simplemagic.is_file_content_matches_with_file_extension(...)	The same test, but only returns the bool result.
simplemagic.is_file_content_matches_with_file_suffix(...)	Alias of the above.
simplemagic.register_mimetype_extensions({mimetype: ext or [exts]})	Add your own mimetype / extension mappings.
simplemagic.magic.disable_using_magic()/enable_using_magic()	Turn the libmagic engine on/off globally.
simplemagic.magic.disable_using_file_command()/enable_using_file_command()	Turn the file command engine on/off globally.
simplemagic.magic.disable_using_puremagic()/enable_using_puremagic()	Turn the puremagic engine on/off globally.
simplemagic.magic.set_file_command(cmd)	Change the file command path/name.

There are also extension category constants exported as simplemagic.IMAGE_EXTENSIONS, DOC_EXTENSIONS, ARCHIVE_EXTENSIONS, APPLICATION_EXTENSIONS and LAX_IMAGE_EXTENSIONS.

All functions share these parameters:

Parameter	Description
stream	An opened file object. It is read from the beginning and restored to its original position afterwards.
enable_using_magic	Use the libmagic engine for this call. Default True.
enable_using_file_command	Use the file command for this call. Default True.
enable_using_puremagic	Use the puremagic engine for this call. Default True.
magic_content_length	Max bytes fed to the file command. Default 64MB. Content is never fully read into memory.
lax_extensions	List of extension sets (e.g. [simplemagic.LAX_IMAGE_EXTENSIONS]). Extensions inside one set are interchangeable during the compare.

Examples

Detect a mimetype by filename

import simplemagic

mimetype = simplemagic.get_mimetype_by_filename("photo.jpg")
print(mimetype)  # image/jpeg

Detect a mimetype from a stream

The stream position is restored after detection, so you can reuse the stream:

import simplemagic

with open("photo.jpg", "rb") as fobj:
    mimetype = simplemagic.get_mimetype_by_stream(fobj, "photo.jpg")
    print(mimetype)  # image/jpeg
    # fobj is still readable from its original position

Check whether the content matches the file extension

This is the most common use case, e.g. rejecting a text file renamed to .jpg:

import simplemagic

# photo.jpg really contains a jpeg -> True
print(simplemagic.is_file_content_matches_with_file_suffix("photo.jpg"))

# photo.png really contains a jpeg -> False
print(simplemagic.is_file_content_matches_with_file_suffix("photo.png"))

If you want the detected mimetype as well:

import simplemagic

matched, ext, mimetype = simplemagic.file_content_matches_with_file_extension_test("ok.docx")
if matched:
    print(f"{ext} content matches, mimetype is {mimetype}")
else:
    print(f"content mimetype is {mimetype}, which does not match extension {ext}")

Allow extensions to be interchangeable (lax compare)

Some image formats can legally be stored with another image extension. Pass a lax set so these are treated as a match:

import simplemagic

# the file content is a jpeg, but is named .png
print(simplemagic.is_file_content_matches_with_file_suffix("photo.png"))
# False (strict)

print(simplemagic.is_file_content_matches_with_file_suffix(
    "photo.png",
    lax_extensions=[simplemagic.LAX_IMAGE_EXTENSIONS],
))
# True (png/jpg/jpeg/gif/bmp/tif/ico/webp ... are interchangeable)

Register your own mimetype / extension mapping

import simplemagic

simplemagic.register_mimetype_extensions({
    "application/x-foo": [".foo", ".foo2"],
})

print(simplemagic.guess_all_extensions("application/x-foo"))
# ['.foo', '.foo2']

Disable engines

Detection runs libmagic -> file command -> puremagic. If a system package is missing or you only trust one engine, disable the others globally or per call:

import simplemagic

# global: only use the puremagic engine
simplemagic.magic.disable_using_magic()
simplemagic.magic.disable_using_file_command()

# per call: skip the `file` command engine this time
mimetype = simplemagic.get_mimetype_by_filename(
    "ok.docx",
    enable_using_file_command=False,
)

filemagic command util

simplemagic also ships a command util filemagic.

Usage of the command filemagic

test@test simplemagic % filemagic --help
Usage: filemagic [OPTIONS] [FILENAME]...

  Get file's mimetype information.

Options:
  --disable-magic         Don't use libmagic.
  --disable-file-command  Don't use file command.
  --disable-puremagic     Don't use puremagic.
  --help                  Show this message and exit.

Example files test result

test@test simplemagic % filemagic *

ok.bash_history: text/plain
ok.bash_profile: text/plain
ok.bashrc: text/plain
ok.conf: text/plain
ok.coverage: application/vnd.sqlite3
ok.csv: text/plain
ok.dat: application/octet-stream
ok.doc: application/msword
ok.docx: application/vnd.openxmlformats-officedocument.wordprocessingml.document
ok.dot: application/vnd.openxmlformats-officedocument.wordprocessingml.document
ok.dps: application/vnd.openxmlformats-officedocument.presentationml.presentation
ok.dpt: application/vnd.openxmlformats-officedocument.presentationml.presentation
ok.et: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
ok.ett: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
ok.gif: image/gif
ok.gitignore: text/plain
ok.htaccess: text/plain
ok.in: text/plain
ok.ini: text/plain
ok.java: text/x-java
ok.jpg: image/jpeg
ok.less: text/plain
ok.log: text/plain
ok.md: text/plain
ok.pages: application/zip
ok.pdf: application/pdf
ok.pl: text/x-perl
ok.png: image/png
ok.pptx: application/vnd.openxmlformats-officedocument.presentationml.presentation
ok.properties: text/plain
ok.py: text/x-script.python
ok.rpm: application/x-rpm
ok.scss: text/plain
ok.sh: text/x-shellscript
ok.sql: text/plain
ok.svg: image/svg+xml
ok.tar.gz: application/gzip
ok.ttf: font/sfnt
ok.txt: text/plain
ok.txt.bz2: application/x-bzip2
ok.whl: application/zip
ok.woff: application/octet-stream
ok.woff2: application/octet-stream
ok.wps: application/vnd.openxmlformats-officedocument.wordprocessingml.document
ok.wpt: application/vnd.openxmlformats-officedocument.wordprocessingml.document
ok.wsdl: text/xml
ok.xlsx: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
ok.xmind: application/zip
ok.xml: text/xml
ok.xsl: text/xml
ok.yml: text/plain
ok.zip: application/zip
private.DS_Store: application/octet-stream
private.bpmn: text/xml
private.cab: application/vnd.ms-cab-compressed
private.class: application/x-java-applet
private.dll: application/x-dosexec
private.dmg: application/x-bzip2
private.doc: application/msword
private.dwg: image/vnd.dwg
private.fla: application/CDFV2
private.ftl: text/html
private.ico: image/vnd.microsoft.icon
private.img: application/octet-stream
private.inf: text/plain
private.jsp: text/html
private.mht: message/rfc822
private.mp4: video/mp4
private.mpp: application/vnd.ms-office
private.msi: application/x-msi
private.pcap: application/vnd.tcpdump.pcap
private.pps: application/vnd.ms-powerpoint
private.ppt: application/vnd.ms-powerpoint
private.psd: image/vnd.adobe.photoshop
private.pyc: application/x-bytecode.python
private.rar: application/x-rar
private.reg: text/x-ms-regedit
private.swf: application/x-shockwave-flash
private.tar: application/x-tar
private.tif: image/tiff
private.vsd: application/vnd.ms-office
private.xls: application/vnd.ms-excel
private.xps: application/zip
private.xsd: text/xml

Notice

Always upgrade your libmagic to the latest, old libmagic may get wrong answer.

Compatibility

• test passed on python3.6, python3.7, python3.8, python3.9, python3.10, python3.11 and python3.12
• test failed on python2.7, python3.3, python3.4, python3.5

Releases

v0.1.13

• Adapt to puremagic 2.x. puremagic >= 2.0 renamed its internal helpers (e.g. _stream_details -> stream_details, _max_lengths -> get_max_lengths), fixpuremagic now supports both naming schemes.
• Disable puremagic 2.x's deep scan. Its scanners may read the whole file into memory (e.g. json.load on files of any size), which is forbidden for large files (e.g. > 10G), and its stream detection would inspect the on-disk file instead of the passed stream. Detection stays bounded to the file header/footer only.
• Add application/javascript extension mapping to keep .js/.mjs detection consistent on python3.12.
• Packaging is now based on pyproject.toml, setup.py is removed. Use python3 -m build to build sdist and wheel.

v0.1.12

• Doc update.

v0.1.11

• Add many items in EXTRA_MIMETYPE_EXTENSIONS.

v0.1.10

• Add mimetype and file extension binding from nginx's default mimetype settings.
• Add mimetype and file extension binding from https://docs.w3cub.com/http/basics_of_http/mime_types/complete_list_of_mime_types.html.
• Add mimetype adn file extension binding from https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types.

v0.1.9

• Remove .svg from text/plain, for both libmagic and puremagic are not treat .svg file as text/plain. libmagic treat it as image/svg+xml, and puremagic treat it as application/xml. If put .svg in candidate extensions of text/plain, in image lex compares model, will allow user upload plain text script in image file field.

v0.1.8

• Add lax_extensions parameter in function is_file_content_matches_with_file_extension to support lax extension compares, especially for user missing .jpg, .png extension for images.
• Add LAX_IMAGE_EXTENSIONS = [".png", ".jpg", ".jpe", ".jpeg", ".gif", ".bmp", ".tif", ".tiff", ".webp", ".ico"].

v0.1.7

• Add magic_content_length parameter in function is_file_content_matches_with_file_extension to control the stream content read length locally.
• Fix export api name problem.

v0.1.5

• Put function is_file_content_matches_with_file_extension to public.
• Using magic.detect_from_fobj instead of magic.detect_from_content to improve the recognition.
• Change register_mimetype_extensions' parameters, and fix the problem.
• Fix .dps, .dpt, .et, .ett extension problems.
• Fix .dox problem.
• Fix .mptt problem.
• Fix .csv problem.
• Fix .pcap problem.
• Fix .rpm problem.
• Fix .dmg problem.
• Fix .reg problem.
• Fix .dwg problem.
• Fix .xps problem.
• Fix .ttf problem.
• Fix .woff and .woff2 problem.
• Fix java .class problem.
• Fix .jsp problem.
• Fix .less and .scss problem.
• Fix .pyc problem.
• Fix .fla problem.
• Fix .vsd problem.

v0.1.1

• Recover stream position after mimetype detect.
• Fix small file handling problem in puremagic.
• Fix .gz extension problem.
• Fix .bz2 extension problem.

v0.1.0

• First release.