fastpybase64

A Python binding for the base64 functions of simdutf, the library that Node.js uses for base64. It is a drop-in replacement for the standard library's base64 module: every function returns the same bytes and raises the same exceptions as its counterpart in base64, for every input. It decodes up to 80 times faster than the standard library and up to 13 times faster than pybase64, and it is several times faster than both on small inputs.

With pip:

pip install fastpybase64

With uv:

uv pip install fastpybase64      # in a uv project: uv add fastpybase64

Wheels are built for Linux, macOS and Windows, for Python 3.10 to 3.14, including free-threaded Python 3.14.

Usage

import fastpybase64 as base64

base64.b64encode(b"hello")                     # b'aGVsbG8='
base64.b64decode("aGVsbG8=")                   # b'hello'
base64.b64decode(b"aGVsbG8=", validate=True)   # b'hello'
base64.urlsafe_b64encode(b"\xfb\xff")          # b'-_8='
base64.encodebytes(data)                       # MIME: lines of 76 characters

The functions accept what base64 accepts: bytes, bytearray, memoryview and other buffers, and ASCII str for the decoders.

Same results as base64

simdutf decodes base64 following the WHATWG forgiving-base64 rules, used by atob in web browsers. These rules differ from those of Python's binascii module: simdutf skips ASCII spaces and accepts missing padding, whereas the standard library requires padding and, with validate=True, rejects spaces. fastpybase64 keeps Python's behavior. Each call decodes with simdutf, then a cheap check confirms that base64 would return the same bytes for this input. When it cannot confirm it, fastpybase64 calls base64 with the same arguments and returns its result or lets its exception propagate. So invalid input raises exactly the binascii.Error that base64 would raise, with the same message, and the rare inputs where base64 is lenient in odd ways (characters after the padding, for example) give the same bytes.

• With validate=False (the default), spaces, newlines and other characters outside the alphabet are ignored, as in base64. simdutf skips spaces and line breaks while it decodes: 1 MiB of MIME data (a newline every 76 characters) decodes at 12.0 GB/s on an Apple M4 Max, against 14.3 GB/s without newlines (pybase64: 4.1 GB/s, base64: 1.3 GB/s).
• With validate=True, any character outside the alphabet, including a space or a newline, raises binascii.Error.
• Characters after the padding are handled as in the version of Python you run: Python 3.13 and earlier stop decoding at the padding, Python 3.14 continues or raises.
• urlsafe_b64decode, like the standard library, accepts both alphabets (+/ and -_).
• An altchars other than b"+/" and b"-_" is handled by encoding with simdutf and then replacing the two characters; decoding with such an alphabet is left to base64.

Threads

The GIL is released while encoding or decoding 64 KiB or more, so several threads can work at once. The module is marked free-threading compatible (Py_MOD_GIL_NOT_USED), so importing it on a free-threaded build of Python does not enable the GIL. Do not change a bytearray while another thread encodes or decodes it: the result would be unpredictable (its size cannot change, so memory stays safe).

Build and test

Python 3.10 or newer, and a C++17 compiler (clang, GCC or MSVC). The simdutf 9.3.1 amalgamation is in vendor/. Only its base64 functions are compiled (see setup.py).

pip:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip
python -m pip install -e ".[test]"
pytest tests

uv:

uv venv
. .venv/bin/activate
uv pip install -e ".[test]"
pytest tests

The tests compare every function with base64, results and exceptions, on inputs of every length from 0 to 199 bytes and larger ones, on the edge cases of padding and of characters outside the alphabet, on every supported input type, on wrong arguments, and on thousands of randomly corrupted inputs.

bench.py compares fastpybase64 with base64 and pybase64 (the bench extra) from 16 bytes to 16 MiB:

python -m pip install -e ".[bench]"
python bench.py
python bench.py --sizes 64,4096,65536 --ops b64encode,b64decode

Benchmarks

Intel Xeon Gold 6548N (Emerald Rapids), one core, Python 3.14.6, fastpybase64 0.1.0 (simdutf 9.3.1, icelake kernel), pybase64 1.5.1 (its AVX512VBMI kernel), random bytes. Speeds are in GB/s of binary data, for encoding and decoding alike (higher is better).

Decoding

pybase64 is fast only with validate=True; by default (validate=False, as in base64) and in urlsafe_b64decode, it is 4 to 13 times slower than fastpybase64 from 4 KiB on. pybase64 has no decodebytes.

Encoding

From 64 KiB on, b64encode runs at the same speed in pybase64 and fastpybase64: both use the same AVX-512 algorithm, and which one is ahead changes from run to run (within 6%). On smaller inputs, and for the URL-safe alphabet and MIME line breaks, fastpybase64 is faster.

Small inputs

On small inputs, the cost of the call dominates. fastpybase64 parses its arguments without creating objects and writes the result directly into the bytes object it returns.

Apple M4 Max

macOS, Python 3.14.5, simdutf's arm64 (NEON) kernel, pybase64 with its NEON kernel. GB/s:

Releases

./release.sh X.Y.Z sets the version in pyproject.toml and src/fastpybase64.cpp, runs the tests in a fresh virtual environment, commits, tags vX.Y.Z and pushes. The tag starts the buildwheels workflow, which builds the wheels and the source distribution and uploads them to PyPI.

References

• Daniel Lemire, Fast Forgiving Base64 Decoding with SIMD Instructions, ACM Transactions on Architecture and Code Optimization, 2026.
• Wojciech Muła, Daniel Lemire, Base64 encoding and decoding at almost the speed of a memory copy, Software: Practice and Experience 50 (2), 2020.
• Wojciech Muła, Daniel Lemire, Faster Base64 Encoding and Decoding using AVX2 Instructions, ACM Transactions on the Web 12 (3), 2018.

License

Apache-2.0 or MIT, at your option, like simdutf.