pybased

Library for creating arbitrary binary encodings. Includes variations on base32, base64, base85, and more.

WARNING: Although these encodings do work end-to-end, they are not compatible with traditional implementations!

I am working on finding out why.

Encodings

Types

• Sliding: Random-access bitwise implementation, theoretically compatible with bytestrings of arbitrary size.
• BigInt: Converts bytes into large integers, then incrementally divides by the radix. Incompatible with large files. Suitable for hashes and shorter bytestrings.
• Biterator: Iterates through each byte from the input stream and adds it to a buffer, then extracts the required number of bits to convert to the other radix. Based roughly off of CPython's base64 encoder, but heavily modified to enable arbitrary conversions. WIP.

Available Encodings

Status

base85: c = 99, len = 85, bpc = 7

Getting started

$ pip install pybased

Doing stuff

# Lets's assume we want to use the Crockford32 encoding scheme.
from based.standards.base32 import crockford32

# And let's assume the variable data has what we want to encode.
data: bytes = ...

# Encode to string.
encoded: str = crockford32.encode_bytes(data)

# ...

# Decode the string back to bytes.
data: bytes = crockford32.decode_bytes(encoded)

based Command-Line Tool

$ based --help

usage: based [-h] {dump,encode,decode} ...

positional arguments:
  {dump,encode,decode}

optional arguments:
  -h, --help            show this help message and exit

NOTE: The based CLI tool is currently only useful for testing, and is under very active development.

Encode string to Base94

$ based encode --standard=base94 --input-string 'Hello, world!'

/P|?l:+>Nq\sr<+r

Encode string to Base94 and output JSON

$ based encode --standard=base94 --input-string 'Hello, world!' --output-format json

{"encoded": "/P|?l:+>Nq\\sr<+r"}

Decode data from Base94

$ based decode --standard=base94 --input-string '/P|?l:+>Nq\sr<+r'

>>> input chars: 16B
Bytes representation: b'Hello, world!'
Hex representation: 48656c6c6f2c20776f726c6421

Various Output Encoding Formats

$ based decode --standard=base94 --input-string '/P|?l:+>Nq\sr<+r' --output-format json

>>> input chars: 16B
{"b64": "SGVsbG8sIHdvcmxkIQ==", "utf-8": "Hello, world!"}

$ based decode --standard=base94 --input-string '/P|?l:+>Nq\sr<+r' --output-format yaml

>>> input chars: 16B
encoded: !!binary |
  SGVsbG8sIHdvcmxkIQ==