Fast YAML 1.2 Parser for Python 3.6+
Project description
🚀 zaml
Fast YAML 1.2 parsing library for Python 3.6+ 🐍
What's This?
Proof-of-concept for my PyCon DE 2022 talk, video, Speeding Up Python with Zig, not yet recommended for production use!
Library with the following objectives:
- Written in pure Zig, importing
Python.h
headers directly, no FFI,ctypes
orcffi
. - Compiled using the Zig toolchain / CLI, no other tool (eg.
clang
) required. - Tested to be compatible with mac OSX, Linux and Windows.
- Installable via PyPI
- Should not require Zig toolchain locally in order to install and use.
- Fastest available YAML 1.2 parser for Python.
Help wanted to achieve the full objectives, PRs welcome.
Installation
pip install zaml
Note: currently source distribution only, ie. sdist
- no binary wheels
(yet), therefore requires Zig 0.10.0
installed locally. Any other Zig version is untested.
Installing Locally
Some pre-requisites (linting etc.), pyenv
also recommended:
pre-commit install
pre-commit run --all-files
The simplest possible extension module is a module with one function, that takes no arguments and returns an integer. This repo demonstrates a pure Zig module that does exactly that:
python -m venv .venv
source .venv/bin/activate
pip install -e .
Tests
Tests that the most basic possible Zig extension can in-fact be installed and returns the expected result:
python test.py
Benchmark
To run a benchmark of the current zaml
prototype (also runs in CI and asserts that the YAML structure is correctly
parsed):
cd benchmark
python benchmark.py
Results on my 2,3 GHz Quad-Core Intel Core i7 Mac:
Running benchmarks...
Benchmark results:
zaml took 0.89 seconds
PyYAML CSafeLoader took 13.36 seconds
ruamel took 38.86 seconds
PyYAML SafeLoader took 81.78 seconds
Credits
Would not exist without kubkon's, zig-yaml
: https://github.com/kubkon/zig-yaml
Cross-platform Local Testing
Linux
To test in Linux, the easiest way is probably to use Docker:
docker run --name zaml -v $PWD:/root/zaml -it fedora
This kicks you into a shell in a running a container with this library mounted in
the /root/zaml
directory. Changes you make on your host machine will be immediately
reflected in the container.
Install Python 3 headers, zig and test the library:
dnf install zig python3-devel
cd /root/zaml
python3 -m venv .venvlinux
source .venvlinux/bin/activate
pip install -e .
To re-attach to the container after exiting:
docker start -ia zaml
Windows
To test in Windows from a Mac, the easiest way I have found is to use Parallels.
MacOSX
I am writing this README
on a Mac. Consequently, I have not attempted testing this library in MacOSX from another
operating system host. If you manage this, please do add documentation about it here.
Publishing to PyPI
Note: Temporary instructions (until full CI setup).
You may need to upgrade build
and twine
(with your virtualenv
activated):
python -m pip install --upgrade build
python -m pip install --upgrade twine
Then:
rm -rf dist
python3 -m build --sdist
python3 -m twine upload --repository pypi dist/*
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for zaml-0.0.9-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 08df81def4485b604c452b2cfb959a99f67f8dcd1c31297f4d3d68d527bd8ffc |
|
MD5 | c15ed2db462d9d26f5fc8f681b24fd34 |
|
BLAKE2b-256 | 7c8a8962e4c5fdd9fd25e2ec7a7d32d6d14288ca7826000e7f873b9a5d7f9078 |
Hashes for zaml-0.0.9-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 391e45b8f86efbbcf7815516e49a02f31c358bc4c13f7ae95939d22597df6370 |
|
MD5 | 48cd7f4e154130505c18a4745a299386 |
|
BLAKE2b-256 | da9e09440f77c2db96f55705d7ad1e134948e9717bac5d0b60ca58de8a68c680 |
Hashes for zaml-0.0.9-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 48ca7832aea5b7b362fd2ce58740530d4ba85dce31bb93a2c357659147bc3a9a |
|
MD5 | b21afee3c49666ae9ce111a7436e9ce0 |
|
BLAKE2b-256 | 5427998d14a46d6b21c909737c4df66601dff703efcb7ac2b34921912330a3ad |
Hashes for zaml-0.0.9-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 14d637676247357505a531254ec884d77b218a3b1c05ad0ff50edceedebe923b |
|
MD5 | 200e948c7cb8deae6df24ac7d4d03fdb |
|
BLAKE2b-256 | e6f2093feae397965cfd60c40047ae25c4dfb448c35b87dbef4ca26bc19e4ea4 |
Hashes for zaml-0.0.9-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 737c5aab2da0ae2728752ebee15dbfa5755eb5cebc8aa79b0cbf93eda5dfe324 |
|
MD5 | 5e63006497c367e0c85324e36b62d232 |
|
BLAKE2b-256 | 702f69730f21286899121164c61cb33983081adcba23f8a3bfd87f0c6bd14066 |
Hashes for zaml-0.0.9-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ca8a377fd959d12d53853fc42913edfd29091af7b915a8d93e3309de74bc642 |
|
MD5 | 16e0a6a3ba9125e68e200cb5f9ea41e1 |
|
BLAKE2b-256 | 50d031559e08887af48129baf84a2c84cea58d91c4aa12c0e92b7ddbf8c27b83 |
Hashes for zaml-0.0.9-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7de80eb7df7a2fd4dc7cff8d15d19ef385ee8d09b2caedc7ff9068589ac190a0 |
|
MD5 | 6d3b1cb4e28c750f0dfee7d397f787fd |
|
BLAKE2b-256 | 56f618f86f2c7d728eb60363b2bee6f113251e9797984502b2b897b8ed4d254b |
Hashes for zaml-0.0.9-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 37fdf546f6479ed7fe58c649a0a831f7409407eeca48b44f49ac2c317f5d7e3d |
|
MD5 | c4b7bf6ff8a77c6ec91aa6d27d330a4e |
|
BLAKE2b-256 | 5ba6895a2f518bf1e945f4fd06461d6f4aaad1f90dcbde829892e618c2756378 |
Hashes for zaml-0.0.9-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c4812c683f0f89a9363c32710e45825b1d1441e72678c9dd3479ad87a614f9a5 |
|
MD5 | 7945a4461ae5d42c210db02db4336e87 |
|
BLAKE2b-256 | fda0153bd40e3655c829764d090825332340366ae1595921fe60cdaf62e6b270 |
Hashes for zaml-0.0.9-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e62f03b906103bb9beea16ee62a06792ad5c6bcc5461bea9dbd6aa1197f4141 |
|
MD5 | e8e2f9c5cad8a9c3eb501a6cc165a686 |
|
BLAKE2b-256 | 1c01d348b6784d767c1413a4969ba5371079483a109827f61f1d963cfa838d35 |
Hashes for zaml-0.0.9-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d9ed336e23b96c89865d9d0c494972b0a12e13ca98f0476cefdef02740209a1b |
|
MD5 | 9c55a93897f1af162e4b7959acc92b0e |
|
BLAKE2b-256 | 828994c3a3907f85faf68eb65cd6a18d1cde26fb4a1484c59ef52d769a3da6b7 |
Hashes for zaml-0.0.9-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3ff9d37f7422c59c06f2bd429aca3b303d1f10ca265d7c2857e58d39cf6fd5b2 |
|
MD5 | 72dc85d3933dd66dc5f87ce75b362ddb |
|
BLAKE2b-256 | 464e1568d694d651392598edb189035113aa23a5dc98bc9b9a15f8a285b9f127 |