A Fast, spec compliant Python 3.12+ tokenizer that runs on older Pythons.
Project description
pytokens
A Fast, spec compliant Python 3.12+ tokenizer that runs on older Pythons.
WORK IN PROGRESS!
Installation
pip install pytokens
Usage
TODO
Local Development / Testing
- Create and activate a virtual environment
- Run
pip install -r requirements-dev.txtto do an editable install - Run
pytestto run tests
Type Checking
Run mypy .
Create and upload a package to PyPI
Make sure to bump the version in setup.cfg.
Then run the following commands:
rm -rf build dist
python setup.py sdist bdist_wheel
Then upload it to PyPI using twine:
twine upload dist/*
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pytokens-0.0.0.tar.gz
(3.2 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pytokens-0.0.0.tar.gz.
File metadata
- Download URL: pytokens-0.0.0.tar.gz
- Upload date:
- Size: 3.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.0.1 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
49b6eb2e3eb6ad6a60916393470cf1d694b64a39224853db349c89b9687459ce
|
|
| MD5 |
d125e320eb336885ffa9d5b7bd954aa7
|
|
| BLAKE2b-256 |
a23290616452ea56386a4c15e2c313cb642b2c4e84d1b40e80140b1d81999895
|
File details
Details for the file pytokens-0.0.0-py3-none-any.whl.
File metadata
- Download URL: pytokens-0.0.0-py3-none-any.whl
- Upload date:
- Size: 4.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.0.1 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
5799a7f53acb2519e5f746c21adb6a9cc8dec2e2bba77e84e15edfc126702da8
|
|
| MD5 |
c5ca3e163251f11b41305b34652513a2
|
|
| BLAKE2b-256 |
02ca510a4e2fd184dac576253a64d7fbfa8669e983a29fc508ac1e38451af740
|
