⏳ tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
import tiktoken
enc = tiktoken.get_encoding("gpt2")
assert enc.decode(enc.encode("hello world")) == "hello world"
# To get the tokeniser corresponding to a specific model in the OpenAI API:
enc = tiktoken.encoding_for_model("text-davinci-003")
The open source version of tiktoken can be installed from PyPI:
pip install tiktoken
The tokeniser API is documented in tiktoken/core.py.
Example code using tiktoken can be found in the
OpenAI Cookbook.
Performance
tiktoken is between 3-6x faster than a comparable open source tokeniser:
Performance measured on 1GB of text using the GPT-2 tokeniser, using GPT2TokenizerFast from
tokenizers==0.13.2, transformers==4.24.0 and tiktoken==0.2.0.
Getting help
Please post questions in the issue tracker.
If you work at OpenAI, make sure to check the internal documentation or feel free to contact @shantanu.
Extending tiktoken
You may wish to extend tiktoken to support new encodings. There are two ways to do this.
Create your Encoding object exactly the way you want and simply pass it around.
cl100k_base = tiktoken.get_encoding("cl100k_base")
# In production, load the arguments directly instead of accessing private attributes
# See openai_public.py for examples of arguments for specific encodings
enc = tiktoken.Encoding(
# If you're changing the set of special tokens, make sure to use a different name
# It should be clear from the name what behaviour to expect.
name="cl100k_im",
pat_str=cl100k_base._pat_str,
mergeable_ranks=cl100k_base._mergeable_ranks,
special_tokens={
**cl100k_base._special_tokens,
"<|im_start|>": 100264,
"<|im_end|>": 100265,
}
)
Use the tiktoken_ext plugin mechanism to register your Encoding objects with tiktoken.
This is only useful if you need tiktoken.get_encoding to find your encoding, otherwise prefer
option 1.
To do this, you'll need to create a namespace package under tiktoken_ext.
Layout your project like this, making sure to omit the tiktoken_ext/__init__.py file:
my_tiktoken_extension
├── tiktoken_ext
│ └── my_encodings.py
└── setup.py
my_encodings.py should be a module that contains a variable named ENCODING_CONSTRUCTORS.
This is a dictionary from an encoding name to a function that takes no arguments and returns
arguments that can be passed to tiktoken.Encoding to construct that encoding. For an example, see
tiktoken_ext/openai_public.py. For precise details, see tiktoken/registry.py.
Your setup.py should look something like this:
from setuptools import setup, find_namespace_packages
setup(
name="my_tiktoken_extension",
packages=find_namespace_packages(include=['tiktoken_ext*']),
install_requires=["tiktoken"],
...
)
Then simply pip install ./my_tiktoken_extension and you should be able to use your
custom encodings! Make sure not to use an editable install.
1 maintainer
Release files for pandora-tiktoken 0.3.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
Total release size: 58.7 MB
Release files / pandora_tiktoken-0.3.1-cp311-cp311-win_amd64.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-win_amd64.whl |
|---|---|
| Size | 581.0 kB |
| Tags | CPython 3.11 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
1a271f8c52befcf7f218d40801c6220e8e77470a1c4da0ae7339c75459b44279
|
|
BLAKE2b-256 checksum How to use checksums |
bf370cf235f09a39e87e44666fa8b326e4b4ade6fddba25270dd5ed4e3f22358
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-win32.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-win32.whl |
|---|---|
| Size | 525.3 kB |
| Tags | CPython 3.11 Windows x86-32 |
|
SHA-256 checksum How to use checksums |
847c91288086757bfbe4837009c7ee90130794f0a5aecfbcf18e20b1c8b8ab3e
|
|
BLAKE2b-256 checksum How to use checksums |
aec20756e71952a1d8e70fc4cf4abb97dc54f61ba5ee548904414cd4803d83db
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-musllinux_1_1_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-musllinux_1_1_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.11 Linux musl 1.1+ x86-64 |
|
SHA-256 checksum How to use checksums |
759749f1f21228c98246e404325a5745a9633a9013a2f3fbca3fdc7740a156c0
|
|
BLAKE2b-256 checksum How to use checksums |
4f439ac7300aeb61e2206696fd1e28a33f4d614b928061b93faea7dc99d0e191
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-musllinux_1_1_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-musllinux_1_1_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.11 Linux musl 1.1+ ARM64 |
|
SHA-256 checksum How to use checksums |
153c4980e7ea87d2828b36075cfb18c13148cd7027b1c68ffde6d1b07a42e515
|
|
BLAKE2b-256 checksum How to use checksums |
25cf8230d84ad318a2a2b4e116f62ca2041aec3b1a754125e7e72273189778b6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
eafd04c9de68d9d3066a9c5760b2948d2fd621fd4e8bb335c47824bc54621059
|
|
BLAKE2b-256 checksum How to use checksums |
ad201ccb85ce16082f6385760a15ccfb07835b2112de5df08281d5228458bae1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
2eadb6e288f4c0f487c9a110f9d9692d9b3f099117d87f7e3f5b1444553cb76d
|
|
BLAKE2b-256 checksum How to use checksums |
04a649b14b71d48b5008d4aaad9ffb0d065e52de0be2956ed8eb6f7524955d23
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-32 Linux glibc 2.5+ x86-32 |
|
SHA-256 checksum How to use checksums |
b449c75c4ab2733b0bd228237e80eca2792b903194e3908c34eb4197026514aa
|
|
BLAKE2b-256 checksum How to use checksums |
c0ff4ca9c24fa967a73851876800b195ee396135cd539e1cbbf1eaed4e677bde
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-macosx_11_0_arm64.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-macosx_11_0_arm64.whl |
|---|---|
| Size | 701.7 kB |
| Tags | CPython 3.11 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
67cb9b9c864353f7c191b2022e6a532ebaa31ac9bfb47d28515fbeee3411a0b4
|
|
BLAKE2b-256 checksum How to use checksums |
b092c7f3d90ee09d23e3d63b24a3e9a6765292341f7e49b6390fcb5ce622753e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-macosx_10_9_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-macosx_10_9_x86_64.whl |
|---|---|
| Size | 734.9 kB |
| Tags | CPython 3.11 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
77ef50ece7535c7ed2646b26b764bab6e32e74d8f38efa185bca43957ac4c509
|
|
BLAKE2b-256 checksum How to use checksums |
6254c8a6ee428a90bac7e13fddc94130cf0188a460f90936b872099eafcafc30
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp311-cp311-macosx_10_9_universal2.whl
| Download URL | pandora_tiktoken-0.3.1-cp311-cp311-macosx_10_9_universal2.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.11 macOS 10.9+ universal2 (ARM64, x86-64) |
|
SHA-256 checksum How to use checksums |
484278a4537385b62a4f02b0cbc25b0ada662c91cc48136cf10eabd3e19dff4b
|
|
BLAKE2b-256 checksum How to use checksums |
be83f69d1b070d0376e03f51e9412e1b66ef709989987b1a30d08addf0b6d204
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-win_amd64.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 581.0 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
1cf526345e04b02f074110c302b690bf350128180a5aef83558cab2f6578dd67
|
|
BLAKE2b-256 checksum How to use checksums |
be7a97be87cec0db7fd57eb5c333619aaa9eaaf750154bc695e9c75817263537
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-win32.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-win32.whl |
|---|---|
| Size | 525.3 kB |
| Tags | CPython 3.10 Windows x86-32 |
|
SHA-256 checksum How to use checksums |
93386fb276cb0dd9af64dd0582ae497462b1aaa8e93a0aae606a2f0c4ca682ce
|
|
BLAKE2b-256 checksum How to use checksums |
f0008c953ccb6d7a1285fccb1ed6ca4b8e45bc2f8905c71d6f21653d222d4d8c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-musllinux_1_1_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-musllinux_1_1_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.10 Linux musl 1.1+ x86-64 |
|
SHA-256 checksum How to use checksums |
5eb0edbc28218881475b1538009649188ff4f6cb3bc1666863fdc29e05fa705d
|
|
BLAKE2b-256 checksum How to use checksums |
19b6774dd0f0c1832977ef8cdf4537b2a6a91d03b67c05548e788af8d03778b8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-musllinux_1_1_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-musllinux_1_1_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.10 Linux musl 1.1+ ARM64 |
|
SHA-256 checksum How to use checksums |
e0bf0357ada6690ee01ace756643e0ae12971ac34e2ff77a75275741fa9fe858
|
|
BLAKE2b-256 checksum How to use checksums |
52b37018b9167e7eb0b10fb2eb31c6a545b3658eb0132a28d03935c9189a7a92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
6c12d3208fd96edc04ff6098a687a62b8d41e1583e7b04eb10c930b5c22acb30
|
|
BLAKE2b-256 checksum How to use checksums |
4f721c36a79ca203bffc814157146d9b0091a160e113957a453a39d0fdb90875
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
67f07a5c159ed457157ee3f7b785b3a8e551e04a518229cf2431f4b41037bc3e
|
|
BLAKE2b-256 checksum How to use checksums |
4186fc33af3dcd1a360bc40927d9828515e2d29c73edf6b04c07886f5fc7a8f1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-32 Linux glibc 2.5+ x86-32 |
|
SHA-256 checksum How to use checksums |
994bb87a0f6888c6a8a4fea4cfe1ee8331eb6179dbf6516ef3e31b876419d319
|
|
BLAKE2b-256 checksum How to use checksums |
297cb2c27145cbb86c4433300031b7f88c7c3d63f7c2478a532aa7e10b6e8341
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-macosx_11_0_arm64.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-macosx_11_0_arm64.whl |
|---|---|
| Size | 701.7 kB |
| Tags | CPython 3.10 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
07edcf390bc9b130d96c1fc76abb3f6339ded3b90f2bd02237e41f9a5b8912c1
|
|
BLAKE2b-256 checksum How to use checksums |
d9f4eb1e52fdb48bfb0fca997c4b45609d0cdd49d950a25d86cadf8e6cd5a550
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-macosx_10_9_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-macosx_10_9_x86_64.whl |
|---|---|
| Size | 734.9 kB |
| Tags | CPython 3.10 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
fa70f48c93989cdedbc4b482e143fab148957dabb032204948c4b6451be3acea
|
|
BLAKE2b-256 checksum How to use checksums |
db82407736916fd4278bc24083e03fbf9a7c58cf606850baed97f23753c9fbc4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp310-cp310-macosx_10_9_universal2.whl
| Download URL | pandora_tiktoken-0.3.1-cp310-cp310-macosx_10_9_universal2.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64) |
|
SHA-256 checksum How to use checksums |
0ac0b97e2fb1ce4f65e748988319dc35d3aa602057a0637ec116e6735416fd50
|
|
BLAKE2b-256 checksum How to use checksums |
12761c1b94cafd2cf9c5948164f621aa8875f157647d54fbd6eb249981cb986d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-win_amd64.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-win_amd64.whl |
|---|---|
| Size | 581.4 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
9947050207d1edad9ba9434e042e288ea50c162b9f3ec0946cffadc0058cdc79
|
|
BLAKE2b-256 checksum How to use checksums |
b9f620d0129d34d8d1599daf61e5732ce0882d1fd5612b4fac3869dbf419f33b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-win32.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-win32.whl |
|---|---|
| Size | 525.3 kB |
| Tags | CPython 3.9 Windows x86-32 |
|
SHA-256 checksum How to use checksums |
5f06b5bea01f937654e8a6e61ba0fdb6f018a5156efcff2e31d06ed157464c51
|
|
BLAKE2b-256 checksum How to use checksums |
3c57f8ecc1e01045210e9ea1c2ab477a8a9cae6f3b66a298225f31d9ec184abe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-musllinux_1_1_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-musllinux_1_1_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.9 Linux musl 1.1+ x86-64 |
|
SHA-256 checksum How to use checksums |
e01e3104739ef0f6d487e901c05719f51998aec9a8055e6858dc71cb3accb5a4
|
|
BLAKE2b-256 checksum How to use checksums |
98dd6e171a40de817893b3dc28ca7bb4e946a030c0dda8dd2d6b8e261b3c16a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-musllinux_1_1_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-musllinux_1_1_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.9 Linux musl 1.1+ ARM64 |
|
SHA-256 checksum How to use checksums |
c13a4b6aa9d10222ae39b4b3f4d92f32ee9e8ae6d85ba1a598728b6f959db28a
|
|
BLAKE2b-256 checksum How to use checksums |
1b23e83131eb636d04c3e3bfba1d63b0848f458d44a1fd4c043da0ea1a505a30
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
7a5cc8a50445a6f3b0c32dda5eb2309ad5a3e04650f90b41cd2a9a5a5607a289
|
|
BLAKE2b-256 checksum How to use checksums |
1407caaf7a61bf6cf625adb8c24f34b4c8b3ad0daabb72950fccd74f5bf66839
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
0337ef85086a2695162466fe24db9962b348735acac4dce40aa58d34169d10f7
|
|
BLAKE2b-256 checksum How to use checksums |
862d9afa4e1134f6a8289943bbf0d291bb358ca399a50f79413cebb2b0046923
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ x86-32 Linux glibc 2.5+ x86-32 |
|
SHA-256 checksum How to use checksums |
44201294dc27b952254885221198908ac587658170ed1510375124716f5ea434
|
|
BLAKE2b-256 checksum How to use checksums |
9eb70c3d769f8eac9e7e88b5cd5c2cf688e4cc4b264b959934eea9a50c03f6cf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-macosx_11_0_arm64.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-macosx_11_0_arm64.whl |
|---|---|
| Size | 702.2 kB |
| Tags | CPython 3.9 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
b422e1b067c26a8605e5972983a4b549e6c2bd1481ad1591b99b33c4f89aa97c
|
|
BLAKE2b-256 checksum How to use checksums |
3e713ce134f992198d479301ce8480beaae4084cf1d2e65784403462ae461c75
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-macosx_10_9_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-macosx_10_9_x86_64.whl |
|---|---|
| Size | 735.2 kB |
| Tags | CPython 3.9 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
241bd3f77849660f390c0841ae18fc013894e52388dc72a0b513696740b94574
|
|
BLAKE2b-256 checksum How to use checksums |
b2739e890a2eacab4e7e9af7cb64b1294fea53bed183eb5ce8cc4af7e90a60ab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp39-cp39-macosx_10_9_universal2.whl
| Download URL | pandora_tiktoken-0.3.1-cp39-cp39-macosx_10_9_universal2.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64) |
|
SHA-256 checksum How to use checksums |
d86a2bebe6ddc3de514fc7eb2cb07fdaef04b56d39688ed06960e71797542121
|
|
BLAKE2b-256 checksum How to use checksums |
541f45ab2b58563fae5f2dc18c7939f9ae68b5fc4737075d8d7f34dc7d603369
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-win_amd64.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-win_amd64.whl |
|---|---|
| Size | 581.2 kB |
| Tags | CPython 3.8 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
3630e0197b7b9f33bad8a206d08ac413d78bb7d4518b909952c1ccbc4134b3a5
|
|
BLAKE2b-256 checksum How to use checksums |
5c3b2b27b4e438f68494b5195e4d6252fb74e01bc41fbacbd300fb21963f658a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-win32.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-win32.whl |
|---|---|
| Size | 525.3 kB |
| Tags | CPython 3.8 Windows x86-32 |
|
SHA-256 checksum How to use checksums |
36d16dd24e2be0d248b1567985445cdff81736ad06778f2f286f43ced98ec82a
|
|
BLAKE2b-256 checksum How to use checksums |
ab93ed7688e17c76d32b25525e884ad8874677bd50c7de6d021458535e86e34f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-musllinux_1_1_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-musllinux_1_1_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.8 Linux musl 1.1+ x86-64 |
|
SHA-256 checksum How to use checksums |
d340f422b96047082606b26e5c7075e25c9c4fe0ae8d87bca61f060615f0a447
|
|
BLAKE2b-256 checksum How to use checksums |
a9166c670982aad920f7f09159b0fce2f3b21a8d0923715553f91156ae947a00
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-musllinux_1_1_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-musllinux_1_1_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.8 Linux musl 1.1+ ARM64 |
|
SHA-256 checksum How to use checksums |
1d63590123544ac6f44a6492bfdf52cec17b6c95be78f0826001193272ac8758
|
|
BLAKE2b-256 checksum How to use checksums |
44958ed6d16b75d23f5f58cd1e235b1818fea7debed713dcea4648770bfc801f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
a27f6b86cd146629825d41076b6a64794b976f61d45cd9ee15e1ec12bd7f6baa
|
|
BLAKE2b-256 checksum How to use checksums |
ac7b0d1fbfdf7d62409aa5c78f4ded6920caace827440c93c1d132f3c2a77e5e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
26c9fe4a88155db5bcfce43b36c6c1c87cdd89ba3157e2fd619f5d78372063a8
|
|
BLAKE2b-256 checksum How to use checksums |
57c1e0392b4ae6d5d88582c296e958f16e099f6cecad46b9424a221876b380b1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-32 Linux glibc 2.5+ x86-32 |
|
SHA-256 checksum How to use checksums |
d7981d0e7f644c6316420764654ec6681846a910435658fbe5adca75cbcb8d2c
|
|
BLAKE2b-256 checksum How to use checksums |
845c4df0f761c56b8afe0c5f9dc5cd3f1657734e3b3c00e87ff10743080dc83b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-macosx_11_0_arm64.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-macosx_11_0_arm64.whl |
|---|---|
| Size | 702.5 kB |
| Tags | CPython 3.8 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
0b5073892028dd753b3b33045061dd3fb96e6ad5737baa520cc7e3d318e93b4f
|
|
BLAKE2b-256 checksum How to use checksums |
631b9fc81c51ff1584c261d029d2b81866f44ef8b9818da7ec91f2e2495d0ba1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-macosx_10_9_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-macosx_10_9_x86_64.whl |
|---|---|
| Size | 735.2 kB |
| Tags | CPython 3.8 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
53f0a3c5de325ec0755517cb9d147c5f877da12584462c2fbdc161b95fa85db5
|
|
BLAKE2b-256 checksum How to use checksums |
b8c2f0f12a9330eb7e5f7c3d15540052948b0f62281032370d8e371b9a26f700
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp38-cp38-macosx_10_9_universal2.whl
| Download URL | pandora_tiktoken-0.3.1-cp38-cp38-macosx_10_9_universal2.whl |
|---|---|
| Size | 1.4 MB |
| Tags | CPython 3.8 macOS 10.9+ universal2 (ARM64, x86-64) |
|
SHA-256 checksum How to use checksums |
19fda4f4eb3d7b78731cab1d99d1252c24c76ebc8e400c6cdeee464b9ba6cec4
|
|
BLAKE2b-256 checksum How to use checksums |
14e33f1b6442552430c1519807cd6f2c98defc5eade17e42b02d23e6c1de508d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-win_amd64.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-win_amd64.whl |
|---|---|
| Size | 581.2 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
aaa94766b6dc7523a9204a1ac69a7c63e2e7f6d7fad72a3f694c97b083842e8f
|
|
BLAKE2b-256 checksum How to use checksums |
72c862c42b5d6f59ca92a9c81718eb63102892e46cee6f5796f3444636dfa850
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-win32.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-win32.whl |
|---|---|
| Size | 525.4 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Windows x86-32 |
|
SHA-256 checksum How to use checksums |
dad635fb609f0783f78efce2521db8df283a0b42ec465348acb9a91eafb4f40c
|
|
BLAKE2b-256 checksum How to use checksums |
ab22ddb8932313dfa9b169c3e4ca8ea9be10358f525678ef6303d2fef09ce838
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-musllinux_1_1_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-musllinux_1_1_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux musl 1.1+ x86-64 |
|
SHA-256 checksum How to use checksums |
4a5f930c1468f765c6b125f20720a3e11fd1c3ea841ca626aed672a7c730d3aa
|
|
BLAKE2b-256 checksum How to use checksums |
7137d871ebf0763a522dfa2cc281733eec128b560a94e723f16bac9820244849
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-musllinux_1_1_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-musllinux_1_1_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux musl 1.1+ ARM64 |
|
SHA-256 checksum How to use checksums |
b38204d017b8a4adc574d8c7185ff9305a8d0b591899a373da2821155869b532
|
|
BLAKE2b-256 checksum How to use checksums |
c424597d641d5d7cf98a664264f8e948080c7f648d09d72bb64d5f75c2bd8bc7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
c2151bd62ad11490d7cf518452bc9d300bf69c9fb106bf87d50d60b18df007be
|
|
BLAKE2b-256 checksum How to use checksums |
0c4a62d24ff580920ef00d78f239dc40b2238af53f9b3f43cd9fe5bb0d4ec6a7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
e1144e400fb671bc472cb9f1a9c708d9dcaf9b603afe7300d88ed3883d65020d
|
|
BLAKE2b-256 checksum How to use checksums |
655dd3f26e533d4b958a392a86c4270679c0cab620a562aefa74ea9069d6f97e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl |
|---|---|
| Size | 1.6 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.17+ x86-32 Linux glibc 2.5+ x86-32 |
|
SHA-256 checksum How to use checksums |
dc8d7b40f46cfe3d372c448a0b947b2a34a073e53c2c14df2b314640dc842a3c
|
|
BLAKE2b-256 checksum How to use checksums |
9681be466eca0e62e5e7166cae75e0cba8b8eb7093ccc300ea91f6bcf322809e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
Release files / pandora_tiktoken-0.3.1-cp37-cp37m-macosx_10_9_x86_64.whl
| Download URL | pandora_tiktoken-0.3.1-cp37-cp37m-macosx_10_9_x86_64.whl |
|---|---|
| Size | 734.8 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
a025c104fd955b8c4506d3787fb775c57f496c7f22b0af3057a01a3377129aec
|
|
BLAKE2b-256 checksum How to use checksums |
4625768387b9a262727d304a1c8b9e77101f99cc383b927a1f395a53e7c3ed45
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.6
|
