spacy-alignments: Align tokenizations for spaCy + transformers
A spaCy package for Yohei Tamura's Rust tokenizations library with Python bindings.
Installation
pip install -U pip setuptools wheel
pip install spacy-alignments
If no binary wheel is available for your platform, you will need to install
Rust in order to build
spacy-alignments from source.
spacy-alignments vs. pytokenizations
The spacy_alignments module is a drop-in replacement for tokenizations:
import spacy_alignments as tokenizations
a2b, b2a = tokenizations.get_alignments(["å", "BC"], ["abc"])
assert a2b == [[0], [0]]
assert b2a == [[0, 1]]
The only difference between this package and the original
pytokenizations is that it
switches the build system to setuptools-rust to make it easier for us at
Explosion to build source and binary packages for a wider range of platforms.
Metadata
Release files for spacy-alignments 0.8.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| spacy-alignments-0.8.6.tar.gz | 6.8 kB | Details |
Built distributions (wheels)
Total release size: 12.5 MB
Release files / spacy-alignments-0.8.6.tar.gz
| Download URL | spacy-alignments-0.8.6.tar.gz |
|---|---|
| Size | 6.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
96240074a58bb9eb911ac904fbf25034882b4999acba4f6a98291f59749c625f
|
|
BLAKE2b-256 checksum How to use checksums |
fc1e9ce55ac7e9eee0947314677ec7060c80035928902c474dac62ffc52cd1c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp311-cp311-win_amd64.whl
| Download URL | spacy_alignments-0.8.6-cp311-cp311-win_amd64.whl |
|---|---|
| Size | 190.0 kB |
| Tags | CPython 3.11 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
c6bcdfa708c1b81ee5b40cd672cdae3e1f7e67118b25e1a3e1870b0b3214512c
|
|
BLAKE2b-256 checksum How to use checksums |
da98652bf629af2ea0c85072dead895f80279d9dfce3f805db11eb65b4d1a3b2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
381dd5a7f76ad22e8d4ec075926389fffc3f163e205219ec016d8088b9a3af0c
|
|
BLAKE2b-256 checksum How to use checksums |
42b219ce7f17e707528a7d3e76d7db619761ad728c48eee380f760a98ca03b3b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | spacy_alignments-0.8.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 272.5 kB |
| Tags | CPython 3.11 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
fd584e778c6122c6ba70c0ed2190ac7e9edfc6d10148c6c94f99ba9e1d58b60b
|
|
BLAKE2b-256 checksum How to use checksums |
ed080c49f892d0c6b1285d7e23c8c7fe1340264d6e965aff344dbfbb0c9fd500
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp311-cp311-macosx_11_0_arm64.whl
| Download URL | spacy_alignments-0.8.6-cp311-cp311-macosx_11_0_arm64.whl |
|---|---|
| Size | 311.9 kB |
| Tags | CPython 3.11 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
b2752e9a9d93784fa939f03ffa2431dcfa8fb0794387470e59ea070b74980c00
|
|
BLAKE2b-256 checksum How to use checksums |
c2764337bd8b6acc4fbe174205ea984dd9e0f603e94c3676d5a861adc01bd477
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp311-cp311-macosx_10_9_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp311-cp311-macosx_10_9_x86_64.whl |
|---|---|
| Size | 319.8 kB |
| Tags | CPython 3.11 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
d34b4b7f4950e21855af296d0c464a6873e15405efcd64d27d674ff1ccd147fe
|
|
BLAKE2b-256 checksum How to use checksums |
4fe00db91098aa03cf775641c35e9b09727b4b285de2c2f2f46850ca86146bdb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp310-cp310-win_amd64.whl
| Download URL | spacy_alignments-0.8.6-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 190.0 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
a0bf32e965ea03ffe974d43209e0c576d05365c2fec524e651a3a5ee235f82a6
|
|
BLAKE2b-256 checksum How to use checksums |
a0293318a8002341552a80c35c7119e97b4fbd6ea66191d2bb3f5246b1ccb74e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
5110c61b02f2cf248a133019340a79757c1aef4ab5d6b2e7296aa40b0f4f8b4b
|
|
BLAKE2b-256 checksum How to use checksums |
490572acfdcc758a2e7b4cc93111ab691d18ce89ce4319c7a925447143f83e5c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | spacy_alignments-0.8.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 272.5 kB |
| Tags | CPython 3.10 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
4dd037a4c2f7f18599f1fee50d61fcc6cac554583079aabadd31c50bef67a0a6
|
|
BLAKE2b-256 checksum How to use checksums |
30da975e376666c191baeb149806234478f7e74b9ae59fceb1cc83d640f667b2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp310-cp310-macosx_11_0_arm64.whl
| Download URL | spacy_alignments-0.8.6-cp310-cp310-macosx_11_0_arm64.whl |
|---|---|
| Size | 311.9 kB |
| Tags | CPython 3.10 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
8e23ae7d35755e305f3a29bcec9c857e1019eaa14cdbe1b916d0bd91d7c174f9
|
|
BLAKE2b-256 checksum How to use checksums |
20c2102b1f93254d3efc75ec5b63fb236dfd33d6b0ab6e08388fad3108f5179e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp310-cp310-macosx_10_9_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp310-cp310-macosx_10_9_x86_64.whl |
|---|---|
| Size | 319.8 kB |
| Tags | CPython 3.10 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
d4e20c5a63c26703f154f27d789717fccb5cc5ce9ce87fdbfac15c1e48c5f72f
|
|
BLAKE2b-256 checksum How to use checksums |
d9a9c1d82a88f86ac0553b0a0dc7814d9adfe5cccfd8f4e5d43bb03780a60387
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp39-cp39-win_amd64.whl
| Download URL | spacy_alignments-0.8.6-cp39-cp39-win_amd64.whl |
|---|---|
| Size | 190.0 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
c379e794cd22a0680709f5eee5008e96a986369f99d58e1422e3dadbe6bb963a
|
|
BLAKE2b-256 checksum How to use checksums |
b9d6ee4533de19e4435fcd59d7d955dda599ed5b4811fa2cf6d9806b4456bb39
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
fdd7fb3bf13630ee42fefe54a68f870485d4aab946bce25814e93aae3ba08b5d
|
|
BLAKE2b-256 checksum How to use checksums |
477e3c59e28175223c9a5e5b0720815df0145065400dff98601a05c1969703b2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | spacy_alignments-0.8.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 272.3 kB |
| Tags | CPython 3.9 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
ce31abe26232210039e0ccff9ea8ac81ce9dfac2d8653519d611fb1d2a1429b6
|
|
BLAKE2b-256 checksum How to use checksums |
a5860f89b231a71a05f85293f3b48f286545c1b47a4f4532c7ccd2428d9bb3c5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp39-cp39-macosx_11_0_arm64.whl
| Download URL | spacy_alignments-0.8.6-cp39-cp39-macosx_11_0_arm64.whl |
|---|---|
| Size | 311.9 kB |
| Tags | CPython 3.9 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
0861622b2c29cfdb62e98d1f5adfc218f960babd0dc86370d6c9a218d7563242
|
|
BLAKE2b-256 checksum How to use checksums |
153ac523a2d5e4f98108903c182118b4d90054f380b901a4f6de3c7aae7e2f42
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp39-cp39-macosx_10_9_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp39-cp39-macosx_10_9_x86_64.whl |
|---|---|
| Size | 319.8 kB |
| Tags | CPython 3.9 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
0ecff8612cc87adc7cd783cc1bd636262e380372055bbc3b3ae6f468bd8c4faa
|
|
BLAKE2b-256 checksum How to use checksums |
0143c47e3b1676781102cc0d5a6bdc761ec9cdbb21320aa4d11e2da8c5457f5a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp38-cp38-win_amd64.whl
| Download URL | spacy_alignments-0.8.6-cp38-cp38-win_amd64.whl |
|---|---|
| Size | 189.8 kB |
| Tags | CPython 3.8 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
24965aba440355e5ee8f08b04c73774f92fbfae86b8da45a89561f2bf52abafc
|
|
BLAKE2b-256 checksum How to use checksums |
2d90b189fe48e19b06ff7fb9c81a31cc923a5ae905077f0a11e852490cc164cc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
cf6f9e4f04bca91f21896d7d50961562a74fb23b23ee449f0ba4d29d6455db2b
|
|
BLAKE2b-256 checksum How to use checksums |
0889b7219de36c2d42f45c14d5943d31781989687b4c0f8d0bf9f68944424a75
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | spacy_alignments-0.8.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 272.2 kB |
| Tags | CPython 3.8 Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
7856462e1ca2ccafa572e0a9fc5b36228adcc3ea13734f2ba16234a99bba47d8
|
|
BLAKE2b-256 checksum How to use checksums |
317520da36b3bab7d782582edbe7c6d147dd6cf9a4b2cbc880184291d0b31634
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp38-cp38-macosx_11_0_arm64.whl
| Download URL | spacy_alignments-0.8.6-cp38-cp38-macosx_11_0_arm64.whl |
|---|---|
| Size | 311.9 kB |
| Tags | CPython 3.8 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
2172e4b3a7654446cdee52edaac2423d06b83fad11e80af2279a078a0ec983b1
|
|
BLAKE2b-256 checksum How to use checksums |
daa70266946dd29dd1d9197300cb9d2c865180a73419a06af12ce19b1885dcef
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp38-cp38-macosx_10_9_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp38-cp38-macosx_10_9_x86_64.whl |
|---|---|
| Size | 319.8 kB |
| Tags | CPython 3.8 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
bf8da9e1f16491142c402f8ec44c02b71b8291eae631ccba75be313ab852ab3e
|
|
BLAKE2b-256 checksum How to use checksums |
e55c1482120ab0fdea3efce8cb3d7e4e992dc35c95c94e2dbea4856ae0a4d3f7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp37-cp37m-win_amd64.whl
| Download URL | spacy_alignments-0.8.6-cp37-cp37m-win_amd64.whl |
|---|---|
| Size | 189.8 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
41bc3951da7c7565ed792cfb7c2cee97efd8b4c8fd83b61d84ce31715f999d2d
|
|
BLAKE2b-256 checksum How to use checksums |
0ca80835d62da22cb8fe3ad74e5a24808383322a76d5082555b9fa3832aaf16b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
7a419950cab37fe84123b4b859ca6de49fe259f88ac20347c7d7c30094ad3c1c
|
|
BLAKE2b-256 checksum How to use checksums |
665fb093550d14e20e7b7660df7a5d8058fa4188fa01a728b827d01f50e504b2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | spacy_alignments-0.8.6-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 272.3 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
c0cc271073a93f71e884ce6735a5469765e150d8d304208cadcc8463771a0eab
|
|
BLAKE2b-256 checksum How to use checksums |
64f6ff57996ea6d4a507dbaabf603c10bc8ff72536839cc556686323d1d36222
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp37-cp37m-macosx_10_9_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp37-cp37m-macosx_10_9_x86_64.whl |
|---|---|
| Size | 319.8 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
ab9874e01c043f5c2b06452b3b629eb61d937c674919a17f9f5c39dcc9554768
|
|
BLAKE2b-256 checksum How to use checksums |
a40b0384ac598f41c8fff2ff54354b87f957b7e14cfa1febaf9398ea8137ea8d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp36-cp36m-win_amd64.whl
| Download URL | spacy_alignments-0.8.6-cp36-cp36m-win_amd64.whl |
|---|---|
| Size | 190.4 kB |
| Tags | CPython 3.6 CPython 3.6 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
6037d0b8448d81d7cbac17f5abdfb47dd1dd5dbee61ae0f9db64181606dd5bb8
|
|
BLAKE2b-256 checksum How to use checksums |
cdc69147b8aef2cf38997f11ae63184ebb2c2c7ca9efaad340c9421ac936e114
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | spacy_alignments-0.8.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
d184096cb84945d35c1b0f718df1b2834ba59ad8b7dcacca4c81cbd67a44f461
|
|
BLAKE2b-256 checksum How to use checksums |
a7cbcd36b7bbf8e4fe88585af01649d0d84b3c54cda4248a8fae30e35f74ae27
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|
Release files / spacy_alignments-0.8.6-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | spacy_alignments-0.8.6-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 272.1 kB |
| Tags | CPython 3.6 CPython 3.6 pymalloc Linux glibc 2.17+ ARM64 |
|
SHA-256 checksum How to use checksums |
9bd481de1b8dc424d92557b1242489adc5933e0c55da46d3d9666c39e536f7fb
|
|
BLAKE2b-256 checksum How to use checksums |
97ff34d34597d82d2bf0ac67100fb2fa35107736462f7d600715e3d7f16d1b7d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.7.9
|