A spaCy package for the Rust tokenizations library
Project description
spacy-alignments: Align tokenizations for spaCy + transformers
A spaCy package for Yohei Tamura's Rust tokenizations library with Python bindings.
Installation
pip install -U pip setuptools wheel
pip install spacy-alignments
If no binary wheel is available for your platform, you will need to install
Rust in order to build
spacy-alignments
from source.
spacy-alignments vs. pytokenizations
The spacy_alignments
module is a drop-in replacement for tokenizations
:
import spacy_alignments as tokenizations
a2b, b2a = tokenizations.get_alignments(["å", "BC"], ["abc"])
assert a2b == [[0], [0]]
assert b2a == [[0, 1]]
The only difference between this package and the original
pytokenizations
is that it
switches the build system to setuptools-rust
to make it easier for us at
Explosion to build source and binary packages for a wider range of platforms.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spacy-alignments-0.8.6.tar.gz
(6.8 kB
view hashes)
Built Distributions
Close
Hashes for spacy_alignments-0.8.6-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6bcdfa708c1b81ee5b40cd672cdae3e1f7e67118b25e1a3e1870b0b3214512c |
|
MD5 | ecf6cda76173a629210812043022bd3b |
|
BLAKE2b-256 | da98652bf629af2ea0c85072dead895f80279d9dfce3f805db11eb65b4d1a3b2 |
Close
Hashes for spacy_alignments-0.8.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 381dd5a7f76ad22e8d4ec075926389fffc3f163e205219ec016d8088b9a3af0c |
|
MD5 | 93e4d9f24882aa5075a452aac143bf6b |
|
BLAKE2b-256 | 42b219ce7f17e707528a7d3e76d7db619761ad728c48eee380f760a98ca03b3b |
Close
Hashes for spacy_alignments-0.8.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd584e778c6122c6ba70c0ed2190ac7e9edfc6d10148c6c94f99ba9e1d58b60b |
|
MD5 | 407077dff428dc1e001a50bea8161606 |
|
BLAKE2b-256 | ed080c49f892d0c6b1285d7e23c8c7fe1340264d6e965aff344dbfbb0c9fd500 |
Close
Hashes for spacy_alignments-0.8.6-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2752e9a9d93784fa939f03ffa2431dcfa8fb0794387470e59ea070b74980c00 |
|
MD5 | dd16e7ab463df153651acbe0ad09e47b |
|
BLAKE2b-256 | c2764337bd8b6acc4fbe174205ea984dd9e0f603e94c3676d5a861adc01bd477 |
Close
Hashes for spacy_alignments-0.8.6-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d34b4b7f4950e21855af296d0c464a6873e15405efcd64d27d674ff1ccd147fe |
|
MD5 | 890aafeb5439e0b9a6da05eef1eeb1e4 |
|
BLAKE2b-256 | 4fe00db91098aa03cf775641c35e9b09727b4b285de2c2f2f46850ca86146bdb |
Close
Hashes for spacy_alignments-0.8.6-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a0bf32e965ea03ffe974d43209e0c576d05365c2fec524e651a3a5ee235f82a6 |
|
MD5 | 6a6731a4463fbc6613ceb92015583a42 |
|
BLAKE2b-256 | a0293318a8002341552a80c35c7119e97b4fbd6ea66191d2bb3f5246b1ccb74e |
Close
Hashes for spacy_alignments-0.8.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5110c61b02f2cf248a133019340a79757c1aef4ab5d6b2e7296aa40b0f4f8b4b |
|
MD5 | c56c1c411c7807e36ed8e888f43c0b91 |
|
BLAKE2b-256 | 490572acfdcc758a2e7b4cc93111ab691d18ce89ce4319c7a925447143f83e5c |
Close
Hashes for spacy_alignments-0.8.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4dd037a4c2f7f18599f1fee50d61fcc6cac554583079aabadd31c50bef67a0a6 |
|
MD5 | 64c95966c00afc2785a6f7b28c72ff2e |
|
BLAKE2b-256 | 30da975e376666c191baeb149806234478f7e74b9ae59fceb1cc83d640f667b2 |
Close
Hashes for spacy_alignments-0.8.6-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8e23ae7d35755e305f3a29bcec9c857e1019eaa14cdbe1b916d0bd91d7c174f9 |
|
MD5 | b1d2bd1de3a9ce0097cbc999d7c20ead |
|
BLAKE2b-256 | 20c2102b1f93254d3efc75ec5b63fb236dfd33d6b0ab6e08388fad3108f5179e |
Close
Hashes for spacy_alignments-0.8.6-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4e20c5a63c26703f154f27d789717fccb5cc5ce9ce87fdbfac15c1e48c5f72f |
|
MD5 | 6526f3480ee47ee81dfd26f9b6c5dc6c |
|
BLAKE2b-256 | d9a9c1d82a88f86ac0553b0a0dc7814d9adfe5cccfd8f4e5d43bb03780a60387 |
Close
Hashes for spacy_alignments-0.8.6-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c379e794cd22a0680709f5eee5008e96a986369f99d58e1422e3dadbe6bb963a |
|
MD5 | 94d6e5bb98283674e3c81cebc47e151b |
|
BLAKE2b-256 | b9d6ee4533de19e4435fcd59d7d955dda599ed5b4811fa2cf6d9806b4456bb39 |
Close
Hashes for spacy_alignments-0.8.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fdd7fb3bf13630ee42fefe54a68f870485d4aab946bce25814e93aae3ba08b5d |
|
MD5 | 268c8cd07b781af7a9396ff2d3afd41c |
|
BLAKE2b-256 | 477e3c59e28175223c9a5e5b0720815df0145065400dff98601a05c1969703b2 |
Close
Hashes for spacy_alignments-0.8.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce31abe26232210039e0ccff9ea8ac81ce9dfac2d8653519d611fb1d2a1429b6 |
|
MD5 | 2781b7f482298f7adfd342b51b4f5197 |
|
BLAKE2b-256 | a5860f89b231a71a05f85293f3b48f286545c1b47a4f4532c7ccd2428d9bb3c5 |
Close
Hashes for spacy_alignments-0.8.6-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0861622b2c29cfdb62e98d1f5adfc218f960babd0dc86370d6c9a218d7563242 |
|
MD5 | bc5667583cd5cdc76d100bac62e54ca5 |
|
BLAKE2b-256 | 153ac523a2d5e4f98108903c182118b4d90054f380b901a4f6de3c7aae7e2f42 |
Close
Hashes for spacy_alignments-0.8.6-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0ecff8612cc87adc7cd783cc1bd636262e380372055bbc3b3ae6f468bd8c4faa |
|
MD5 | 9dc22a0cb46533dcbb5ef39f45ab0b25 |
|
BLAKE2b-256 | 0143c47e3b1676781102cc0d5a6bdc761ec9cdbb21320aa4d11e2da8c5457f5a |
Close
Hashes for spacy_alignments-0.8.6-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24965aba440355e5ee8f08b04c73774f92fbfae86b8da45a89561f2bf52abafc |
|
MD5 | 703519b03d49899026f3928945190559 |
|
BLAKE2b-256 | 2d90b189fe48e19b06ff7fb9c81a31cc923a5ae905077f0a11e852490cc164cc |
Close
Hashes for spacy_alignments-0.8.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cf6f9e4f04bca91f21896d7d50961562a74fb23b23ee449f0ba4d29d6455db2b |
|
MD5 | b0f42397955bafc1433a197bbdd4af6f |
|
BLAKE2b-256 | 0889b7219de36c2d42f45c14d5943d31781989687b4c0f8d0bf9f68944424a75 |
Close
Hashes for spacy_alignments-0.8.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7856462e1ca2ccafa572e0a9fc5b36228adcc3ea13734f2ba16234a99bba47d8 |
|
MD5 | 00810498547a72c108dd54a678b4631f |
|
BLAKE2b-256 | 317520da36b3bab7d782582edbe7c6d147dd6cf9a4b2cbc880184291d0b31634 |
Close
Hashes for spacy_alignments-0.8.6-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2172e4b3a7654446cdee52edaac2423d06b83fad11e80af2279a078a0ec983b1 |
|
MD5 | 313034e086bfea26ab1d49e6779d601f |
|
BLAKE2b-256 | daa70266946dd29dd1d9197300cb9d2c865180a73419a06af12ce19b1885dcef |
Close
Hashes for spacy_alignments-0.8.6-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bf8da9e1f16491142c402f8ec44c02b71b8291eae631ccba75be313ab852ab3e |
|
MD5 | 0a47d95594a952724e63242679bf92e7 |
|
BLAKE2b-256 | e55c1482120ab0fdea3efce8cb3d7e4e992dc35c95c94e2dbea4856ae0a4d3f7 |
Close
Hashes for spacy_alignments-0.8.6-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 41bc3951da7c7565ed792cfb7c2cee97efd8b4c8fd83b61d84ce31715f999d2d |
|
MD5 | 51361fbefe44f22746d6bf4c491e3789 |
|
BLAKE2b-256 | 0ca80835d62da22cb8fe3ad74e5a24808383322a76d5082555b9fa3832aaf16b |
Close
Hashes for spacy_alignments-0.8.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7a419950cab37fe84123b4b859ca6de49fe259f88ac20347c7d7c30094ad3c1c |
|
MD5 | baefb1b985bfecd8a10c3e3a99d9c5de |
|
BLAKE2b-256 | 665fb093550d14e20e7b7660df7a5d8058fa4188fa01a728b827d01f50e504b2 |
Close
Hashes for spacy_alignments-0.8.6-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c0cc271073a93f71e884ce6735a5469765e150d8d304208cadcc8463771a0eab |
|
MD5 | 15df6c3c7506199d50d01a4362ab783f |
|
BLAKE2b-256 | 64f6ff57996ea6d4a507dbaabf603c10bc8ff72536839cc556686323d1d36222 |
Close
Hashes for spacy_alignments-0.8.6-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ab9874e01c043f5c2b06452b3b629eb61d937c674919a17f9f5c39dcc9554768 |
|
MD5 | 593daf9bf3a7f5d61acd8546a0960fde |
|
BLAKE2b-256 | a40b0384ac598f41c8fff2ff54354b87f957b7e14cfa1febaf9398ea8137ea8d |
Close
Hashes for spacy_alignments-0.8.6-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6037d0b8448d81d7cbac17f5abdfb47dd1dd5dbee61ae0f9db64181606dd5bb8 |
|
MD5 | a70b7f20b62c65d08ae91d1794d9cafe |
|
BLAKE2b-256 | cdc69147b8aef2cf38997f11ae63184ebb2c2c7ca9efaad340c9421ac936e114 |
Close
Hashes for spacy_alignments-0.8.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d184096cb84945d35c1b0f718df1b2834ba59ad8b7dcacca4c81cbd67a44f461 |
|
MD5 | 74a80e19a487a48a9482aa6de53eba59 |
|
BLAKE2b-256 | a7cbcd36b7bbf8e4fe88585af01649d0d84b3c54cda4248a8fae30e35f74ae27 |
Close
Hashes for spacy_alignments-0.8.6-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9bd481de1b8dc424d92557b1242489adc5933e0c55da46d3d9666c39e536f7fb |
|
MD5 | f12b3f5196fd09e3ece54ccfcd202675 |
|
BLAKE2b-256 | 97ff34d34597d82d2bf0ac67100fb2fa35107736462f7d600715e3d7f16d1b7d |