A spaCy package for the Rust tokenizations library
Project description
spacy-alignments: Align tokenizations for spaCy + transformers
A spaCy package for Yohei Tamura's Rust tokenizations library with Python bindings.
Installation
pip install -U pip setuptools wheel
pip install spacy-alignments
If no binary wheel is available for your platform, you will need to install
Rust in order to build
spacy-alignments
from source.
spacy-alignments vs. pytokenizations
The spacy_alignments
module is a drop-in replacement for tokenizations
:
import spacy_alignments as tokenizations
a2b, b2a = tokenizations.get_alignments(["å", "BC"], ["abc"])
assert a2b == [[0], [0]]
assert b2a == [[0, 1]]
The only difference between this package and the original
pytokenizations
is that it
switches the build system to setuptools-rust
to make it easier for us at
Explosion to build source and binary packages for a wider range of platforms.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spacy-alignments-0.9.0.tar.gz
(6.8 kB
view hashes)
Built Distributions
Close
Hashes for spacy_alignments-0.9.0-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cc8d7da28d4b4ff8d12484f8878bd88adf3c8c4a28e748196829a7f9a4185cfe |
|
MD5 | e1396e75bc4f8499eceacff7b3c6f9dc |
|
BLAKE2b-256 | 8edd441216527310ca09f949d23aeccf1341287fc319f00b02322d215d9138c7 |
Close
Hashes for spacy_alignments-0.9.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c07371b825ca26f05959fbd00c630854c984afa48b45dfdd603cd36e319f2999 |
|
MD5 | dff8bd0f1e7791d546be0dbdcd656bcb |
|
BLAKE2b-256 | c53c8580d27d6f7be3078a11defe51dd4f776ddcefa259fe7919c5e99ce3366e |
Close
Hashes for spacy_alignments-0.9.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7b3980b3b49a8a059cbb40770ed17a5933d57e5b6408a5c878817ffac53a2306 |
|
MD5 | 78dbb8377eecd0bef5217a5fb73c7d97 |
|
BLAKE2b-256 | 9101ce1ca1efaba331989e024ba54f4c74c53cdb86f5513aa1cfc959b66caed7 |
Close
Hashes for spacy_alignments-0.9.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 008a1b63d4719ee6935985df0d57df4ad70af210aac84a0387f14f4ae84ee681 |
|
MD5 | 9d6d4ffbd998a11156e51a915bfcbfdb |
|
BLAKE2b-256 | a849b9452a5d6a3318aae7195e9398f91f545d0ef4e882ab5bbdd5942e6f6abf |
Close
Hashes for spacy_alignments-0.9.0-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9aba5ea11330670ad8416154110c7b82c18726d419e5a489ecaeee5a503cd107 |
|
MD5 | 6f045df750906240d36102ee0d6142b3 |
|
BLAKE2b-256 | 9f9a1eb41aeb6cf36fe98540ca9bead920a80b6292a47afef549a337d466c89e |
Close
Hashes for spacy_alignments-0.9.0-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c1a5be0a4b53b44d6f7aad255b3a2ae50ce5f5c20bcc2f299ce934db4c391591 |
|
MD5 | 817404edfe4f7494082d816e89167335 |
|
BLAKE2b-256 | 0995233ae136291ccc9204b06a2d28c659b6ed4561f1a98ca27c92b2b66cf381 |
Close
Hashes for spacy_alignments-0.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0dd006d7adb18b1a09024409cbd7e2ac9af2a2c76b9a89c34fbb2c80836f1f94 |
|
MD5 | 1cbeeced8b68a0aec4654e5e3446c40b |
|
BLAKE2b-256 | f0d54ef346f5f4292328fa68f9f85cd56bf71bc9455294cb19fda539f0297ecb |
Close
Hashes for spacy_alignments-0.9.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 25890bb155edf1af684572776670a1c54dcfa9c4b6f7cd61191dec552c98dda6 |
|
MD5 | e808a2fbad40de9d7b0950a3b38d04a0 |
|
BLAKE2b-256 | 9c4aeb364b1d51adef93f26d1ae5bae9a60a288f4920673cf7f256e9d22e7ca7 |
Close
Hashes for spacy_alignments-0.9.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7107300f13bf48f84cbcb89d8156215503f9e9b9f556cda525a6b45a1a13877f |
|
MD5 | ae4ff633a7e7347c6aee0f58cc08ae1a |
|
BLAKE2b-256 | f99c254d9003111123f7256bd37246dcf4f2bcc3faec97d48666d65557c1115a |
Close
Hashes for spacy_alignments-0.9.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec473cada82bb26d097a200f20317199fe6e7595aa9922bb506f7fffc6b4bfc5 |
|
MD5 | 13da09ffdd75a64f9b7ba7bf19e0a588 |
|
BLAKE2b-256 | 915e0cdb6c3c3d4e775a842bf6e842e9f374f6c064d856674ef772a02305d7e3 |
Close
Hashes for spacy_alignments-0.9.0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8ad3270373574d67775bc11fb6e171a92228fb3469f40f4688a07fd62e533bcf |
|
MD5 | 66f8eda52ac85096a8f5d09f7215f344 |
|
BLAKE2b-256 | cf9591282b5d1c7390e1d4bedc9595e002a78c4e1f446023df53b09ff9f83d77 |
Close
Hashes for spacy_alignments-0.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e0dbec39ca1c4902c6be5f36f95390d896701e4b616d0b92548f0763a0b15b0 |
|
MD5 | 30a3d0075e140b71ec1e3f6177846c0a |
|
BLAKE2b-256 | a5578270084bf9e2def924120100ac4d93667cb1652fa34d20217d21b6d7b51e |
Close
Hashes for spacy_alignments-0.9.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 81c6d9145a7a5d627d98d7b7c75bab344d3e28c29cc522e5b88557c35a870f14 |
|
MD5 | c9927a411059071332a43ddf9b3f326d |
|
BLAKE2b-256 | d29065214ed550e67c2457918bb56b2b9de35a27392fcd0931c9764f3a7db5a3 |
Close
Hashes for spacy_alignments-0.9.0-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ccf1389d22f47217f0f57ba88be966552d6befdc9f8047f2ec9dc442933781c |
|
MD5 | 9e4e35a4ca2ce76a1d1cfa2edd85d3bf |
|
BLAKE2b-256 | c8969cc52fcd72f2840d3b39bad178858c7328ebfff2bc1fa08a3c900f02c682 |
Close
Hashes for spacy_alignments-0.9.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4b11e56eb9a6d723db791d16f11d617d2f9c3f42c044a001cd7fb09d11e57032 |
|
MD5 | f4dfbf676f2736074f45ecacbc91ee21 |
|
BLAKE2b-256 | 0269c6984c984c826bf1d30a0985e4d36dabf0bc84c702a8206c761c4702b483 |
Close
Hashes for spacy_alignments-0.9.0-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aad1b2e94e837ebfce200db862e1b7a5352620af0bb5b990a41b320e59dce23b |
|
MD5 | 4343023188508554585e0db15a6be2f8 |
|
BLAKE2b-256 | cbce28416862ababc429390e3aef3c6159ed61c306d796ad28d2aa76a6da7887 |
Close
Hashes for spacy_alignments-0.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 05bc58df82d310395e980e19f74fd9149061131eb2520889391486ea189d0a33 |
|
MD5 | 8e59ea6ef08ad9e0e09d6feb77d1c131 |
|
BLAKE2b-256 | b33519eb8ebfe45ecb8159b515f940c6b5cd2c8028cdacc07cbe3ad7169a2014 |
Close
Hashes for spacy_alignments-0.9.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 86943f81ee3dc9ebaa07d745b996b5f2432a69ea1c85fff209c772cf541354c8 |
|
MD5 | 067e2c9682c9e405f77d695df6b886bd |
|
BLAKE2b-256 | b9b69a848daf9f04da0f93fa45bec8bd9dc3b341cc14cf58cf368336f29ce28d |
Close
Hashes for spacy_alignments-0.9.0-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f8f866bc51a56a1e287d1a983fb52a3b55c395a3c4f24896d193773977c65b7 |
|
MD5 | ea4565d3c24a7287c590a88e49db7856 |
|
BLAKE2b-256 | 8adef43c36b6c17ab4906860935a156452b43787fba700c6e2e5bdd8267d1b59 |
Close
Hashes for spacy_alignments-0.9.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16aed72624e1d65c69bb0b183e72fc75dde60c590aa94ca310d6067815b94c86 |
|
MD5 | 838860f3057535684e679c03293bb212 |
|
BLAKE2b-256 | 3fbcf6d8bb213f83625263e3261e01a08b1944f771bfedb1553f28e0dd2269dd |
Close
Hashes for spacy_alignments-0.9.0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5d1ef9cd1426ebcf4dcbb220ff746db72a26778c4ac07ecd3981c44470904675 |
|
MD5 | 9baa35945ce058f7ebd9752b372953b0 |
|
BLAKE2b-256 | 3a4794fd4230ae9e869ac4873b0b27a402c93f09244d42f8bcac5343eadd26f3 |
Close
Hashes for spacy_alignments-0.9.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef1558c8dcfe08c58a4a7043ea082aa8f159f5f23d661bbc4c0d3f2bfa24bd86 |
|
MD5 | 503bce4a87956b6b7e5a828cba0b8509 |
|
BLAKE2b-256 | f93d25d4952a4823aced3d5331f7496457e2dbbb81c22d892d6a6e9862ca8c01 |
Close
Hashes for spacy_alignments-0.9.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd6a45db3c96c59c137af0d2d1aa1ca15caae5fe640004b2113940b4caa977fc |
|
MD5 | 7454d890ae8e821569374ebd662c9226 |
|
BLAKE2b-256 | a4e242c4968db003c6a5c20b1c56ba37a17a3514dfc4303097d0795bda1b32dc |
Close
Hashes for spacy_alignments-0.9.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55a135860c8cceac85c54c86b26b4133bf1cf93a4b526dfccc3fd8e4ee54e25a |
|
MD5 | 12c37ce8538ad63840b33a99fa036daa |
|
BLAKE2b-256 | 47df59d12f553116142960a63ff90d68554361885d39beda0b17f21ef8175f57 |