A spaCy package for the Rust tokenizations library
Project description
spacy-alignments: Align tokenizations for spaCy + transformers
A spaCy package for Yohei Tamura's Rust tokenizations library with Python bindings.
Installation
pip install -U pip setuptools wheel
pip install spacy-alignments
If no binary wheel is available for your platform, you will need to install
Rust in order to build
spacy-alignments
from source.
spacy-alignments vs. pytokenizations
The spacy_alignments
module is a drop-in replacement for tokenizations
:
import spacy_alignments as tokenizations
a2b, b2a = tokenizations.get_alignments(["å", "BC"], ["abc"])
assert a2b == [[0], [0]]
assert b2a == [[0, 1]]
The only difference between this package and the original
pytokenizations
is that it
switches the build system to setuptools-rust
to make it easier for us at
Explosion to build source and binary packages for a wider range of platforms.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spacy-alignments-0.8.4.tar.gz
(6.9 kB
view hashes)
Built Distributions
Close
Hashes for spacy_alignments-0.8.4-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b5cd5beb15dd34d72304b5d07dbedd0676c4cbb018b2ad1994ee8560260810e |
|
MD5 | 6e9ddb3b532513102fd9b67ed0069ab9 |
|
BLAKE2b-256 | e01df9806cd6249cf04176c788387b6518fec67a73d3ed23aadaf2dd3fc13cd4 |
Close
Hashes for spacy_alignments-0.8.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32400cc9eff8e4ea9d61d3f6974df979a2c272ca8e9cd65e2811e0aa1b4598ad |
|
MD5 | 84fd446a3dbb19bf11e0b037f18c85cf |
|
BLAKE2b-256 | b982e6bfa8f383938325e7d1f90e0873e7fb770b768de37058b9b1958629f897 |
Close
Hashes for spacy_alignments-0.8.4-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fde0b20a9c629b07ce7bd3ccb31d1b2ddd465faee1a6f5c09ed1ef9cf07b8609 |
|
MD5 | 8864cafc3b4d5751925b79bea894eb5b |
|
BLAKE2b-256 | a6476205f947e7439489363586e35bd5321779849a4faa8a0a52ff5c6ecde4f5 |
Close
Hashes for spacy_alignments-0.8.4-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e43ca54c61ab97fc98017c41f6b3b0e016d195a2bfbb3f449cb77d552a842798 |
|
MD5 | 7c6e5b0ffc12b5618569fa4241415805 |
|
BLAKE2b-256 | 525173dd22dc6ef07c7fe5b1b8c2381db13a6b15f05b89cc20261148dfc7ce31 |
Close
Hashes for spacy_alignments-0.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 27f0816db4426dfc4954d506c8855618b6589274edce658b84813d83a2f6ed45 |
|
MD5 | beb212f1ee2018be87c7f34677cf7e96 |
|
BLAKE2b-256 | c0e5605cc8c249160b06ad645513d3cb6aa9645e1dfc796ef964d0fa4c0bb43a |
Close
Hashes for spacy_alignments-0.8.4-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7111b2bca1fcd8e61ef8a508e5b25d1ca4a819063ae61b6ceb50074a8c917848 |
|
MD5 | 80e239c458338c4308b1c03d7c26dd73 |
|
BLAKE2b-256 | f70926f8b344430100ff2d0d8eec594bb794f38db72b9dfc78c612cc7d86824c |
Close
Hashes for spacy_alignments-0.8.4-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3eb2f35cee5d7cbc5c5b156a0a97379537186911bd9768fbb262b7e6818137a3 |
|
MD5 | 735059755c6fdd06ddfbc9051908b84a |
|
BLAKE2b-256 | 78092dffc5395983d3950db16bfcaa8dd645f4399f94cb77e96fbf67be088ee3 |
Close
Hashes for spacy_alignments-0.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 43b11507bb058cb29da454cbe8bb2727dcd3fc83398bcb6bf6bc28da5f1b0bae |
|
MD5 | 46120dd73c598703a77b880ccaae8417 |
|
BLAKE2b-256 | 665b7ff4855a27f59721d1d64c5bf1518f2e9e2f52e9ac4a24bab8f7cd6080f4 |
Close
Hashes for spacy_alignments-0.8.4-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 09ca14a8a43022f067521bb976c3a406edaff7e45ac2c61692a4a84e45d7c630 |
|
MD5 | 1e662d2b02bb3a3058d43fa7bdb573f5 |
|
BLAKE2b-256 | 5cc0435ee960c1fa899ed531dd396ade5882f2cf1847cb01e47d230763a72cf7 |
Close
Hashes for spacy_alignments-0.8.4-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 382f487c74533540ff9f82b6a7369d04b74dec6a3a2ddfc9d6f835eddc40b5f0 |
|
MD5 | 7460217d6a6e0bd8912d3322a03e8bd0 |
|
BLAKE2b-256 | 6fefea94624324a4861e155291c215b286a4daecaff0df96faa1f5af674d097e |
Close
Hashes for spacy_alignments-0.8.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 306c2827479ff3b2c377fd8e3cd20c5e9dac2a2d6e6034e62cae117994b56685 |
|
MD5 | c90cbd39dd811e9800eee113134a6be5 |
|
BLAKE2b-256 | 72aeac87fe358846f4e7254b25be3c20904f80832ab7f206fd331cefd28badce |
Close
Hashes for spacy_alignments-0.8.4-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 327ee5e7c6dc82a0ce915b79f431d3119b74b2e94faa0a0059723e072c915e6d |
|
MD5 | 6243fce71d3e90fa6f524ab6585cbce0 |
|
BLAKE2b-256 | 25542083f44330ffdb80913e0e62b5d11baa98287a912228d41eb8af26cd728d |
Close
Hashes for spacy_alignments-0.8.4-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3d0a6c55f130d542efb9c7fccec7674b2ee91c695a1c960e06431614fb7ed7aa |
|
MD5 | 3109749d9beca5c7713d81e93c796727 |
|
BLAKE2b-256 | 9e5af9ccf75be9e7bdf4b54f945a1317bf0c00544e525f30440688b87b2730fe |
Close
Hashes for spacy_alignments-0.8.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b962d9e72e67203f5d8528d723fcdd467df1935e34ab2a94f720bf96f06526ce |
|
MD5 | 0da3d549c40523389a1170e1ec34c9d3 |
|
BLAKE2b-256 | e8e8f8e2cedf9b1f7b6182a3fc19ba2049005374075dd58a40a53d2d352be8f4 |
Close
Hashes for spacy_alignments-0.8.4-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f38834d4f7d161a1f2e32511fa7262b07c79c51ed1c0255bd9ffcf355ed0c111 |
|
MD5 | 921514b3431464b899cdf7717a98eb37 |
|
BLAKE2b-256 | 92f05bb84b8eac10441fb7d35ce89cf8b2f7ddd9196a247c3bab49149ded2d12 |