Skip to main content

A spaCy package for the Rust tokenizations library

Project description

spacy-alignments: Align tokenizations for spaCy + transformers

A spaCy package for Yohei Tamura's Rust tokenizations library with Python bindings.

Installation

pip install -U pip setuptools wheel
pip install spacy-alignments

If no binary wheel is available for your platform, you will need to install Rust in order to build spacy-alignments from source.

spacy-alignments vs. pytokenizations

The spacy_alignments module is a drop-in replacement for tokenizations:

import spacy_alignments as tokenizations
a2b, b2a = tokenizations.get_alignments(["å", "BC"], ["abc"])
assert a2b == [[0], [0]]
assert b2a == [[0, 1]]

The only difference between this package and the original pytokenizations is that it switches the build system to setuptools-rust to make it easier for us at Explosion to build source and binary packages for a wider range of platforms.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spacy-alignments-0.8.5.tar.gz (6.8 kB view details)

Uploaded Source

Built Distributions

spacy_alignments-0.8.5-cp310-cp310-win_amd64.whl (187.2 kB view details)

Uploaded CPython 3.10Windows x86-64

spacy_alignments-0.8.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

spacy_alignments-0.8.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ ARM64

spacy_alignments-0.8.5-cp310-cp310-macosx_11_0_arm64.whl (301.7 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

spacy_alignments-0.8.5-cp310-cp310-macosx_10_9_x86_64.whl (307.2 kB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

spacy_alignments-0.8.5-cp39-cp39-win_amd64.whl (187.2 kB view details)

Uploaded CPython 3.9Windows x86-64

spacy_alignments-0.8.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

spacy_alignments-0.8.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ ARM64

spacy_alignments-0.8.5-cp39-cp39-macosx_11_0_arm64.whl (302.2 kB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

spacy_alignments-0.8.5-cp39-cp39-macosx_10_9_x86_64.whl (307.2 kB view details)

Uploaded CPython 3.9macOS 10.9+ x86-64

spacy_alignments-0.8.5-cp38-cp38-win_amd64.whl (187.4 kB view details)

Uploaded CPython 3.8Windows x86-64

spacy_alignments-0.8.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.17+ x86-64

spacy_alignments-0.8.5-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.17+ ARM64

spacy_alignments-0.8.5-cp38-cp38-macosx_11_0_arm64.whl (302.2 kB view details)

Uploaded CPython 3.8macOS 11.0+ ARM64

spacy_alignments-0.8.5-cp38-cp38-macosx_10_9_x86_64.whl (307.3 kB view details)

Uploaded CPython 3.8macOS 10.9+ x86-64

spacy_alignments-0.8.5-cp37-cp37m-win_amd64.whl (187.2 kB view details)

Uploaded CPython 3.7mWindows x86-64

spacy_alignments-0.8.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.7mmanylinux: glibc 2.17+ x86-64

spacy_alignments-0.8.5-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded CPython 3.7mmanylinux: glibc 2.17+ ARM64

spacy_alignments-0.8.5-cp37-cp37m-macosx_10_9_x86_64.whl (307.3 kB view details)

Uploaded CPython 3.7mmacOS 10.9+ x86-64

spacy_alignments-0.8.5-cp36-cp36m-win_amd64.whl (187.2 kB view details)

Uploaded CPython 3.6mWindows x86-64

spacy_alignments-0.8.5-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.6mmanylinux: glibc 2.17+ x86-64

spacy_alignments-0.8.5-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded CPython 3.6mmanylinux: glibc 2.17+ ARM64

spacy_alignments-0.8.5-cp36-cp36m-macosx_10_9_x86_64.whl (305.1 kB view details)

Uploaded CPython 3.6mmacOS 10.9+ x86-64

File details

Details for the file spacy-alignments-0.8.5.tar.gz.

File metadata

  • Download URL: spacy-alignments-0.8.5.tar.gz
  • Upload date:
  • Size: 6.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.7.9

File hashes

Hashes for spacy-alignments-0.8.5.tar.gz
Algorithm Hash digest
SHA256 94ecb48f884ab8fa479d9929997281ebe0dc5b98fb8b5e53ebc252e88b023e21
MD5 aeb5d8823911fd0b343ee8d2cf4c3813
BLAKE2b-256 eea611f8d4d861ede695fd578662b657f6a908be0407f4b81fa8a79c1ce0d8ef

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 d33e3f02753e0febb469f7700750eb726bb008025efb6e553b6f86c02bc98e67
MD5 7900165d2ab7f85665f09afd9ff55735
BLAKE2b-256 194b5661e33a7eb6117c62b9ff69401813b92acf9dcea1fa3ebd77eff7bff1eb

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 095ac0b876ca823b1a44a4779c0de47d73d643601b934c4d996d79329440f40b
MD5 d2a4b813b3823bbd9d5c4bc505950349
BLAKE2b-256 3064320735b5b52497c246cd86e34eb6c7cacd2ba5da024434852fce32864124

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 3398d4f5b88020f3b7fbca7f6b8371d4c3a95132d7059e343700f65acaaf9975
MD5 d9dcb0c1b9e26e1e4d7eddc2b9671c51
BLAKE2b-256 1223a351ad900b2063dc98bc7d7eb74ee6cef4a3b0ccc24f06d8ed23b10afa9c

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4a0793ddd98ef840ac60be3583a096e0168c99f4604b01d3060ecd578f9920d2
MD5 3c71b83d6e727eccc1268d6e1a94530b
BLAKE2b-256 bc75728fe3abbac76ce0928604d18819df5729c171607d136d47f1709183bbfc

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 96d0df4bbd88a840ac136cb9b191e64fb21faa893afc6dd4dd7061242095f254
MD5 758745668b97a9de480f0009c6767e57
BLAKE2b-256 f0a5e5f9f9691c348b62ab7d881d777f1b1b1ae8b0b9d5087871043956b8a890

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp39-cp39-win_amd64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 176c0cc59da7789daf7a313c0bcbc61d1dde5539068a24c8aba728917e481a2d
MD5 9530a22ae57800cb41abe658d3a7ea4a
BLAKE2b-256 b68f1e46d3c434b932d9635ebb8ee4b7396eb7a2efa2d998b02c17f67ed4cdde

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b2c209d8aa4b0aa07ba4b98f51de679f2e415a1e069cc4caf1efb7bd0f6059ac
MD5 4f8a811a6e0ead1bdc00800399efd696
BLAKE2b-256 f5a0bbad67526412da6b90c45ec5e85643d216289f5f384c1378e2d7070e3379

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 cfc6e34709442257f49a268a536dd9b9cb6ae9200b438b7f85c8489a98e40ce5
MD5 4588bd30a5dc360604196a279e21734e
BLAKE2b-256 dc6290d311c7bd510ebad85bb4bb75535018823f55c2ef5f5b4eeb28dc0eff00

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 46956d142b1972dd992577aca5c1d32b49b6621768ec8f272e73b8c0dba805fd
MD5 583a491b6f6fdc41bb949a3e50df106c
BLAKE2b-256 6eb0d7354af2e18082eb13ee6580e37bb69572bbba2b9f77e1d9b7a296d477fe

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 24642819c1068215c07d1de036a22b00d136050bb6f9de4f898cfb18f4e4ef0f
MD5 801a8f7f70e96e52ae69f6ea03775268
BLAKE2b-256 5ab31c5f2cc07fcb9c505a4ed2c48c0166a35e78096e60694283ac3c709560c7

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp38-cp38-win_amd64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 9288d6ab901d183a9f4c5037a5a20477936d62d8222bd026a15c044bddd20b6f
MD5 203f7a6c16c99e02a4a37d980f86ea39
BLAKE2b-256 859fdcc83f849e43e9ff8bf2af5d5d631bb721c10b8179792b7cae8cef4e5ffb

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 64c6bde918394ed772a9c10997fe6e4c28dd8d093f51c94eeecaeed3c9fb3230
MD5 a454dd15d6a36c427aacd869294269bd
BLAKE2b-256 7a598e6b9694900db68117c61caa5f9bb73867af3fdb7539bea45d4e7f0b59e6

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 d94b79cb1b25be66f897fbfc668f51eac14329d04d566a4c9f0211210107ba1e
MD5 6a59f6b854db846e47bb4a5b4ba1519e
BLAKE2b-256 3f91dfc17ccc340e864b237470f90482f0781e16e26fdb8ea63f25fe88ef5827

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 efa040247a82f46dd285e8144b86d1b06e757543d757a5a98157516a36526df2
MD5 11729907180c7e6ae8ee5151ce2531a4
BLAKE2b-256 c4d994856b7de0243de45c2173c23c904b385eb24f5b7cf0568493413033872d

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 57a142679f609994fe5609aea1e0d779857277f116f71f34891c879d7cec8166
MD5 3a272833d7300f4e23a67c70496e4150
BLAKE2b-256 a6201c281587ded55362fd556f1ff9152e5192707d3ed101aac423978551a6e1

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp37-cp37m-win_amd64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 60a659fc8fdac78f28def69155ccabdf63ff7578e15d79b9a20821f96c9c4744
MD5 67d9c01526b89bab06362b6939ff693f
BLAKE2b-256 dad1338e60f168e5c50fc7f3fe24061d283ea4a855a9c3266ecc9e8c25c745e7

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a15cc4ec3251feaddf5ccd404e1691717594b05e8cd4957baca46f1ce6c3723f
MD5 88cf1263194c7b433070bad654d34ada
BLAKE2b-256 d250330bba31508483ca72b057bffbb8e03def43559e74a501b1cb2efb60ba36

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 78b3a17880d540f1478883a23615abcd26d65d0dd047eaebbc7434873ab5c016
MD5 d37732d53115961fe0e396e3ec9b4253
BLAKE2b-256 abd60bff7ca1ec5093429db803bfd2a60c22169dc413ecd097d629bf72007b48

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 6bf4c27615ddbb50d7920e870157400acdae15a0d5bd43694432dd668798b018
MD5 f750e0e1e5da49efbd1d1b6e1b5c0364
BLAKE2b-256 e28d81ee40f6847ebf76f61dcef80dab126934e33efdb0ce6a69bb78fcdb350b

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp36-cp36m-win_amd64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp36-cp36m-win_amd64.whl
Algorithm Hash digest
SHA256 4591d02a8fbf697fe3142d2b8278fcf28f7fb19c1a527a4876b281d7525ccea0
MD5 bbd1bc8c8b6ecef48f461b7e4edec410
BLAKE2b-256 09732e0c9c856a40395a1a2c9fad1ab5dfa4923073a1cc5d80ff9c2e1fcd5236

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c7eb900cdcbb4ce83f81e7b36fd6b4b05ce305b693f17e5b385ebfb9e859f473
MD5 71054286fb820bd171e7ea33fc16df9c
BLAKE2b-256 d6ae91c10d5a889526d2dca7f3e3e5562f64c34c05bf1154ab794b788bfd194c

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 0adf56d66ecaebbfaee0a906ffaa9dfe34f084aee9ee33166cd5f2d3bfa6e48f
MD5 b3f886a0910c9baa874f33112271a312
BLAKE2b-256 547d600f7ceb8fae203ac48f45cd3db916002f28e36b3aed2c5ad94e8a719c6c

See more details on using hashes here.

File details

Details for the file spacy_alignments-0.8.5-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for spacy_alignments-0.8.5-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 4f4ac27b1f2a5a4a5f3c43ee171fadf10a24220a5921846cd6b92514bd7f46e2
MD5 b2ded51b187e7dcadffddc8904d3189a
BLAKE2b-256 f3ee7b46694fa200f1a0d208a7362541ce52355e68c315704209b643382d3359

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page