Skip to main content

Fast and customizable text tokenization library with BPE and SentencePiece support

Project description

pyonmttok

pyonmttok is the Python wrapper for OpenNMT/Tokenizer, a fast and customizable text tokenization library with BPE and SentencePiece support.

Installation:

pip install pyonmttok

Requirements:

  • OS: Linux, macOS, Windows
  • Python version: >= 3.6
  • pip version: >= 19.3

Table of contents

  1. Tokenization
  2. Subword learning
  3. Vocabulary
  4. Token API
  5. Utilities

Tokenization

Example

>>> import pyonmtok
>>> tokenizer = pyonmttok.Tokenizer("aggressive", joiner_annotate=True)
>>> tokens = tokenizer("Hello World!")
>>> tokens
['Hello', 'World', '■!']
>>> tokenizer.detokenize(tokens)
'Hello World!'

Interface

Constructor

tokenizer = pyonmttok.Tokenizer(
    mode: str,
    *,
    lang: Optional[str] = None,
    bpe_model_path: Optional[str] = None,
    bpe_dropout: float = 0,
    vocabulary: Optional[List[str]] = None,
    vocabulary_path: Optional[str] = None,
    vocabulary_threshold: int = 0,
    sp_model_path: Optional[str] = None,
    sp_nbest_size: int = 0,
    sp_alpha: float = 0.1,
    joiner: str = "■",
    joiner_annotate: bool = False,
    joiner_new: bool = False,
    support_prior_joiners: bool = False,
    spacer_annotate: bool = False,
    spacer_new: bool = False,
    case_feature: bool = False,
    case_markup: bool = False,
    soft_case_regions: bool = False,
    no_substitution: bool = False,
    with_separators: bool = False,
    allow_isolated_marks: bool = False,
    preserve_placeholders: bool = False,
    preserve_segmented_tokens: bool = False,
    segment_case: bool = False,
    segment_numbers: bool = False,
    segment_alphabet_change: bool = False,
    segment_alphabet: Optional[List[str]] = None,
)

# SentencePiece-compatible tokenizer.
tokenizer = pyonmttok.SentencePieceTokenizer(
    model_path: str,
    vocabulary_path: Optional[str] = None,
    vocabulary_threshold: int = 0,
    nbest_size: int = 0,
    alpha: float = 0.1,
)

# Copy constructor.
tokenizer = pyonmttok.Tokenizer(tokenizer: pyonmttok.Tokenizer)

# Return the tokenization options (excluding options related to subword).
tokenizer.options

See the documentation for a description of each tokenization option.

Tokenization

# Tokenize a text.
# When training=False, subword regularization such as BPE dropout is disabled.
tokenizer.__call__(text: str, training: bool = True) -> List[str]

# Tokenize a text and return optional features.
# When as_token_objects=True, the method returns Token objects (see below).
tokenizer.tokenize(
    text: str,
    as_token_objects: bool = False,
    training: bool = True,
) -> Union[Tuple[List[str], Optional[List[List[str]]]], List[pyonmttok.Token]]

# Tokenize a batch of text.
tokenizer.tokenize_batch(
    batch_text: List[str],
    as_token_objects: bool = False,
    training: bool = True,
) -> Union[Tuple[List[List[str]], List[Optional[List[List[str]]]]], List[List[pyonmttok.Token]]]

# Tokenize a file.
tokenizer.tokenize_file(
    input_path: str,
    output_path: str,
    num_threads: int = 1,
    verbose: bool = False,
    training: bool = True,
    tokens_delimiter: str = " ",
)

Detokenization

# The detokenize method converts a list of tokens back to a string.
tokenizer.detokenize(
    tokens: List[str],
    features: Optional[List[List[str]]] = None,
) -> str
tokenizer.detokenize(tokens: List[pyonmttok.Token]) -> str

# The detokenize_with_ranges method also returns a dictionary mapping a token
# index to a range in the detokenized text.
# Set merge_ranges=True to merge consecutive ranges, e.g. subwords of the same
# token in case of subword tokenization.
# Set unicode_ranges=True to return ranges over Unicode characters instead of bytes.
tokenizer.detokenize_with_ranges(
    tokens: Union[List[str], List[pyonmttok.Token]],
    merge_ranges: bool = False,
    unicode_ranges: bool = False,
) -> Tuple[str, Dict[int, Tuple[int, int]]]

# Detokenize a file.
tokenizer.detokenize_file(
    input_path: str,
    output_path: str,
    tokens_delimiter: str = " ",
)

Subword learning

Example

The Python wrapper supports BPE and SentencePiece subword learning through a common interface:

1. Create the subword learner with the tokenization you want to apply, e.g.:

# BPE is trained and applied on the tokenization output before joiner (or spacer) annotations.
tokenizer = pyonmttok.Tokenizer("aggressive", joiner_annotate=True, segment_numbers=True)
learner = pyonmttok.BPELearner(tokenizer=tokenizer, symbols=32000)

# SentencePiece can learn from raw sentences so a tokenizer in not required.
learner = pyonmttok.SentencePieceLearner(vocab_size=32000, character_coverage=0.98)

2. Feed some raw data:

# Feed detokenized sentences:
learner.ingest("Hello world!")
learner.ingest("How are you?")

# or detokenized text files:
learner.ingest_file("/data/train1.en")
learner.ingest_file("/data/train2.en")

3. Start the learning process:

tokenizer = learner.learn("/data/model-32k")

The returned tokenizer instance can be used to apply subword tokenization on new data.

Interface

# See https://github.com/rsennrich/subword-nmt/blob/master/subword_nmt/learn_bpe.py
# for argument documentation.
learner = pyonmttok.BPELearner(
    tokenizer: Optional[pyonmttok.Tokenizer] = None,  # Defaults to tokenization mode "space".
    symbols: int = 10000,
    min_frequency: int = 2,
    total_symbols: bool = False,
)

# See https://github.com/google/sentencepiece/blob/master/src/spm_train_main.cc
# for available training options.
learner = pyonmttok.SentencePieceLearner(
    tokenizer: Optional[pyonmttok.Tokenizer] = None,  # Defaults to tokenization mode "none".
    keep_vocab: bool = False,  # Keep the generated vocabulary (model_path will act like model_prefix in spm_train)
    **training_options,
)

learner.ingest(text: str)
learner.ingest_file(path: str)
learner.ingest_token(token: Union[str, pyonmttok.Token])

learner.learn(model_path: str, verbose: bool = False) -> pyonmttok.Tokenizer

Vocabulary

Example

tokenizer = pyonmttok.Tokenizer("aggressive", joiner_annotate=True)

with open("train.txt") as train_file:
    vocab = pyonmttok.build_vocab_from_lines(
        train_file,
        tokenizer=tokenizer,
        maximum_size=32000,
        special_tokens=["<blank>", "<unk>", "<s>", "</s>"],
    )

with open("vocab.txt", "w") as vocab_file:
    for token in vocab.ids_to_tokens:
        vocab_file.write("%s\n" % token)

Interface

# Special tokens are added with ids 0, 1, etc., and are never removed by a resize.
vocab = pyonmttok.Vocab(special_tokens: Optional[List[str]] = None)

# Read-only properties.
vocab.tokens_to_ids -> Dict[str, int]
vocab.ids_to_tokens -> List[str]
vocab.counters -> List[int]

# Get or set the ID returned for out-of-vocabulary tokens.
# By default, it is the ID of the token <unk> if present in the vocabulary, len(vocab) otherwise.
vocab.default_id -> int

vocab.lookup_token(token: str) -> int
vocab.lookup_index(index: int) -> str

# Calls lookup_token on a batch of tokens.
vocab.__call__(tokens: List[str]) -> List[int]

vocab.__len__() -> int                  # Implements: len(vocab)
vocab.__contains__(token: str) -> bool  # Implements: "hello" in vocab
vocab.__getitem__(token: str) -> int    # Implements: vocab["hello"]

# Add tokens to the vocabulary after tokenization.
# If a tokenizer is not set, the text is split on spaces.
vocab.add_from_text(text: str, tokenizer: Optional[pyonmttok.Tokenizer] = None) -> None
vocab.add_from_file(path: str, tokenizer: Optional[pyonmttok.Tokenizer] = None) -> None
vocab.add_token(token: str, count: int = 1) -> None

vocab.resize(maximum_size: int = 0, minimum_frequency: int = 1) -> None


# Build a vocabulary from an iterator of lines.
# If a tokenizer is not set, the lines are split on spaces.
pyonmttok.build_vocab_from_lines(
    lines: Iterable[str],
    tokenizer: Optional[pyonmttok.Tokenizer] = None,
    maximum_size: int = 0,
    minimum_frequency: int = 1,
    special_tokens: Optional[List[str]] = None,
) -> pyonmttok.Vocab

# Build a vocabulary from an iterator of tokens.
pyonmttok.build_vocab_from_tokens(
    tokens: Iterable[str],
    maximum_size: int = 0,
    minimum_frequency: int = 1,
    special_tokens: Optional[List[str]] = None,
) -> pyonmttok.Vocab

Token API

The Token API allows to tokenize text into pyonmttok.Token objects. This API can be useful to apply some logics at the token level but still retain enough information to write the tokenization on disk or detokenize.

Example

>>> tokenizer = pyonmttok.Tokenizer("aggressive", joiner_annotate=True)
>>> tokens = tokenizer.tokenize("Hello World!", as_token_objects=True)
>>> tokens
[Token('Hello'), Token('World'), Token('!', join_left=True)]
>>> tokens[-1].surface
'!'
>>> tokenizer.serialize_tokens(tokens)[0]
['Hello', 'World', '■!']
>>> tokens[-1].surface = '.'
>>> tokenizer.serialize_tokens(tokens)[0]
['Hello', 'World', '■.']
>>> tokenizer.detokenize(tokens)
'Hello World.'

Interface

The pyonmttok.Token class has the following attributes:

  • surface: a string, the token value
  • type: a pyonmttok.TokenType value, the type of the token
  • join_left: a boolean, whether the token should be joined to the token on the left or not
  • join_right: a boolean, whether the token should be joined to the token on the right or not
  • preserve: a boolean, whether joiners and spacers can be attached to this token or not
  • features: a list of string, the features attached to the token
  • spacer: a boolean, whether the token is prefixed by a SentencePiece spacer or not (only set when using SentencePiece)
  • casing: a pyonmttok.Casing value, the casing of the token (only set when tokenizing with case_feature or case_markup)

The pyonmttok.TokenType enumeration is used to identify tokens that were split by a subword tokenization. The enumeration has the following values:

  • TokenType.WORD
  • TokenType.LEADING_SUBWORD
  • TokenType.TRAILING_SUBWORD

The pyonmttok.Casing enumeration is used to identify the original casing of a token that was lowercased by the case_feature or case_markup tokenization options. The enumeration has the following values:

  • Casing.LOWERCASE
  • Casing.UPPERCASE
  • Casing.MIXED
  • Casing.CAPITALIZED
  • Casing.NONE

The Tokenizer instances provide methods to serialize or deserialize Token objects:

# Serialize Token objects to strings that can be saved on disk.
tokenizer.serialize_tokens(
    tokens: List[pyonmttok.Token],
) -> Tuple[List[str], Optional[List[List[str]]]]

# Deserialize strings into Token objects.
tokenizer.deserialize_tokens(
    tokens: List[str],
    features: Optional[List[List[str]]] = None,
) -> List[pyonmttok.Token]

Utilities

Interface

# Returns True if the string has the placeholder format.
pyonmttok.is_placeholder(token: str)

# Sets the random seed for reproducible tokenization.
pyonmttok.set_random_seed(seed: int)

# Checks if the language code is valid.
pyonmttok.is_valid_language(lang: str).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

pyonmttok-1.37.1-cp311-cp311-win_amd64.whl (14.4 MB view details)

Uploaded CPython 3.11 Windows x86-64

pyonmttok-1.37.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

pyonmttok-1.37.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (16.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

pyonmttok-1.37.1-cp311-cp311-macosx_11_0_arm64.whl (14.3 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

pyonmttok-1.37.1-cp311-cp311-macosx_10_9_x86_64.whl (14.6 MB view details)

Uploaded CPython 3.11 macOS 10.9+ x86-64

pyonmttok-1.37.1-cp310-cp310-win_amd64.whl (14.4 MB view details)

Uploaded CPython 3.10 Windows x86-64

pyonmttok-1.37.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

pyonmttok-1.37.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (16.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

pyonmttok-1.37.1-cp310-cp310-macosx_11_0_arm64.whl (14.3 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

pyonmttok-1.37.1-cp310-cp310-macosx_10_9_x86_64.whl (14.6 MB view details)

Uploaded CPython 3.10 macOS 10.9+ x86-64

pyonmttok-1.37.1-cp39-cp39-win_amd64.whl (14.4 MB view details)

Uploaded CPython 3.9 Windows x86-64

pyonmttok-1.37.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

pyonmttok-1.37.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (16.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

pyonmttok-1.37.1-cp39-cp39-macosx_11_0_arm64.whl (14.3 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

pyonmttok-1.37.1-cp39-cp39-macosx_10_9_x86_64.whl (14.6 MB view details)

Uploaded CPython 3.9 macOS 10.9+ x86-64

pyonmttok-1.37.1-cp38-cp38-win_amd64.whl (14.4 MB view details)

Uploaded CPython 3.8 Windows x86-64

pyonmttok-1.37.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

pyonmttok-1.37.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (16.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

pyonmttok-1.37.1-cp38-cp38-macosx_11_0_arm64.whl (14.3 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

pyonmttok-1.37.1-cp38-cp38-macosx_10_9_x86_64.whl (14.6 MB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

pyonmttok-1.37.1-cp37-cp37m-win_amd64.whl (14.4 MB view details)

Uploaded CPython 3.7m Windows x86-64

pyonmttok-1.37.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

pyonmttok-1.37.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (17.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

pyonmttok-1.37.1-cp37-cp37m-macosx_10_9_x86_64.whl (14.6 MB view details)

Uploaded CPython 3.7m macOS 10.9+ x86-64

pyonmttok-1.37.1-cp36-cp36m-win_amd64.whl (14.4 MB view details)

Uploaded CPython 3.6m Windows x86-64

pyonmttok-1.37.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

pyonmttok-1.37.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (17.1 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ ARM64

pyonmttok-1.37.1-cp36-cp36m-macosx_10_9_x86_64.whl (14.6 MB view details)

Uploaded CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file pyonmttok-1.37.1-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 731653a4aa727e2ec066a1741f54805c913aa9038c06d8f7b7d62f16717b0a0b
MD5 8f5f6c83af94b5028993ac2a44baf6e3
BLAKE2b-256 aac20f02bb3654a7be2cd581deb92c3ebe0151827e80c66224dd905c96dbcfa4

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3ec8f8eedb82c059cff10cc9ac74747e2b61e607cdf7744a2e1c7d0bb2d170c9
MD5 5693ef1ad11d3095b5208592eaf1b6ed
BLAKE2b-256 0b88812ba76801b20262ad1533508c1c6f171828ac2085c9b02b6a8aa2dc734f

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 cee8793c6ccec83ad102b354b07d79e95ed9c003f4cd06607a1bda8ab43ef979
MD5 72af3624313eefeb94ff4e9ec76ad3c3
BLAKE2b-256 4bde7dce44985bc087dd5063a4ade2f659c6d94ede5348af1ca4833889728de9

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d548287d5f739582c95d0166750d322259de405ffd605b7943d0e4f6693d4043
MD5 da0af7c803dafb2adfbd7736ac288bb0
BLAKE2b-256 654b0eaf8420c9d96b25029964fac2e0024b0d66ce9130211a2f44f260b71573

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 1e012d69bbf59ed4cb961a906d0fb4f0e29ff98b4d9f7003ffc30ae8dafa782c
MD5 0875f07d2ae3216be51b7c75c3b75f17
BLAKE2b-256 5041e5cf9ea258c2a2033d4d80a16f20afebc022b327f545b9a0431b7d99183a

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 8fb09da1bee4ef170f9b6f5c5c71ee666d7614953203a8c02c4fe85924ac18c8
MD5 0986bb6bf6920de9d153f30d623b68c7
BLAKE2b-256 8b1fb55d1fd3311d525e4b3edda0cc583db51f2363c5d967a01e1d85456aec2f

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0cdfc6cebbf4921fcfcb4e76388142c777c025d014bd79d9a7858bba54b04129
MD5 e46441c2f4538f3dfb00a6ad270f421f
BLAKE2b-256 0991bc1a1e7d38913b0eca17d2dcef3606ef2f65d57a4971ad64af8607c78aaf

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 171aa91ec1b7013bf0e9db3508f9d04c5a6e6a2cf1703bdf8d038b4773fa114e
MD5 594c2d3fef4993af16159288f269d098
BLAKE2b-256 a32a6ff3e8918716aeff92ac55eb636d187cb2217c14980bbe9311f877e8fec2

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 854c5d194569d98611bdcc105a95d8d8894411a62b0f69ed2df0d84c236464d0
MD5 beacd90777c3267a21cdd6f7979f21c6
BLAKE2b-256 0890cc89e3039c1f495dc1f61e06e734f5ac421e7e7c0eaa0d4f3f78489cd6d2

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 0d916b180f0db1bfd556fda5c34aa8bde30665cb5fe38cbb11da288f6283f766
MD5 5b889c24f2001736f65421ff26df8b7a
BLAKE2b-256 8b450d655a5a5ea384a808dbb88f82d18e74a5715c58ce3696d66e51e431d0e4

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp39-cp39-win_amd64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 a250e6c71cf627587812524eb2421c68a50fa7aae57b15fef18d72476e1b0e25
MD5 5a1610293d67293ad378d2ce16665c95
BLAKE2b-256 f7b65e3154cc0e6c97d245c62179d5531ad5b9eafe7ef8c8d277ab121c24dce3

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 57f2b23791f8818e80e0e1f75434cd7f2660f58915de043dded2783ad56365d5
MD5 f7a38d157773ec4d3494ba9d4912dccf
BLAKE2b-256 6854bbfc29ae433dc6bf575d2084e9a039cfc67d8cdc48e353135d676d48e9c6

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 358df957106ba94b0afdcb90090d10e8a6753c2d3da05c5c3554491037db9713
MD5 563c6745ec57d1535e43a62366e787d3
BLAKE2b-256 b01d193f7b88f93ac1ec25ad6e6b7ec981cd4237c0d5f960a09ae0386ebe3a81

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 71e7617d5fc8b7fda3f38789d32b941952d4db1335c9da72876122139c69b970
MD5 41f88e91a46928fc051f62c906fb08ff
BLAKE2b-256 f9f27551e162f28d7ca8fcb4a36d24bbb18c06d50b6e48cadc8eacae5a3578ad

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 aa8a6d537c9a06ff49acbf5c09e0842e6f7ab473b605e4f70689ff56372f9705
MD5 44ea08c47d1ab2d72ed477e2d4af6f7f
BLAKE2b-256 d4012b666901480b030032512ed04e91e0d8d28459e16b1e898b681aa91dd1e9

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp38-cp38-win_amd64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 f807d28167c4927a0c19b0b5febf1c986ee330288218cd44ed7e0384d2112ef4
MD5 6361331317abfb71b6a153b4111e1eb7
BLAKE2b-256 a845bd9126b4b7e7f03c16fd261cc5a725ae6ff7bc7d983556fea56838ee6172

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3acdb6382d2479cbfe0d312d7b0470a17f3c99f1402cdcbc16ec72ec359f7c2b
MD5 86c78779ee9b238751ab72ff525c73ce
BLAKE2b-256 0a7fcc4cae9aaf220b6ebb39bf5c253da0948b492860ce0b71413682d911d82b

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 5d32d44d56402f1a8ee3009d5c2c26ea369e7f6b0072424e29d1805f0b38bbf5
MD5 132adf053a988304a47e369656a35577
BLAKE2b-256 c12531695e43e94ab7eb9cdd059811f87f3a3094f6935e37df1bbaad4824cdce

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f86aff8b66b898194c5d6db21fb73a61a7c6ebf49dec9a00f6ce087bfc3c83a4
MD5 fd5f3f231b687a6388a014a6b87e2fda
BLAKE2b-256 de82feb8658c1e16f287a8d4112d31e1db334da340d8b0a8fa5a867de4e7e5f0

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 2c89826f697bcd167337438a7a4c3e36c8352f17d6199acec34529c50df4cf5a
MD5 7c39318c7ee362ba938aaf582c796598
BLAKE2b-256 a31e8743002b17a136418d82ea525bfe81c186f537230c87bf8939b35c3c6104

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp37-cp37m-win_amd64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 2061489e56f40210ae0264d9a7a531a8af1251201255dcce8d55908241ecc145
MD5 c2c3eb519ba5c289f87a202428d9f599
BLAKE2b-256 32b72432dc9d8a67dc3e4a4b1516af469be60bf5ef17200ff49fc053db213b5b

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 02c68712a22d64850bf1ae1a2bec3aa0238de6a80d01e817bffa4d7565d8968d
MD5 7a690f475b31b0816e5b3315a6b7dce8
BLAKE2b-256 feb76aca7053cce0ff9623387005e39cf8c2893379666884ecff7cc4aee5cd53

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 d741c7c4193a9b949ad907b976c808c35c294b33936fcecd42a0e1f178977303
MD5 8ea1681b349278e8bf4f40fb1a66246b
BLAKE2b-256 5a9e14439794f40b2664073b1b45ed7065a32c6a11e7ea544984a93f7a27fa25

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 fc3adf282f5bdc0202a0dcfe0e83c1ed66f322df6f05743d7c680f03b47560f0
MD5 e53f4d5d8a7df88409803b0819768d8c
BLAKE2b-256 98648f35129ac011c2afc466a9e137b6242c6e88d8b8fc769039b2ac9c9aca83

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp36-cp36m-win_amd64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp36-cp36m-win_amd64.whl
Algorithm Hash digest
SHA256 3b6da287d4133241e9aa6106e9d6f0a1cb6889811dcbc96071bf17d6019be547
MD5 6c3eeaacb3dbcb967c15ba484d876761
BLAKE2b-256 f27c613e5615416cbc223f979611f2218782ece3045c9931bf6d614e73695e06

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e34d38d2ca6d8795eccf6ef21fbf90a5218e87a26f9335f20821f4712db33441
MD5 07e68bfbc8bdff9b7b3aa5c22667a8c4
BLAKE2b-256 6008bfeacb0a33534b238cb7a2be5ed5340e1fc0df4be94c54931e2efd86a391

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b43e0793bbd4706d2658d75d5609231b8e3cf9be4c059b3df22f9f351564a1ae
MD5 a889e6f32f62f36c0e9f02910b78f456
BLAKE2b-256 bbed4a1032bebda964bada34d933fadf3ca91ad1cc6116fe6230d511884b2938

See more details on using hashes here.

File details

Details for the file pyonmttok-1.37.1-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for pyonmttok-1.37.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 399009259b64ec68d2791698eb7add1d7c8188d524b245ff2385f0ccb0f58323
MD5 5c2e3f0a5d7f00d72705918ebe69f496
BLAKE2b-256 140f4ef3a10979032d050f73ed6b6f9f0918812455084443e2486d93ef297e15

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page