Skip to main content

XML cleaner

Word and sentence tokenization in Python.

[![PyPI version](https://badge.fury.io/py/xml-cleaner.svg)](https://badge.fury.io/py/xml-cleaner) [![Build Status](https://travis-ci.org/JonathanRaiman/xml_cleaner.svg?branch=master)](https://travis-ci.org/JonathanRaiman/xml_cleaner) ![Jonathan Raiman, author](https://img.shields.io/badge/Author-Jonathan%20Raiman%20-blue.svg)

[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE.md)

Usage

Use this package to split up strings according to sentence and word boundaries. For instance, to simply break up strings into tokens:

` tokenize("Joey was a great sailor.") #=> ["Joey ", "was ", "a ", "great ", "sailor ", "."] `

To also detect sentence boundaries:

` sent_tokenize("Cat sat mat. Cat's named Cool.", keep_whitespace=True) #=> [["Cat ", "sat ", "mat", ". "], ["Cat ", "'s ", "named ", "Cool", "."]] `

sent_tokenize can keep the whitespace as-is with the flags keep_whitespace=True and normalize_ascii=False.

Installation

` pip3 install xml_cleaner `

Testing

Run nose2.

Metadata

Release files for xml-cleaner 2.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xml-cleaner 2.0.4
File Size Uploaded Provenance Actions
xml-cleaner-2.0.4.tar.gz 10.8 kB Without provenance Details for xml-cleaner-2.0.4.tar.gz

Release files / xml-cleaner-2.0.4.tar.gz

Download URL xml-cleaner-2.0.4.tar.gz
Size 10.8 kB
Tags Source
SHA-256 checksum
How to use checksums
7b79fbab3d9b9a57dbd5b3b6d7457f576722e2e20f42653b4939de6065fbb90d
BLAKE2b-256 checksum
How to use checksums
c16b859b2f7730a32e049db35b3b6e9640c6b55c1cd4195c19fe5e8a063b4066
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for xml-cleaner 2.0.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page