Skip to main content

tokenizesentences

Python3 module to tokenize english sentences. Based on the answer of D Greenberg in StackOverflow: https://stackoverflow.com/questions/4576077/python-split-text-on-sentences

Installation

Install with pip

pip3 install -U tokenizesentences

Usage

In [1]: import tokenizesentences

In [2]: m = tokenizesentences.SplitIntoSentences()

In [3]: m.split_into_sentences(
    "Mr. John Johnson Jr. was born in the U.S.A but earned his Ph.D. in Israel before joining Nike Inc. as an engineer. He also worked at craigslist.org as a business analyst."
    )

Out[3]: 
[
    'Mr. John Johnson Jr. was born in the U.S.A but earned his Ph.D. in Israel before joining Nike Inc. as an engineer.',
    'He also worked at craigslist.org as a business analyst.'
]

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizesentences-0.1.tar.gz (2.4 kB view details)

Uploaded Source

File details

Details for the file tokenizesentences-0.1.tar.gz.

File metadata

  • Download URL: tokenizesentences-0.1.tar.gz
  • Upload date:
  • Size: 2.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

File hashes

Hashes for tokenizesentences-0.1.tar.gz
Algorithm Hash digest
SHA256 7484a3068e86a9736fffad1dd212d241385aa4fe72139a88f369e277ebc6a8b6
MD5 36de88c1a2c1c2267eb4d80b10c1b5a6
BLAKE2b-256 635b61d586db39da9c12545c4c0a64a4370d306d40ae38fd9d895e0f759aede5

See more details on using hashes here.

Release history Release notifications | RSS feed

0.5.1

2 files

0.5.0

2 files

0.3

2 files

0.2

1 file

This release

0.1 This release

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page