Skip to main content

Sentence Segmentation with sequece tagging

Project description

DeepSegment: A sentence segmenter that actually works!

For the original implementation please use the "master" branch of this repo.

This implementation is trained on data from various sources. (v1 or the model in master branch is trained only on Tatoeba data).

The Demo is available at http://bpraneeth.com/projects

Installation:

pip install --upgrade deepsegment
# please install tensorflow or tensorflow-gpu separately. Tested with tf and tf-gpu versions 1.8 to 2.0

Usage:

from deepsegment import DeepSegment
# The default language is 'en'
segmenter = DeepSegment('en')
segmenter.segment('I am Batman i live in gotham')
# ['I am Batman', 'i live in gotham']

To Do:

  1. Add a sliding window for processing very long texts.
  2. Publish docker tf-serving image and deepsegment-client.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

deepsegment-2.0.0.tar.gz (3.9 kB view hashes)

Uploaded Source

Built Distribution

deepsegment-2.0.0-py2.py3-none-any.whl (15.6 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page