Skip to main content

Sentence splitting & tokenization

Project description

Obeliks - sentence splitting & tokenization

Instalation

Install from PyPI:

pip install obeliks

Usage

Command line:

obeliks -h

Command line parameters:

-if <name*> read input from one or more files
-sif        read input from list of files, specified via stdin
-o <name>   write output to file <name>
-tei        produce XML-TEI output
-c		    produce CoNLL-U output
-d		    pass "newdoc id" to output (implies -c)

Usage examples:

obeliks "To je stavek." "Tudi to je stavek."
echo -e "To je stavek.\nTudi to je stavek." | obeliks
obeliks "To je stavek." "Tudi to je stavek." -o output.txt
echo -e "To je stavek.\nTudi to je stavek." | obeliks > output.txt
obeliks -if text*.txt
cat text*.txt | obeliks

As a Python module:

import obeliks

text = 'Pozdravljen, svet!'

# Store results to string
output = obeliks.run(text, conllu=True)

# Write result to file
obeliks.run(text, out_file='tei.txt', tei=True)

# Write to stdout
obeliks.run(text, to_stdout=True, conllu=True)

# Read input from file(s)
output = obeliks.run(in_file='in.txt', conllu=True)
output = obeliks.run(in_files=['in1.txt', 'in2.txt'], tei=True)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for obeliks, version 1.1.1
Filename, size File type Python version Upload date Hashes
Filename, size obeliks-1.1.1.tar.gz (15.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page