A Language Processing Utility
Project description
LPU
LPU
is a collection of utility classes/functions for language processing
Installation
Installing from PyPI release
$ pip install --user lpu
Installing from GitHub master
$ pip install --user https://github.com/akivajp/lpu/archive/master.zip
Modules
modules in lpu
package
lpu.common.config
inheritable and serializable configurutation classes, useful for calling functions with many arguments
lpu.common.environ
handling environment variables with stacks, useful for changing/reverting global settings, such as debugging modes
lpu.common.files
utility functions for file handling, including transparent file access of gzip files
lpu.common.logging
enhanced logging objects (from standard logging library) with colorizing features and operations with environment variables
lpu.common.progress
utility classes and functions for progress reporting (as known as progress bars), working with file-type objects and iterators
lpu.smt.align.ibm_models
utility classes to train and estimate word alignemt based on IBM models
Commands
LPU package also includes directly executable commands
commands in lpu
package
lpu-abspath
$ lpu-abspath [-h] filepath [filepath ...]
get absolute paths of given files or directories
lpu-clean-parallel
$ lpu-clean-parallel [-h] [--min min_length] [--max max_length] \
[--ratio ratio] [--target-directory directory_path] [--escape] \
[--normalize] filepath [filepath ...] output_tag
lpu-dialog
$ lpu-dialog [-h] [--exist filepath] [--continue] [--yes] [--no]
Show message on condition, wait and receive user's response
lpu-exec-parallel
$ lpu-exec-parallel [-h] [--input filepath] [--output filepath] \
[--splitsize num_lines] [--chunks num_files] [--threads num_threads] \
[--tmpdir directory_path] [--verbose] [--interval seconds] command
Execute command in multiple processes by splitting the targe file
lpu-guess-langcode
$ lpu-guess-langcode [-h] filepath [filepath ...]
Guess the language codes from given files
lpu-progress
$ lpu-progress [-h] [--lines] [--refresh seconds] [--header string] \
[filepath filepath ...]]
Show the progress of pipe I/O
lpu-random-split
$ lpu-random-split [-h] --input filepath [filepath ...] \
[--prefixes prefix [prefix ...]] [--suffixes suffix [suffix ...]] \
--tags tag [tag ...] --split-sizes size [size ...] [--ignore-empty] \
[--quiet] [--debug] [--random-seed seed] [--ids [suffix]]
lpu-wait-files
$ lpu-wait-files [-h] [--quiet] [--debug] [--delay seconds] \
[--interval seconds] [--timeout seconds] filepath [filepath ...]
Wait until file will be found
lpu-word-align-train
$ lpu-word-align-train [-h] [--save-sores filepath] [--decode-align filepath] \
[--iteration-limit num_iterations] [--threshold min_probability] \
[--nbest integer] [--character] [--debug] [--quiet] \
src_path trg_path save_trans_path [save_align_path]
lpu-word-align-score
$ lpu-word-align-score [-h] [--save-scores filepath] \
[--decode-align filepath] [--character] [--debug] [--quiet] \
src_path trg_path trans_path [align_path]
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
File details
Details for the file lpu-0.2.10.tar.gz
.
File metadata
- Download URL: lpu-0.2.10.tar.gz
- Upload date:
- Size: 2.2 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a413c4c3bf3c6c76fc9406a47f6fe38038c9b8c55f65727a668e8e24d8c05194 |
|
MD5 | b934326c60c545bfe1f922da0ba45d7a |
|
BLAKE2b-256 | 67e72a810e12f9c2d4a0f8edf0d95f9dcee4a091f4abd8ebb3b377c9e9d0afa2 |
File details
Details for the file lpu-0.2.10-cp37-cp37m-manylinux1_x86_64.whl
.
File metadata
- Download URL: lpu-0.2.10-cp37-cp37m-manylinux1_x86_64.whl
- Upload date:
- Size: 7.7 MB
- Tags: CPython 3.7m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c53691f921fb2315b0f359f3066c8214b10e1ea208a322e9032315dbf2170fa |
|
MD5 | 2575f3c0c6ca7d01570bbbeb609c50f2 |
|
BLAKE2b-256 | 7ec38f8785eb3a33b661ca3ef8d6941b446fdfca0f5a2c18e578e3799b405452 |
File details
Details for the file lpu-0.2.10-cp37-cp37m-manylinux1_i686.whl
.
File metadata
- Download URL: lpu-0.2.10-cp37-cp37m-manylinux1_i686.whl
- Upload date:
- Size: 7.1 MB
- Tags: CPython 3.7m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6a42f8b777599bb44eb62778dde4e98c95d214d0f13fa08c988e840c56a4e1ae |
|
MD5 | f84216dcf5d69a021db3554409ffd281 |
|
BLAKE2b-256 | 7361760cd096f6d3bde13de6a044d83cc2f3e6edefaf649c460f12e274d6071f |
File details
Details for the file lpu-0.2.10-cp37-cp37m-macosx_10_15_x86_64.whl
.
File metadata
- Download URL: lpu-0.2.10-cp37-cp37m-macosx_10_15_x86_64.whl
- Upload date:
- Size: 2.1 MB
- Tags: CPython 3.7m, macOS 10.15+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 336fecc9d6119f64adea07c78fe96552bade69a55876d463ea82e5f9a6da7e7c |
|
MD5 | 06056bd11c4b7d05c5d6f8fd9b597089 |
|
BLAKE2b-256 | fde8797b4685bdf543754292c18941ab06e78fcde85a909265c20a2e701f0063 |
File details
Details for the file lpu-0.2.10-cp36-cp36m-manylinux1_x86_64.whl
.
File metadata
- Download URL: lpu-0.2.10-cp36-cp36m-manylinux1_x86_64.whl
- Upload date:
- Size: 7.8 MB
- Tags: CPython 3.6m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 630da24c28ee7f7ec21698bddaabd6005ba6000b34d08f673c0e2096b1a24b52 |
|
MD5 | 085584a438df7802f25bc95b1301dee8 |
|
BLAKE2b-256 | 0110a10bbf402a1715d3fd0d7637445a4937aa20572212a6c6bfbd5c21a3325f |
File details
Details for the file lpu-0.2.10-cp36-cp36m-manylinux1_i686.whl
.
File metadata
- Download URL: lpu-0.2.10-cp36-cp36m-manylinux1_i686.whl
- Upload date:
- Size: 7.2 MB
- Tags: CPython 3.6m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8ee2be8b848ea8651dd28b05532a8a1303ddf757fd287e99a0fa8952b1c443d4 |
|
MD5 | 1efa346fda5b69fbc3d296c847c0405f |
|
BLAKE2b-256 | 76fe0251cabf8b25e33b4c14bf39f6f8f47e6093195769d3f15a2094b99755ab |
File details
Details for the file lpu-0.2.10-cp36-cp36m-macosx_10_14_x86_64.whl
.
File metadata
- Download URL: lpu-0.2.10-cp36-cp36m-macosx_10_14_x86_64.whl
- Upload date:
- Size: 2.2 MB
- Tags: CPython 3.6m, macOS 10.14+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ad9f08b440f892b197dd5144a856f2f0642c1b6cd452b12249e6a1fd060ade35 |
|
MD5 | 0c7305a0678cada08ae7f5d669120962 |
|
BLAKE2b-256 | 84acb5736b4d074173afce6608a4cc9156802fda3b9b7281c860982f93b4d383 |
File details
Details for the file lpu-0.2.10-cp35-cp35m-manylinux1_x86_64.whl
.
File metadata
- Download URL: lpu-0.2.10-cp35-cp35m-manylinux1_x86_64.whl
- Upload date:
- Size: 7.4 MB
- Tags: CPython 3.5m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb7afa7d74427c9e2b2211459b55304fe981d6a2e630ff9fdff355388d35194b |
|
MD5 | 8617247d4261c6367964faf2c97bc414 |
|
BLAKE2b-256 | 316ff0f1e53d4bded8cbe608aa91a78066f11b24b9488c935ea89209a7690ab7 |
File details
Details for the file lpu-0.2.10-cp35-cp35m-manylinux1_i686.whl
.
File metadata
- Download URL: lpu-0.2.10-cp35-cp35m-manylinux1_i686.whl
- Upload date:
- Size: 6.8 MB
- Tags: CPython 3.5m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | eca8392a477384b656b5586dd56bfd9f2b8106ce1666014053e0ca647a55fc87 |
|
MD5 | 83fa0deba3ef71eaec8036ed2c42633b |
|
BLAKE2b-256 | 3a560ab49616a6d952cbadb8928624e229fd3344ccfd8762616293e9ad9b9cb1 |