Skip to main content

A Language Processing Utility

Project description

LPU

version python license

LPU is a collection of utility classes/functions for language processing

Installation

Installing from PyPI release

$ pip install --user lpu

Installing from GitHub master

$ pip install --user https://github.com/akivajp/lpu/archive/master.zip

Modules

modules in lpu package

lpu.common.config

inheritable and serializable configurutation classes, useful for calling functions with many arguments

lpu.common.environ

handling environment variables with stacks, useful for changing/reverting global settings, such as debugging modes

lpu.common.files

utility functions for file handling, including transparent file access of gzip files

lpu.common.logging

enhanced logging objects (from standard logging library) with colorizing features and operations with environment variables

lpu.common.progress

utility classes and functions for progress reporting (as known as progress bars), working with file-type objects and iterators

lpu.smt.align.ibm_models

utility classes to train and estimate word alignemt based on IBM models

Commands

LPU package also includes directly executable commands

commands in lpu package

lpu-abspath

  $ lpu-abspath [-h] filepath [filepath ...]

get absolute paths of given files or directories

lpu-clean-parallel

  $ lpu-clean-parallel [-h] [--min min_length] [--max max_length] \
      [--ratio ratio] [--target-directory directory_path] [--escape] \
      [--normalize] filepath [filepath ...] output_tag

lpu-dialog

  $ lpu-dialog [-h] [--exist filepath] [--continue] [--yes] [--no]

Show message on condition, wait and receive user's response

lpu-exec-parallel

  $ lpu-exec-parallel [-h] [--input filepath] [--output filepath] \
      [--splitsize num_lines] [--chunks num_files] [--threads num_threads] \
      [--tmpdir directory_path] [--verbose] [--interval seconds] command

Execute command in multiple processes by splitting the targe file

lpu-guess-langcode

  $ lpu-guess-langcode [-h] filepath [filepath ...]

Guess the language codes from given files

lpu-progress

  $ lpu-progress [-h] [--lines] [--refresh seconds] [--header string] \
      [filepath filepath ...]]

Show the progress of pipe I/O

lpu-random-split

  $ lpu-random-split [-h] --input filepath [filepath ...] \
      [--prefixes prefix [prefix ...]] [--suffixes suffix [suffix ...]] \
      --tags tag [tag ...] --split-sizes size [size ...] [--ignore-empty] \
      [--quiet] [--debug] [--random-seed seed] [--ids [suffix]]

lpu-wait-files

  $ lpu-wait-files [-h] [--quiet] [--debug] [--delay seconds] \
      [--interval seconds] [--timeout seconds] filepath [filepath ...]

Wait until file will be found

lpu-word-align-train

  $ lpu-word-align-train [-h] [--save-sores filepath] [--decode-align filepath] \
      [--iteration-limit num_iterations] [--threshold min_probability] \
      [--nbest integer] [--character] [--debug] [--quiet] \
      src_path trg_path save_trans_path [save_align_path]

lpu-word-align-score

  $ lpu-word-align-score [-h] [--save-scores filepath] \
      [--decode-align filepath] [--character] [--debug] [--quiet] \
      src_path trg_path trans_path [align_path]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lpu-0.2.10.tar.gz (2.2 MB view details)

Uploaded Source

Built Distributions

lpu-0.2.10-cp37-cp37m-manylinux1_x86_64.whl (7.7 MB view details)

Uploaded CPython 3.7m

lpu-0.2.10-cp37-cp37m-manylinux1_i686.whl (7.1 MB view details)

Uploaded CPython 3.7m

lpu-0.2.10-cp37-cp37m-macosx_10_15_x86_64.whl (2.1 MB view details)

Uploaded CPython 3.7m macOS 10.15+ x86-64

lpu-0.2.10-cp36-cp36m-manylinux1_x86_64.whl (7.8 MB view details)

Uploaded CPython 3.6m

lpu-0.2.10-cp36-cp36m-manylinux1_i686.whl (7.2 MB view details)

Uploaded CPython 3.6m

lpu-0.2.10-cp36-cp36m-macosx_10_14_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m macOS 10.14+ x86-64

lpu-0.2.10-cp35-cp35m-manylinux1_x86_64.whl (7.4 MB view details)

Uploaded CPython 3.5m

lpu-0.2.10-cp35-cp35m-manylinux1_i686.whl (6.8 MB view details)

Uploaded CPython 3.5m

File details

Details for the file lpu-0.2.10.tar.gz.

File metadata

  • Download URL: lpu-0.2.10.tar.gz
  • Upload date:
  • Size: 2.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10.tar.gz
Algorithm Hash digest
SHA256 a413c4c3bf3c6c76fc9406a47f6fe38038c9b8c55f65727a668e8e24d8c05194
MD5 b934326c60c545bfe1f922da0ba45d7a
BLAKE2b-256 67e72a810e12f9c2d4a0f8edf0d95f9dcee4a091f4abd8ebb3b377c9e9d0afa2

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

  • Download URL: lpu-0.2.10-cp37-cp37m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 7.7 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 9c53691f921fb2315b0f359f3066c8214b10e1ea208a322e9032315dbf2170fa
MD5 2575f3c0c6ca7d01570bbbeb609c50f2
BLAKE2b-256 7ec38f8785eb3a33b661ca3ef8d6941b446fdfca0f5a2c18e578e3799b405452

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp37-cp37m-manylinux1_i686.whl.

File metadata

  • Download URL: lpu-0.2.10-cp37-cp37m-manylinux1_i686.whl
  • Upload date:
  • Size: 7.1 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp37-cp37m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 6a42f8b777599bb44eb62778dde4e98c95d214d0f13fa08c988e840c56a4e1ae
MD5 f84216dcf5d69a021db3554409ffd281
BLAKE2b-256 7361760cd096f6d3bde13de6a044d83cc2f3e6edefaf649c460f12e274d6071f

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp37-cp37m-macosx_10_15_x86_64.whl.

File metadata

  • Download URL: lpu-0.2.10-cp37-cp37m-macosx_10_15_x86_64.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: CPython 3.7m, macOS 10.15+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp37-cp37m-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 336fecc9d6119f64adea07c78fe96552bade69a55876d463ea82e5f9a6da7e7c
MD5 06056bd11c4b7d05c5d6f8fd9b597089
BLAKE2b-256 fde8797b4685bdf543754292c18941ab06e78fcde85a909265c20a2e701f0063

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

  • Download URL: lpu-0.2.10-cp36-cp36m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 7.8 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 630da24c28ee7f7ec21698bddaabd6005ba6000b34d08f673c0e2096b1a24b52
MD5 085584a438df7802f25bc95b1301dee8
BLAKE2b-256 0110a10bbf402a1715d3fd0d7637445a4937aa20572212a6c6bfbd5c21a3325f

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp36-cp36m-manylinux1_i686.whl.

File metadata

  • Download URL: lpu-0.2.10-cp36-cp36m-manylinux1_i686.whl
  • Upload date:
  • Size: 7.2 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp36-cp36m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 8ee2be8b848ea8651dd28b05532a8a1303ddf757fd287e99a0fa8952b1c443d4
MD5 1efa346fda5b69fbc3d296c847c0405f
BLAKE2b-256 76fe0251cabf8b25e33b4c14bf39f6f8f47e6093195769d3f15a2094b99755ab

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp36-cp36m-macosx_10_14_x86_64.whl.

File metadata

  • Download URL: lpu-0.2.10-cp36-cp36m-macosx_10_14_x86_64.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: CPython 3.6m, macOS 10.14+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp36-cp36m-macosx_10_14_x86_64.whl
Algorithm Hash digest
SHA256 ad9f08b440f892b197dd5144a856f2f0642c1b6cd452b12249e6a1fd060ade35
MD5 0c7305a0678cada08ae7f5d669120962
BLAKE2b-256 84acb5736b4d074173afce6608a4cc9156802fda3b9b7281c860982f93b4d383

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

  • Download URL: lpu-0.2.10-cp35-cp35m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 7.4 MB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 cb7afa7d74427c9e2b2211459b55304fe981d6a2e630ff9fdff355388d35194b
MD5 8617247d4261c6367964faf2c97bc414
BLAKE2b-256 316ff0f1e53d4bded8cbe608aa91a78066f11b24b9488c935ea89209a7690ab7

See more details on using hashes here.

File details

Details for the file lpu-0.2.10-cp35-cp35m-manylinux1_i686.whl.

File metadata

  • Download URL: lpu-0.2.10-cp35-cp35m-manylinux1_i686.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.9

File hashes

Hashes for lpu-0.2.10-cp35-cp35m-manylinux1_i686.whl
Algorithm Hash digest
SHA256 eca8392a477384b656b5586dd56bfd9f2b8106ce1666014053e0ca647a55fc87
MD5 83fa0deba3ef71eaec8036ed2c42633b
BLAKE2b-256 3a560ab49616a6d952cbadb8928624e229fd3344ccfd8762616293e9ad9b9cb1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page