knlm

Modified Kneser-Ney smoothing language model module for Python

Installation

$ pip install knlm
$ pip3 install knlm

Example

from knlm import KneserNey

mode = 'build'
if mode == 'build':
    # build model from corpus text. order = 3, word size = 4 byte
    mdl = KneserNey(3, 4)
    for line in open('corpus.txt', encoding='utf-8'):
        mdl.train(line.lower().strip().split())
    mdl.optimize()
    mdl.save('language.model')
else:
    # load model from binary file
    mdl = KneserNey.load('language.model')
    print('Loaded')
print('Order: %d, Vocab Size: %d, Vocab Width: %d' % (mdl.order, mdl.vocabs, mdl._wsize))

# evaluate sentence score
print(mdl.evaluateSent('I love kiwi .'.split()))
print(mdl.evaluateSent('ego kiwi amo .'.split()))

# evaluate scores for each word
print(mdl.evaluateEachWord('I love kiwi .'.split()))
print(mdl.evaluateEachWord('ego kiwi amo .'.split()))

Metadata

Release files for knlm 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for knlm 0.1.2
File	Size	Uploaded	Provenance	Actions
knlm-0.1.2.tar.gz	10.8 kB	Jun 11, 2019	Without provenance	Details for knlm-0.1.2.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for knlm 0.1.2
File	Interpreter	ABI	Platform	Actions Reset
knlm-0.1.2-cp37-cp37m-win_amd64.whl 185.6 kB Jun 11, 2019 Without provenance	CPython 3.7	CPython 3.7 pymalloc	Windows x86-64	Details for knlm-0.1.2-cp37-cp37m-win_amd64.whl
knlm-0.1.2-cp37-cp37m-win32.whl 162.4 kB Jun 11, 2019 Without provenance	CPython 3.7	CPython 3.7 pymalloc	Windows x86-32	Details for knlm-0.1.2-cp37-cp37m-win32.whl
knlm-0.1.2-cp36-cp36m-win_amd64.whl 185.6 kB Jun 11, 2019 Without provenance	CPython 3.6	CPython 3.6 pymalloc	Windows x86-64	Details for knlm-0.1.2-cp36-cp36m-win_amd64.whl
knlm-0.1.2-cp36-cp36m-win32.whl 162.4 kB Jun 11, 2019 Without provenance	CPython 3.6	CPython 3.6 pymalloc	Windows x86-32	Details for knlm-0.1.2-cp36-cp36m-win32.whl
knlm-0.1.2-cp35-cp35m-win_amd64.whl 185.6 kB Jun 11, 2019 Without provenance	CPython 3.5	CPython 3.5 pymalloc	Windows x86-64	Details for knlm-0.1.2-cp35-cp35m-win_amd64.whl
knlm-0.1.2-cp35-cp35m-win32.whl 162.4 kB Jun 11, 2019 Without provenance	CPython 3.5	CPython 3.5 pymalloc	Windows x86-32	Details for knlm-0.1.2-cp35-cp35m-win32.whl

Total release size: 1.1 MB

Release files / knlm-0.1.2.tar.gz

Download URL	knlm-0.1.2.tar.gz
Size	10.8 kB
Tags	Source
SHA-256 checksum How to use checksums	`b9c931fa4968016c8e031c81c65cd4614aa3dab7f33f3db5694ea35a5ce9ad4f`
BLAKE2b-256 checksum How to use checksums	`1c8990d52b8445db078539321daa538361ed6ecd38ed3362ec01259631fedb8d`
Upload date	Jun 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`Python-urllib/3.7`

Release files / knlm-0.1.2-cp37-cp37m-win_amd64.whl

Download URL	knlm-0.1.2-cp37-cp37m-win_amd64.whl
Size	185.6 kB
Tags	CPython 3.7 CPython 3.7 pymalloc Windows x86-64
SHA-256 checksum How to use checksums	`748296ad60328472334a60a0a472b5106735400304c836d152b60e003bb64a39`
BLAKE2b-256 checksum How to use checksums	`0e9dce9c73fbae1189ce2692435d7847b44e282687e917281eaa7a122d9e9ebe`
Upload date	Jun 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`Python-urllib/3.7`

Release files / knlm-0.1.2-cp37-cp37m-win32.whl

Download URL	knlm-0.1.2-cp37-cp37m-win32.whl
Size	162.4 kB
Tags	CPython 3.7 CPython 3.7 pymalloc Windows x86-32
SHA-256 checksum How to use checksums	`6e08bb53f96d6d39fb50500b928b84e64943554c9b02ba7a691a641151b0afad`
BLAKE2b-256 checksum How to use checksums	`1f4789ec6f4a4916130ba786fb5e376353af1491e9b8579834797ce212074e7d`
Upload date	Jun 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`Python-urllib/3.7`

Release files / knlm-0.1.2-cp36-cp36m-win_amd64.whl

Download URL	knlm-0.1.2-cp36-cp36m-win_amd64.whl
Size	185.6 kB
Tags	CPython 3.6 CPython 3.6 pymalloc Windows x86-64
SHA-256 checksum How to use checksums	`599a06f68ef58d8b5fcc2098e02f47d1eebf0f6d33bb11b97fbb3a21fa66c51b`
BLAKE2b-256 checksum How to use checksums	`d1996622b4a66b40b0365e09a89d0f0f92a18a062c004d2cac11e7afcf637933`
Upload date	Jun 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`Python-urllib/3.6`

Release files / knlm-0.1.2-cp36-cp36m-win32.whl

Download URL	knlm-0.1.2-cp36-cp36m-win32.whl
Size	162.4 kB
Tags	CPython 3.6 CPython 3.6 pymalloc Windows x86-32
SHA-256 checksum How to use checksums	`5f6cbe1a1681a3e47bef2ffad6f91c07168fa99ce431ded550545d926ce56bb8`
BLAKE2b-256 checksum How to use checksums	`b0caf51cea3792ce229f7c472553a396a6b7daf49b1eda8aa881c9349fd34a87`
Upload date	Jun 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`Python-urllib/3.6`

Release files / knlm-0.1.2-cp35-cp35m-win_amd64.whl

Download URL	knlm-0.1.2-cp35-cp35m-win_amd64.whl
Size	185.6 kB
Tags	CPython 3.5 CPython 3.5 pymalloc Windows x86-64
SHA-256 checksum How to use checksums	`1f5471db52e5e5ffe2e1d5eb390346d3c0707cb8806d6a14a4955612ae3de472`
BLAKE2b-256 checksum How to use checksums	`5b44dbc7b3a03055c9338f35604e930a4643478ee67818f4475622d6f7de6a76`
Upload date	Jun 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`Python-urllib/3.5`

Release files / knlm-0.1.2-cp35-cp35m-win32.whl

Download URL	knlm-0.1.2-cp35-cp35m-win32.whl
Size	162.4 kB
Tags	CPython 3.5 CPython 3.5 pymalloc Windows x86-32
SHA-256 checksum How to use checksums	`a9b9afa86e5b7340949c0bdc24210853ca7ce7743da60c7a1678416afb95c948`
BLAKE2b-256 checksum How to use checksums	`64ebb8018c0babd575cbd97bf46befed7b1fd44407491950760056ebbf47c80c`
Upload date	Jun 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`Python-urllib/3.5`

Security for knlm 0.1.2

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

knlm 0.1.2