To computes transition probability of text
Project description
text-trans
It computes a transition probability of a text.
Description
I want to determine if a word was randomly generated. I guess that it can be determined by text transition probabilities trained from correct words.
Install
$ pip install texttrans
Usage
default
Transition probability is computed for English words. I use "words_alpha.txt" of dwyl/english-words to train default probability.
from texttrans.texttrans import TextTrans
p = TextTrans().prob("pen")
print(p)
0.11640052876679541
training
It prepares a text file that lists words, e.g. like below.
hogehoge
piyopiyo
It train text transtion of input text.
from texttrans.texttrans import TextTrans
train_path = "train.txt"
model_path = "model.pki"
tt1 = TextTrans(lang=None)
tt1.train(train_path= train_path, save_path= model_path)
print("p =", tt.prob("hoge"))
It computes the probability according to trained model.
tt2 = TextTrans(model_path=model_path)
print("p =", tt.prob("hoge"))
Licence
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
texttrans-1.0.9-py3-none-any.whl
(11.4 kB
view hashes)
Close
Hashes for texttrans-1.0.9-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 314e5aeac47c2c5540ef82fa1bef0718fe959ee630f6028fcc4ea6275036627c |
|
MD5 | f7e9a83b67c1fe9831a6f985665d2243 |
|
BLAKE2b-256 | 68eae02fd0bb4139dd2803d3c9cd96e9e3cf553566cdbf397d314690a6d072fa |