Vocab is a python package that provides vocabulary objects for natural language processing.
Installation
pip install vocab
pip install git+https://github.com/vzhong/vocab.gitUsage
>>> from vocab import Vocab, UnkVocab
>>> v = Vocab()
>>> v.word2index('hello', train=True)
0
>>> v.word2index(['hello', 'world'], train=True)
[0, 1]
>>> v.index2word([1, 0])
['world', 'hello']
>>> v.index2word(1)
'world'
>>> small = v.prune_by_count(2)
>>> small.to_dict()
{'counts': {'hello': 2}, 'index2word': ['hello']}
>>> u = UnkVocab()
>>> u.word2index(['hello', 'world'], train=True)
[1, 2]
>>> u.word2index('hello friend !'.split())
[1, 0, 0]
>>> u.index2word(0)
'<unk>'1 maintainer
Metadata
1 maintainer
Release files for vocab 0.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| vocab-0.0.5.tar.gz | 6.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| vocab-0.0.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 14.5 kB
Release files / vocab-0.0.5.tar.gz
| Download URL | vocab-0.0.5.tar.gz |
|---|---|
| Size | 6.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
cab92e20c13f964c9c1d319267fbdbe523754e80b17ad7b93ce46bce9089e06b
|
|
BLAKE2b-256 checksum How to use checksums |
a5abd0a7c3dffef6146a3d09796ee195153241f61c054909b0f4169faa670913
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.4.0.post20200518 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.7.7
|
Release files / vocab-0.0.5-py3-none-any.whl
| Download URL | vocab-0.0.5-py3-none-any.whl |
|---|---|
| Size | 7.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0bba440212204f0427576434264605c118cfea4274342ef1c03a2620014bbe33
|
|
BLAKE2b-256 checksum How to use checksums |
a263c3f14ca498f1f811eaf8f2c6817e1bdc9724cd6294183066cd43fa124aaf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.4.0.post20200518 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.7.7
|
