LM_Dataformat 
Utilities for storing data for LM training.
Basic Usage
To write:
ar = Archive('output_dir')
for x in something():
# do other stuff
ar.add_data(somedocument, meta={
'example': stuff,
'someothermetadata': [othermetadata, otherrandomstuff],
'otherotherstuff': True
})
# remember to commit at the end!
ar.commit()
To read:
rdr = Reader('input_dir_or_file')
for doc in rdr.stream_data():
# do something with the document
1 maintainer
Metadata
1 maintainer
Release files for lm-dataformat 0.0.20
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| lm_dataformat-0.0.20.tar.gz | 5.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| lm_dataformat-0.0.20-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 11.5 kB
Release files / lm_dataformat-0.0.20.tar.gz
| Download URL | lm_dataformat-0.0.20.tar.gz |
|---|---|
| Size | 5.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0016165b34d8f004753ac265348c3525532e55088f6c9c160f3597e660207145
|
|
BLAKE2b-256 checksum How to use checksums |
759dbd07ed572bde0e1c0eefb8c3ef9ca2fb31592946e82fef0ce902534e55e0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.6.3 pkginfo/1.7.1 requests/2.22.0 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.8.10
|
Release files / lm_dataformat-0.0.20-py3-none-any.whl
| Download URL | lm_dataformat-0.0.20-py3-none-any.whl |
|---|---|
| Size | 5.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
247468181c9c2fea33a663cdb2f6fea489ddf6741d216fe6b466e60f002705af
|
|
BLAKE2b-256 checksum How to use checksums |
7c5bddcd67b81e0a41c367c1e52e761fd6071407985cc67f621d5e16c5209ff0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.6.3 pkginfo/1.7.1 requests/2.22.0 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.8.10
|
