This module provide a easy, non intrusive way to process a big list of files in a parallel way. Also provides the option to process theses files with a different packs of options, evaluate and generate reports.
Requirements:
You need the PPSS script in same dir of this file.
Instructions:
-
Import this module from your main file
import pyCorpus
-
Create the function that process the file
def my_process(file_name, config): # Some science stuff with the file
-
(Optional) Create a function that return a argument parser that capture all the configs that you need.
def my_parser(): # Set up your argparse parser # Return the parser return my_parser_instance
-
Add at the end of the file something like this:
if __name__ == "__main__": corpus_processor = pyCorpus.CorpusProcessor(parse_cmd_arguments, process_file) corpus_processor.run_corpus()
NOTES:
-
Dot not ADD the () to my_parser and my_process arguments.
-
If you don't need options you can ignore step 3 and the config file come as None. But never use the --config parameter.
-
The files are processed in a concurrent way so if you might store any results don't use the sys.out use a file.
Metadata
Release files for pycorpus 2.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pycorpus-2.3.tar.gz | 5.6 kB | Without provenance | Details for pycorpus-2.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pycorpus-2.3-py3-none-any.whl | Python 3 | none | any | Details for pycorpus-2.3-py3-none-any.whl |
Total release size: 11.4 kB
Release files / pycorpus-2.3.tar.gz
| Download URL | pycorpus-2.3.tar.gz |
|---|---|
| Size | 5.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
aa07ea9d54d5b9903ad492f2ca6dc18e37179f7dca07106826964378607e2219
|
|
BLAKE2b-256 checksum How to use checksums |
7244dc7cb8e3e1a473d16b44d802d8c293555ac43aa0c47a48f56ba55881f898
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.1.5 CPython/3.8.5 Linux/5.4.0-73-generic
|
Release files / pycorpus-2.3-py3-none-any.whl
| Download URL | pycorpus-2.3-py3-none-any.whl |
|---|---|
| Size | 5.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6c6709e7137411d8994ea13dcfec992a19c92c23f9c84184c37319c2dcbf553b
|
|
BLAKE2b-256 checksum How to use checksums |
a6335ecf4003f876ac7b8100cbe2896c439cf6a2015b78ef54b291f4593d769b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.1.5 CPython/3.8.5 Linux/5.4.0-73-generic
|
Security for pycorpus 2.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.