Data Playbook
:book: Playbooks for data. Open, process and save table based data.
Automate repetitive tasks on table based data. Include various input and output tasks. Can be extended with custom modules.
Install: pip install dataplaybook
Use: dataplaybook playbook.yaml
Playbook structure
The playbook.yaml file allows you to load additional modules (containing tasks) and specify the tasks to execute in sequence, with all their parameters.
The tasks to perform typically follow the the structure of read, process, write.
Example yaml: (please note yaml is case sensitive)
modules: [list, of, modules]
tasks:
- task: *name
tables: # List of tables used by this task
target: # Target table name of this function
debug*: True/False # default: False
# task specific properties, refer to each task
Tasks
Tasks are implemented as simple Python functions and the modules can be found in the dataplaybook/tasks folder.
Default tasks
- build_lookup
- combine
- drop
- extend
- filter
- fuzzy_match (
pip install fuzzywuzzy) - replace
- unique
- vlookup
Module io_xlsx (loaded by default)
- read_excel
- write_excel
Module io_misc (loaded by default)
- read_csv
- read_tab_delim
- read_text_regex
- wget
- write_csv
Module io_mongo (uses pymongo)
- read_mongo
- write_mongo
- columns_to_list
- list_to_columns
Module io_pdf (requires pdftotext)
- read_pdf_pages
- read_pdf_files
Module io_xml
Module ietf
Module gis
Module fnb
Yaml Tags
!re <expression>Regular expression!es <search string>Search a file using Everything
Install development version
- Clone the repo
pip install <path> -e
1 maintainer
Release files for dataplaybook 0.3.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| dataplaybook-0.3.2.tar.gz | 25.5 kB | Details |
Release files / dataplaybook-0.3.2.tar.gz
| Download URL | dataplaybook-0.3.2.tar.gz |
|---|---|
| Size | 25.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c428ed4afab7f52e6651729f8f86efae0d2adadca1419c6075cdc879d8d91c32
|
|
BLAKE2b-256 checksum How to use checksums |
0306f87141e7d8a80a59cf1cbca1bbc5f6c79b54a3f137f20110bcdad7535f33
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/39.0.1 requests-toolbelt/0.8.0 tqdm/4.23.4 CPython/3.6.5
|
