Gencon Miner
A general content miner that leverages on Beautiful Soup and Requests to handle extraction. The main goal is to always imagine in terms of targetting parent elements in an HTML form then getting group of tags given that parent.
from gencon_miner import GenconMiner
From URL
url_miner = GenconMiner(url="http://google.com")
txt = url_miner.extract('title')
print(txt[0].text) # Google
From text
text_miner = GenconMiner(text="<p class='myclass'>Hello</p>")
txt = text_miner.extract('.myclass')
print(txt[0].text) # Hello
Convert all tag content to string
Note that contents in a tag will be delimited using newline.
meaning_of_life = """
<p class='myclass'>
Hello
<span>darkness my old friend</span>
</p>
<b>And another one</b>
"""
bulk_miner = GenconMiner(text=meaning_of_life)
print(bulk_miner.to_text()) # Hello\ndarkness my old friend\nAnd another one
Parent to target
Use-case on walking document and extracting the targets.
song_of_the_day = """
<table id="mother">
<tr>
<td class="target-1">Mamma Mia</td>
<td class="target-2">Here I go again</td>
<td class="target-3">My my</td>
<td class="target-4">How can I resist you</td>
</tr>
</table>
"""
walk_miner = GenconMiner(text=song_of_the_day)
print(walk_miner.extract('#mother', '.target-1')[0].text) # Mamma Mia
print(walk_miner.extract('#mother', '.target-3')[0].text) # My my
print(walk_miner.extract('#mother', 'td'))
# [
# <td class="target-1">Mamma Mia</td>,
# <td class="target-2">Here I go again</td>,
# <td class="target-3">My my</td>,
# <td class="target-4">How can I resist you</td>
# ]
Author
Almer Mendoza
Metadata
Release files for genconminer 0.1.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| GenconMiner-0.1.7.tar.gz | 3.7 kB | Without provenance | Details for GenconMiner-0.1.7.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| GenconMiner-0.1.7-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details for GenconMiner-0.1.7-py2.py3-none-any.whl |
Total release size: 7.9 kB
Release files / GenconMiner-0.1.7.tar.gz
| Download URL | GenconMiner-0.1.7.tar.gz |
|---|---|
| Size | 3.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e1576f32f6af443ca444c3cc8ed2c94a1072908ac98bb944d873b07c997cb220
|
|
BLAKE2b-256 checksum How to use checksums |
9529474d36ebe4e47fa9fd4bd125b72090ae431bd54e2a114a5634d821df39b3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/0.12.16 CPython/3.7.3 Darwin/18.6.0
|
Release files / GenconMiner-0.1.7-py2.py3-none-any.whl
| Download URL | GenconMiner-0.1.7-py2.py3-none-any.whl |
|---|---|
| Size | 4.2 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
2c0f6677acb51ae9906415437a47ff7c689815263531c79c1c86c102d5b47c0a
|
|
BLAKE2b-256 checksum How to use checksums |
6e64aab7dd924a06d4c5f3f6d7da7884afcdce0df60e672bfc08d3f720f9e7b0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/0.12.16 CPython/3.7.3 Darwin/18.6.0
|
Security for genconminer 0.1.7
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.