A small Python package to extract content from web pages.
Project description
Markout
A small Python package I made to extract HTML content from web pages. It is very customizable and I made it to fit my needs (extract multiple pages' code to Markdown, but only some HTML tags which I needed). Due to its purpose being able to convert specific HTML tags into a desired Markdown format this script does not generate any standard output, rather, it uses custom tokens specified in a configuration file, so the output can be formatted into any anything.
Usage
Importing into your code
To use this package you'll need to install it using pip:
pip install markout-html
Then just import it into your code:
from markout_html import *
After that you can use the extract_url and extract_html functions:
result = extract_url(
# HTML page link
'http://example.page.com/blog/some_post.html',
# Tokens to format each HTML tags contents (you can extract only the ones you want)
{
'p': "\n** {} **"
},
# Only extract contents inside this tag
'article'
)
result = extract_html(
# HTML code string
'<html>some html code</html>',
# Tokens to format each HTML tags contents (you can extract only the ones you want)
{
'p': "\n** {} **"
},
# Only extract contents inside this tag
'article'
)
Using the CLI command
Below are a few examples with better description on how to use this package command if you don't want to create a Python script!
If you just want to extract using a string in the terminal, you can use markout_html --extract [string].
You can use the command markout_html with the flag --help for more info.
Configuration
All configurations can be found into a single file: .markoutrc.json (you can specify another name in the terminal with the flag --config), if you don't load a configuration file the script will use its default values. There is an example of configuration in the repository root!
To specify a different configuration file use:
markout_html --config [filename]
The configuration file values
links - object of links to be extracted, each link has a destination value (output file).
Example:
{
"links": {
"http://example.page.com/blog/some_post.html": "out/post.md",
"http://example.page.com/blog/some_other_post.html": "out/other_post.md"
}
}
The example above will get the HTML from http://example.page.com/blog/some_post.html and extract the results into out/post.md.
only_on - string that specify where (which HTML tag) to extract the contents from (e.g. : html, body, main).
Example:
{
"only_on": "article"
}
tokens - object in which each specified HTML tag will be extract into a formatted string and then placed on the output file.
Example:
{
"tokens": {
"header": "# {}",
"h1": "\n# {}",
"h2": "\n# {}",
"b": "\n## {}",
"li": "+ {}",
"i": "** {} **",
"p": "\n{}",
"span": "{}"
}
}
On the example above, the contents of the HTML tag <header> will be extract into the # {} string, so for example, if we had <header>Some text here!</header> the result would've been # Some text here! (this formats the text into Markdown).
Contributions
Feel free to leave your contribution here, I would really appreciate it! Also, if you have any doubts or troubles using this package just contact me or leave an issue.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file markout_html-0.1.7.tar.gz.
File metadata
- Download URL: markout_html-0.1.7.tar.gz
- Upload date:
- Size: 4.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 26bcc96ad66ee3abb96aa2b88935c90b7462b19ae1f2bd3c0ac0ecd2935c4822 |
|
| MD5 | 576be5187eded76d27d93f39eeb065cd |
|
| BLAKE2b-256 | b89ab9c836031c9869e82a2f6aba65bf3ac14b77728a15bb2aa6d59bc101ec61 |
File details
Details for the file markout_html-0.1.7-py3-none-any.whl.
File metadata
- Download URL: markout_html-0.1.7-py3-none-any.whl
- Upload date:
- Size: 7.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 4f05854dd57b92b8fcfbca3c5d8f09166bde91d4231431aca99b328dcacd8f40 |
|
| MD5 | d2094cc08107ec8566ff0e04edb55c46 |
|
| BLAKE2b-256 | e8659aaf0f6475366a37cf3144a50c1e7c2701e73811d7ffd5355e6ee8557556 |
