requests_viewer!
Project description
## requests_viewer
The idea is that `requests_viewer` can tell us information about our requests-objects quickly.
It opens up an HTML page with information for the request.
```python
import requests
from requests_viewer.web import view_request
view_request(requests.get("https://xkcd.com/"))
# or
from requests_viewer import main
main("https://xkcd.com/") # considers different mime types
```
### Main features
- HTML page is being shown as how the crawler sees it
* Can extract the domain and hot-link so that it looks almost indistuinguishable
- Contains other nice functions to show lxml tree nodes
- Can visually show diffs between 2 html pages / trees
```python
from requests_viewer.web import view_diff_tree, get_tree
url1 = "http://xkcd.com/"
url2 = "http://xkcd.com/1/"
tree1 = get_tree(url1) # get tree from request object directly
tree2 = get_tree(url2) # could instead use `make_tree` if you already have a req
view_diff_tree(tree1, tree2)
```
Results in:
<p align="center">
<img src="https://raw.githubusercontent.com/kootenpv/requests_viewer/master/resources/screenshot1.png" width="80%" />
</p>
### Installation
pip install requests_viewer
pip3 install requests_viewer
Note that in order to do the real fancy stuff, you should install:
pip install requests_viewer[fancy]
pip3 install requests_viewer[fancy]
this will install `lxml`, `bs4` and `tldextract`.
### Types it can show currently:
- text/html
- image/*
- application/json
### Usability
Some example `web.py` functions:
``` python
def slugify(value):
def view_request(r, domain=None):
def view_html(x):
def view_node(node, attach_head=False, question_contains=None):
def view_tree(node):
def view_diff_tree(tree1, tree2, url, diff_method):
def view_diff_html(html1, html2, url, diff_method):
def view_diff(html1, html2, tree1, tree2, url, diff_method):
def make_parent_line(node, attach_head=False, question_contains=None):
def extract_domain(url):
def make_tree(html, domain=None):
def get_tree(url, domain=None):
def get_local_tree(url, domain=None):
```
### Contribute
This package is very small at the moment. I very much encourage you to contribute:
- Most likely we will want to show headers on the top of the package (html)
The idea is that `requests_viewer` can tell us information about our requests-objects quickly.
It opens up an HTML page with information for the request.
```python
import requests
from requests_viewer.web import view_request
view_request(requests.get("https://xkcd.com/"))
# or
from requests_viewer import main
main("https://xkcd.com/") # considers different mime types
```
### Main features
- HTML page is being shown as how the crawler sees it
* Can extract the domain and hot-link so that it looks almost indistuinguishable
- Contains other nice functions to show lxml tree nodes
- Can visually show diffs between 2 html pages / trees
```python
from requests_viewer.web import view_diff_tree, get_tree
url1 = "http://xkcd.com/"
url2 = "http://xkcd.com/1/"
tree1 = get_tree(url1) # get tree from request object directly
tree2 = get_tree(url2) # could instead use `make_tree` if you already have a req
view_diff_tree(tree1, tree2)
```
Results in:
<p align="center">
<img src="https://raw.githubusercontent.com/kootenpv/requests_viewer/master/resources/screenshot1.png" width="80%" />
</p>
### Installation
pip install requests_viewer
pip3 install requests_viewer
Note that in order to do the real fancy stuff, you should install:
pip install requests_viewer[fancy]
pip3 install requests_viewer[fancy]
this will install `lxml`, `bs4` and `tldextract`.
### Types it can show currently:
- text/html
- image/*
- application/json
### Usability
Some example `web.py` functions:
``` python
def slugify(value):
def view_request(r, domain=None):
def view_html(x):
def view_node(node, attach_head=False, question_contains=None):
def view_tree(node):
def view_diff_tree(tree1, tree2, url, diff_method):
def view_diff_html(html1, html2, url, diff_method):
def view_diff(html1, html2, tree1, tree2, url, diff_method):
def make_parent_line(node, attach_head=False, question_contains=None):
def extract_domain(url):
def make_tree(html, domain=None):
def get_tree(url, domain=None):
def get_local_tree(url, domain=None):
```
### Contribute
This package is very small at the moment. I very much encourage you to contribute:
- Most likely we will want to show headers on the top of the package (html)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
requests_viewer-0.0.7.tar.gz
(4.6 kB
view hashes)
Built Distribution
Close
Hashes for requests_viewer-0.0.7-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b672f82f623ebcfe1b2d6be81252efcc635a1e503eb056d7c2a4eff8668d76f1 |
|
MD5 | 929814fb1d0663ffe9ea7b0b41c699e0 |
|
BLAKE2b-256 | b1dbe3abffdc944b34aec39365070440f229df8071d9be3fa7c0fb1706a9200f |