Skip to main content

Allows to get an HTML, coming from several previous URLs. Sometimes this is needed to get webpages that requires cookies or a HTTP referrer to get a certain page.

Project description

Welcome to HTML-Jumping

Author:
  • Daniel Perez Rada <@dperezrada>

What?

Allows to get an HTML, coming from several previous URLs. Sometimes this is needed to get webpages that requires cookies or a HTTP referrer to get a certain page.

Pre-requisites

You will need:

To run the test you will also need:

  • lxml

Example

No proxy

from html_jumping import HtmlJumping
handler = HtmlJumping()
urls = [
    {
        'url': 'http://pypi.python.org/pypi',
        'method': 'GET'
    },
    {
        'url': 'http://pypi.python.org/pypi',
        'method': 'GET',
        'body': {
            'term': 'html_jumping',
            ':action': 'search',
            'submit': 'search'
        }
    }
]
received_header, received_content = handler.get(urls)

With proxy

Allow you to use a HTTP proxy, you will need to install the socksipy library.

from html_jumping import HtmlJumping
handler = HtmlJumping()
urls = [
    {
        'url': 'http://pypi.python.org/pypi',
        'method': 'GET'
    },
    {
        'url': 'http://pypi.python.org/pypi',
        'method': 'GET',
        'body': {
            'term': 'html_jumping',
            ':action': 'search',
            'submit': 'search'
        }
    }
]
received_header, received_content = handler.get(
    urls,
    proxy_info = {'host': '127.0.0.1', 'port': '8081'}
)

With permanent headers

This will sent in each call the headers ‘Accept-Language’.

from html_jumping import HtmlJumping
handler = HtmlJumping()
urls = [
    {
        'url': 'http://pypi.python.org/pypi',
        'method': 'GET'
    },
    {
        'url': 'http://pypi.python.org/pypi',
        'method': 'GET',
        'body': {
            'term': 'html_jumping',
            ':action': 'search',
            'submit': 'search'
        }
    }
]
received_header, received_content = handler.get(
    urls,
    permanent_headers = {'Accept-Language': 'es, en-cl;q=0.5'}
)

Tests

Run

>> nosetests

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for html_jumping, version 0.2.4
Filename, size File type Python version Upload date Hashes
Filename, size html_jumping-0.2.4.tar.gz (3.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page