Skip to main content

Declarative hypertext client

Project description

Declarative hypertext client


pip install peppertext


from peppertext import Hypertext, SimpleURLField, resolve, register, selector

class GoogleBlogPage(Hypertext):
    url = SimpleURLField(

    title = selector.find(".title[itemprop=name]").text()
    body = selector.find(".post-body").text()

It resolves given headers, url and query string to hypertext object.

>>> p = resolve("")
>>> p
<GoogleBlogPage at 0x108a4d1f0 >

>>> p.fetch()
>>> p['title']
'Google gobble: Thanksgiving trends on Search'
>>> p['body']
'In just a few hours, people across the U.S. will be settling...'

You can create GoogleBlogPage object with profile variables which are declared as fields in GoogleBlogPage class.

>>> p = GoogleBlogPage(
...     year="2015",
...     month="11",
...     title="google-gobble-thanksgiving-trends-on"
... )

>>> p.fetch()
>>> p['title']
'Google gobble: Thanksgiving trends on Search'


class GoogleBlogPage(Hypertext):
    # ...
    title = selector.find(".title[itemprop=name]").text()
    # ...

Selectors process a document which is returned from server as response. In the GoogleBlogPage example above, title selector parses document and find an element specified with “.title[itemprop=name]” css selector. You can access the value title with subscribing the GoogleBlogPage object with selector name.

document = pq("""<div>
    <a href="">Link1</a>
    <a href="">Link2</a>
    <a href="">Link3</a>

find_selector = selector.find('a')
selected_els =
self.assertEqual( [pq(el).attr["href"] for el in selected_els],


Select html elements which match to given css selector string.


Get an element’s attribute value with given attribute name.


Select the html element’s inner text value.


Get an item on index


sub_selector = selector.sub(pattern="\d+", repl="")

Do regex substitution.


int_cast_selector = selector.cast(int)

Pass the data to the function given as a parameter.


Peppertext supports Python 2.7 and 3.

Features in developing

  • Interface for parse error handling

  • Polymorphic access to page selectors

  • Interface to resolve and traverse links in a page

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

peppertext-0.1.3.tar.gz (7.8 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page