soupstars 1.0.0

Declarative web parsers

Soupstars

Build declarative web parsers in python. Install it with pip.

pip install soupstars

A full example:

from soupstars import Parser, serialize

class NytimesArticleParser(Parser):
    "Parse data from a NY times article"

    @serialize
    def title(self):
        return self.h1.text

    @serialize
    def author(self):
        return self.find(attrs={'itemprop': 'author creator'}).text


if __name__ == "__main__":
    url = "https://www.nytimes.com/2019/04/25/us/politics/joe-biden-anita-hill.html"
    parser = NytimesArticleParser(url)
    print(parser.to_json())

Running the script above produces:

{
    'author': 'By Sheryl Gay Stolberg and Carl Hulse',
    'title': 'Joe Biden Expresses Regret to Anita Hill, but She Says ‘I’m Sorry’ Is Not Enough'
}