lst 0.4.0

Declarative Scraping Tools

LST

Declarative Scraping Tools

Fetch - get content (requests library)

Scan - select tags by css selector (bs4 library)

Example

from lst import Fetch, Scan

parser = Fetch() >> Scan('a') >> Fetch() >> {'h1': Scan('h1'), 'h5': Scan('h5')}

for res in parser('http://some.site'):
    if 'h1' in res:
        tag = res['h1']
        print('H1', tag.get_text())
    if 'h5' in res:
        tag = res['h5']
        print('H5', tag.get_text())