scrape 0.0.6

a webpage scraping tool

# scrape

## a web scraping tool

## Installation * pip install scrape

## Usage usage: scrape.py [-h] [-c [CRAWL [CRAWL …]]] [-ca] [-l LIMIT] [-t]       url [keywords [keywords …]]

a web scraping tool

  positional arguments:

    url   url to scrape

  optional arguments:

    -h, –help  show this help message and exit

    -k [KEYWORDS [KEYWORDS …]], –keywords [KEYWORDS [KEYWORDS …]]       filter lines by keywords, text only

    -c [CRAWL [CRAWL …]], –crawl [CRAWL [CRAWL …]]       crawl links based on these keywords

    -ca, –crawl-all  crawl all links

    -l LIMIT, –limit LIMIT  crawl page limit

    -t, –text  write to text instead of pdf

    -vb, –verbose  show pdfkit errors

    -v, –version  display current version

## Author * Hunter Hammond ([e-mail])

## Notes * Unless specified using the –text flag, all webpages are saved as pdf files using pdfkit.

• Entering keyword arguments while using the –text flag allows users to save only lines matching one of the given keywords.

• You can crawl subsequent webpages using by passing a substring of the url you wish to match using –crawl, or by using –crawl-all.

• There is no limit to the number of pages to be crawled unless one is set using the –limit flag.

News

0.0.6

• use –keywords flag for filtering text

• can pass multiple links now

• will not write empty files anymore

0.0.5

• added –verbose argument for use with pdfkit

• improved output file name processing

0.0.4

• accepts 0 or 1 url’s, allowing a call with just –version

0.0.3

• Moved utils.py to scrape/

0.0.2

• First entry