Skip to main content
Help us improve PyPI by participating in user testing. All experience levels needed!

Top News Selectors (tns): Top news parsing from select websites

Project description

# Top-news-selectors (tns)

A static HTML site parser for parsing the top story titles and URIs for the following websites:

This parser is built on [Beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) based on the CSS selectors found inside the respective HTML documents. These selectors were chosen based on the selectors that were present during the 30 days for the month of November 2016.

This package has not been tested beyond the month of November 2016, 11/01/2016 - 11/30/2016. Use at your own risk when going beyond that range.

## Install

This package is available via pip:

` $ pip install tns `

## Usage

`python >>> from tns import SiteParser >>> html = open('npr.html').read() >>> parser = SiteParser(html) >>> parser.npr() { 'hero_text': "foo", 'hero_link': 'http://bar.com', 'headlines': [{'splash_title': 'baz', 'link': 'http://qux.com'}] } >>> parser.washingtonpost() {'hero_text': '', 'hero_link': '', 'headlines': []} `

The keys hero_text and hero_link refer to the top most identified news post, often identified by a enlarged picture or text across the top of the page. Headlines refer to the subsequent headlines after the hero article, where splash_title refers to the title found on the home page of the site not the actual title of the article.

You can see that the second function call with parser, parser.washingtonpost(), has zero entries because the document passed to SiteParser was not intended for that parser.

Each of the sites are paired to a function:

`python "http://abcnews.go.com/":          parser.abcnews() "https://www.cbsnews.com/":        parser.cbsnews() "https://www.nbcnews.com/":        parser.nbcnews() "https://www.washingtonpost.com/": parser.washingtonpost() "https://www.npr.org/":            parser.npr() "http://www.latimes.com/":         parser.latimes() "https://www.usatoday.com/":       parser.usatoday() "https://www.wsj.com/":            parser.wsj() "https://www.nytimes.com/":        parser.nytimes() "http://www.foxnews.com":          parser.foxnews() "http://www.chicagotribune.com/":  parser.chicagotribune() `

## Debugging

When the parser fails to find specific headlines or returns no headlines at all, this could be due to:

  • Iframes loading content dynamically
  • Headlines being injected via Javascript
  • The wrong parser is being used for the specified site

Project details


Release history Release notifications

This version
History Node

1.0.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
tns-1.0.0-py3-none-any.whl (7.4 kB) Copy SHA256 hash SHA256 Wheel py3 Feb 4, 2018
tns-1.0.0.tar.gz (5.3 kB) Copy SHA256 hash SHA256 Source None Feb 4, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page