Skip to main content
Help us improve PyPI by participating in user testing. All experience levels needed!

Redone port of Readability API in Python

Project description

breadability - another readability Python port

I’ve tried to work with the various forks of some ancient codebase that ported readability to Python. The lack of tests, unused regex’s, and commented out sections of code in other Python ports just drove me nuts.

I put forth an effort to bring in several of the better forks into one codebase, but they’ve diverged so much that I just can’t work with it.

So what’s any sane person to do? Re-port it with my own repo, add some tests, infrastructure, and try to make this port better. OSS FTW (and yea, NIH FML, but oh well I did try)

This is a pretty straight port of the JS here:

Installation

Currently it’s git only until I get everything ready for a submission to PyPi.

Usage

cmd line

$ breadability http://wiki.python.org/moin/BeginnersGuide

Add the -v flag to get some details on how we actually parsed this thing. I want to grow that debugging info into enough to try to track good/bad things we did in processing.

$ breadability -v http://wiki.python.org/moin/BeginnersGuide

Using from Python

from breadability.readable import Article
readable_article = Article(html_text, url=url_came_from)
print readable_article

Work to be done

Yep, I’ve got some catching up to do. I don’t do pagination, I’ve got a lot of custom tweaks I need to get going, there are some articles that fail to parse. I also have more tests to write on a lot of the cleaning helpers, but hopefully things are setup in a way that those can/will be added.

Fortunately, I need this library for my tools:

so I really need this to be an active and improving project.

Off the top of my heads todo list:

  • Support metadata from parsed article [url, confidence scores, all candidates we thought about?]
  • More tests, more thorough tests
  • More sample articles we need to test against in the test_articles
  • Tests that run through and check for regressions of the test_articles
  • Tidy’ing the HTML that comes out, might help with regression tests ^^
  • Multiple page articles
  • Performance tuning, we do a lot of looping and re-drop some nodes that should be skipped. We should have a set of regression tests for this so that if we implement a change that blows up performance we know it right away.
  • Get up on pypi along with the rest of the ports
  • More docs for things, but sphinx docs and in code comments to help understand wtf we’re doing and why. That’s the biggest hurdle to some of this stuff.

Helping out

If you want to help, shoot me a pull request, an issue report with broken urls, etc.

You can ping me on irc, I’m always in the #bookie channel in freenode.

News

0.2a1

Release date: UNRELEASED

  • Example news entry for the in-development version

0.1

Release date: 15-Mar-2010

  • Example news entry for a released version

Project details


Release history Release notifications

History Node

0.1.20

History Node

0.1.19

History Node

0.1.18

History Node

0.1.17

History Node

0.1.16

History Node

0.1.15

History Node

0.1.14

History Node

0.1.13

History Node

0.1.12

History Node

0.1.11

History Node

0.1.10

History Node

0.1.9

History Node

0.1.8

History Node

0.1.7

History Node

0.1.6

History Node

0.1.5

History Node

0.1.4

History Node

0.1.3

History Node

0.1.2

History Node

0.1.1

This version
History Node

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
breadability-0.1.0.tar.gz (17.2 kB) Copy SHA256 hash SHA256 Source None May 7, 2012

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page