boxfish: lightweight table extraction from HTML

What is it?

Boxfish is a lightweight tool for table extraction from HTML pages.

Main features

• Easy configuration. No knowledge of CSS or Xpaths required.
• Fast table extraction to CSV files.
• Integration of requests and selenium.

Quick start

import boxfish as bf
import pandas as pd

# Define table layout of an url with strings from two rows.
aurl = ""
row1 = ""
row2 = ""

# Build a configuration 
aconfig = bf.build(url=aurl, rows = [row1, row2])

# Extract a table
data = bf.extract(aconfig, url=aurl)

# View results
df = pd.DataFrame(data)
df.head() 

Where to get it?

Boxfish is available on Pypi and Github.

pip install boxfish

Dependencies

The main dependencies are:

• BeautifulSoup4, a Python library for pulling data out of HTML and XML files.
• lxml, a powerful and Pythonic XML processing library.
• Requests, a simple, yet elegant, HTTP library.
• Selenium, automated web browser interaction from Python.

License

Boxfish is available with an MIT license.

Limitations

Boxfish extracts text from HTML. To see if the HTML file contains the text of interest, open the page in a browser, then access the HTML in the developer tools via Cntrl+Shift+ I.

Documentation

Full documentation is available here.