merkury 0.5

Turn Python scripts into HTML reports

Merkury

Merkury is a command line utility to run Python and SQL scripts and render static HTML or Markdown reports with code and produced output. It uses standard .py or .sql files as input - any valid script that can be run from command line, can also be turned into a report.

• Example Python report
• Example SQL report
• Documentation

It's a lightweight alternative to tools such as jupyter and papermill. While these have their advantages (and problems), when everything you need is to generate a report from a data analysis script, they might be an overkill. This project is meant to address that scenario.

Non-goals of the project:

• interactive code execution in the browser (see jupyter),
• generating data apps that require backend server (see e.g. dash),
• converting any input into static HTML (see e.g. nikola).

NOTE: this is an early experimental project, stuff might break and change

Installation

pip3 install merkury

Usage

$ merkury -h
merkury

Usage:
    merkury [options] <script>

Options:
    -h --help                       Show this screen.
    -d <db>, --database <db>        Specify database location (if missing, in memory SQLite). Valid for SQL scripts.
    -o <file>, --output <file>      Specify report file (if missing, <script_name>_<date>).
    -f <format>, --format <format>  Specify report format: html (default), md.
    -i, --interactive               Make tables interactive (search, sort, paging). Valid for HTML output.
    -a <author>, --author <author>  Specify author (if missing, user name).
    -v, --version                   Show version and exit.

Database connection

For running SQL scripts, merkury supports PostgreSQL and SQLite. Regarding --database option:

• For SQLite, you need to specify path to db file (or empty if you want to run script in memory without an existing db)
• For PostgreSQL, you need to specify connection string, i.e.: postgresql://[userspec@][hostspec][/dbname][?paramspec]

Automatic interactive tables

When setting --interactive option, merkury will try to make all HTML <table> elements in the report interactive (i.e. add search, sort, and pagination). It's always safe to use with SQL scripts, for Python you need to ensure that printed table uses proper elements (i.e. <th> indicating headers rather that data cells). If this is not the case, interactive elements will not function properly.

PDF reports

It is also possible to obtain PDF reports with usage of additional conversion tools (e.g., pandoc). For example:

merkury -o /dev/stdout -f md <your_script> | pandoc --highlight-style=tango -t pdf -o report.pdf

Note, in case your report file contains raw html chunks (such as plots or images), you will need use wkhtmltopdf pdf engine.

Formatting and plots

Python

When it comes to report formatting, there are 3 types of outputs in a Python script: Standard <code> block (default), HTML, or Markdown.

By default merkury treats any code printing some output (e.g., print()) as one containing code and puts it into <code> blocks. If your output is actually HTML or Markdown, you need to indicate that by placing a magic comment after print statement in your script.

HTML

You need to put a comment #HTML after a line that outputs raw HTML. For example:

print(pandas_df.to_html(border=0))
#HTML

In addition to writing HTML by hand or using libraries that allow formatting output as HTML, merkury provides utility functions to format plots from common libraries. See plotting docs for details.

Markdown

It's also possible to render text formatted in markdown. You need to put magic comment #MARKDOWN after print statement.

For example:

print("""
# I'm a markdown header

List:

* l1
* l2

""")
#MARKDOWN

SQL

Unlike Python, there are no special formatting instructions you can specify in comments. Outputs from all queries will be formatted as a HTML table.

Acknowledgements

• Reports styling is made possible by great frontend libs pico, prism, and list.js
• SO discussion that inspired this project
• pyreport - similar but long abandoned project