py-tbparse

Read Tableau workbooks (.twb and .twbx) from Python. You get datasources, fields, calculated fields, joins, relationships and dashboards as pandas DataFrames. It is plain Python on top of lxml and pandas, so Tableau and R are not needed.

It began as a port of PrigasG's R package twbparser. The command-line tool, the browser GUI, renaming, templates, workbook diffing and folder scanning are new here.

What it does

• Lists what is in a workbook: datasources, parameters, fields, calculations, joins, relationships, dashboards, custom SQL, published sources.
• Checks relationships and finds calculations that refer to fields the workbook does not have.
• Shows which worksheets, dashboards and calculations use each field.
• Suggests clean names for fields, sheets, dashboards and other objects, and writes a renamed copy.
• Turns a finished workbook into a template you can fill with other data: a CSV, an Excel sheet (.xlsx, .xlsm), another workbook, or a database table described in a target file. The database is never contacted. Supported: MySQL, PostgreSQL, SQL Server and Snowflake.
• Fills {{token}} placeholders in titles, text and captions, so one template gives each customer its own dashboard.
• Makes one workbook per file in a folder (template apply-folder), brings a workbook up to a newer template revision (template update) and lints a template (template check).
• Checks a folder or glob of monthly CSV/Excel files against a template or saved answers (template drift): missing, renamed and extra columns, type conflicts, encoding, separator, moved Excel headers, empty files, with fingerprints per file. It does not write a union workbook yet.
• Audits a workbook (audit: unused or duplicate calculations, missing references, custom SQL, absolute-path leftovers) and writes a Markdown data dictionary of it (docs). Read-only; never opened in Tableau.
• Writes a share-safe copy of a workbook (sanitize IN OUT --report): user names, servers, databases, paths, custom SQL, extracts, comments and thumbnails out, a report of what was removed and what it could not judge, optional placeholders and synthetic data. A clean-up, not a guarantee.
• Removes what the audit finds unused (prune WORKBOOK: unused calculations and parameters, worksheets on no dashboard only with --sheets), only if nothing that stays refers to it. A dry run by default; --write -o OUT writes a new file.
• Keeps selected dashboards of a workbook and drops the rest (slice WORKBOOK --dashboards A,B --write -o OUT): the other dashboards, the worksheets no kept dashboard uses, their windows, and then the calculations, parameters and datasources only they used. Actions that depend on removed sheets are dropped and reported (--strict refuses instead). Not opened in Tableau. Also a Slice and copy view in the GUI.
• Copies worksheets from one workbook into another (sheet copy SRC --sheets A,B --to TARGET --write -o OUT): the target needs a datasource with the same connection and the same internal name; the calculations and parameters the sheets need are added, action filters are dropped and listed, and a clash fails unless you pick --on-clash rename or skip. A dry run by default; the same in the GUI's Slice and copy view, which shows the plan and ends in a download.
• Copies dashboards, with all their sheets, from one workbook into another (dashboard copy SRC --dashboards A,B --to TARGET --write -o OUT): the rules of sheet copy, plus the dashboard's own datasources and filter, parameter and legend dependencies, a new dashboard window, and the actions whose source and targets are all copied (the others are dropped and listed). One sheet that cannot be copied refuses its dashboard. A dry run by default.
• Writes the findings of validate, audit and template check as JUnit, SARIF or GitHub annotations for CI (--format junit|sarif|github), with a pre-commit hook and a sample workflow.
• Takes calculated fields and parameters out of one workbook into a library file and adds them to another (library export, library import). The output follows what Tableau writes but has not been opened in Tableau.
• Saves the layout of a dashboard (containers, text, styles, one slot per sheet) as a scaffold and makes a new dashboard from it with your own sheets (scaffold make, show, apply). First slice: a single tiled root only; filters, legends, controls, images and device layouts are dropped and listed. It has not been opened in Tableau.
• Takes named colour palettes out of workbooks and Preferences.tps files and writes them into a new Preferences.tps, a JSON file or a copy of a workbook (style show, export, import, check; also a Styles view in the GUI). Palettes only: it adds them to the colour picker and recolours nothing, and it has not been opened in Tableau.
• Runs as a Docker image behind a TLS proxy. From the next release, the image is also published to the GitHub Container Registry (ghcr.io/ddsna/py-tbparse).
• Compares two workbooks, scans a folder of them, and draws the data model as a graph.
• Works from Python, from the command line, or in a local browser page.

Install

pip install py-tbparse

For Excel input to templates, add the optional extra: pip install "py-tbparse[excel]" (it installs openpyxl).

Python 3.9 or newer. The package, the import and both commands all use the same name: pip install py-tbparse, import py_tbparse, py-tbparse, py-tbparse-gui. Releases up to 0.2.0 used twbparser_py and twbparser; those names are gone.

Quick start

From Python:

from py_tbparse import TwbParser

p = TwbParser("workbook.twb")    # or a .twbx
p.get_overview()                 # counts of everything
p.get_calculated_fields()        # each calculation and its formula
p.get_relationships()            # how the tables connect

Every getter returns a DataFrame. The others are get_datasources, get_parameters, get_fields, get_raw_fields, get_joins, get_relations, get_inferred_relationships, get_dashboards, get_dashboard_sheets, get_custom_sql, get_initial_sql, get_published_refs and get_field_usage. get_relationship_graph_dot() returns the data model as Graphviz text and validate() looks for problems in the relationships. In Jupyter, a bare p shows the overview.

From the command line:

py-tbparse workbook.twb                    # overview
py-tbparse workbook.twb calculated-fields
py-tbparse workbook.twb fields --format csv -o fields.csv
py-tbparse workbook.twb validate           # exit code 2 if it finds a problem

In the browser:

py-tbparse-gui workbook.twb

Across workbooks:

from py_tbparse import TwbParser, diff_workbooks, scan_folder

diff_workbooks(TwbParser("v1.twb"), TwbParser("v2.twb"), table="datasources")
scan_folder("./workbooks", table="datasources")   # one table, every workbook in the folder

Guides

• Command line: every table, the diff, batch, rename, template, library, style and scaffold commands, and their options.
• Browser GUI: opening files, the table, the overview, the audit, the graph, themes, filling a template with new data, and making a template from the open workbook (Templates).
• Running as a server: the Docker image, a reverse proxy that ends TLS, sessions, limits.
• Renaming: clean names after a datasource switch, editing the suggestions, what will stay broken.
• Templates: make a template from a workbook and apply it to new data.
• Sanitize: the share-safe copy, its categories, the report and the optional synthetic data.
• CI output and pre-commit: --format junit|sarif|github, exit codes, rule ids, sample workflow.
• Prune: what it removes, what keeps a field, the report, the limits.
• Slice: keep selected dashboards, what is dropped, actions, errors, the limits.
• Audit and data dictionary: the audit rules A001 to A011, what "used" means, the Markdown data dictionary.
• Libraries: export calculated fields and parameters, add them to another workbook, what is not covered; also a Libraries view in the GUI.
• Dashboard scaffolds: make a dashboard layout from another one, what is kept and dropped, the integrity check, and what was not checked in Tableau.
• Sheet copy: copy worksheets between workbooks, what is added and dropped, the clash policies, and what was not checked in Tableau. The library pieces under it are in Sheet copy core.
• Dashboard copy: copy dashboards with their sheets between workbooks, what comes along, the clash policies, and what was not checked in Tableau.
• Colour palettes: list, export and import named palettes, the clash policy, why nothing is recoloured and what was not checked in Tableau.
• Normalised XML diff: py-tbparse diff-xml A B and normalised_diff(), a line diff of two workbooks' XML that ignores attribute order, quotes and indentation.
• .twbx files: they are read straight from the zip; how to extract the contents.
• Development: running the tests, the workbook corpus, the browser tests.

Limits

• Little of what the tool writes has been opened in Tableau. Everything it writes is checked against Tableau's published schema and for dangling references, on 200 real workbooks. A template applied to a CSV and to a workbook has been opened in Tableau once, for one workbook, and drew its sheets (see verify-in-tableau.md); renamed workbooks, workbooks with a library added, palettes imported and other shapes have not. Open one on a copy and check it before relying on it. The original file is never modified or overwritten.
• Only part of the R package is ported. Missing: formatting, tooltips, colors, axes and sorts, dashboard layout and actions, calculation complexity, the replication brief and the Shiny inspector. The GUI covers some of what the inspector did.
• Where the R version has a bug, this one does not copy it: joins and relationships on more than one key, nested joins, the include-parameters option, and calculations with brackets inside brackets.
• No login. The GUI is meant to run on your own machine for one person. It refuses requests that come from other websites, but there is no login, so do not put it on a shared network. To share it, use the Docker image behind a proxy that adds TLS and a login.

Why a Tableau parser

A .twb is XML and a .twbx is a zip containing one, so reading them from Python is not hard. Power BI's .pbix is a binary format built on a proprietary storage engine, and getting into it from code took reverse-engineering projects like PBIXRay and pbi-tools. On the server side it goes the other way: Tableau's own Python tooling (tableauserverclient and tabcmd) is more mature and more open than what Microsoft has for the Power BI REST API.

Contributing

Issues and pull requests are welcome at github.com/DDSNA/py-tbparse. The test setup is in the development guide. AGENTS.md describes the code layout and the rules for porting a function from the R package.

Credit

Based on twbparser by George Arthur, MIT licensed.

License

MIT, see LICENSE.