pytde 0.1.0

Python parser for legacy Tableau Data Extract (TDE) files

pytde

A Python parser for legacy Tableau Data Extract (TDE) files.

Overview

pytde reads TDE files (the legacy Tableau data format used before Hyper) and converts them to pandas DataFrames. This is useful for accessing data from older Tableau extracts without needing Tableau Desktop or Server.

Installation

pip install pytde

Or install from source:

git clone https://github.com/ronreiter/pytde.git
cd pytde
pip install -e .

Quick Start

from pytde import read_tde

# Read a TDE file
tables = read_tde('data.tde')

# Get the Extract table as a DataFrame
df = tables['Extract']

# Work with the data
print(df.head())
print(df.describe())

API Reference

read_tde(file_path: str) -> dict[str, pd.DataFrame]

Read a TDE file and return its contents as pandas DataFrames.

from pytde import read_tde

tables = read_tde('sales_data.tde')
df = tables['Extract']

read_tde_metadata(file_path: str) -> dict

Read TDE file metadata without fully parsing the data.

from pytde import read_tde_metadata

metadata = read_tde_metadata('sales_data.tde')
print(metadata['columns'])       # List of column names
print(metadata['column_types'])  # Column data types
print(metadata['file_size'])     # File size in bytes
print(metadata['format_version']) # TDE format version

TDEParser class

For more control over parsing, use the TDEParser class directly:

from pytde import TDEParser

parser = TDEParser('data.tde')
tables = parser.parse()
metadata = parser.get_metadata()

# Access internal state
print(parser.xml_metadata)  # Embedded XML schema
print(parser.column_entries)  # Column index entries

Command Line Interface

pytde includes a CLI tool for quick inspection of TDE files:

pytde data.tde

Output:

Parsing TDE file: data.tde
------------------------------------------------------------
Format version: 2
File size: 51306 bytes
Columns found: ['Region', 'Sales', 'Sales Person']
Column types: {'Region': 'string', 'Sales': 'double', 'Sales Person': 'string'}

============================================================
Table: Extract
Shape: (43, 3)
Columns: ['Region', 'Sales', 'Sales Person']
...

Supported Data Types

TDE Type	Python/Pandas Type
string	object (str)
double	float64
integer	int64
date	object*
datetime	object*
boolean	bool

*Date/datetime support is limited in the current version.

TDE File Format

TDE files are binary files that store columnar data optimized for Tableau's data engine. Key features:

• Little-endian byte ordering
• Block-based structure with markers (f0ca1278 for data, f1ca1278 for index)
• Dictionary encoding for string columns
• Embedded XML metadata for schema definitions

For detailed format specification, see TDE.MD.

Limitations

• Index decoding: String column row-to-value mapping uses fallback distribution when exact indices cannot be decoded
• Date/time columns: Limited support for date and datetime types
• Compression: Some compressed TDE files may not be fully supported
• Large files: Memory usage scales with file size (entire file is loaded into memory)

Development

Setup

git clone https://github.com/ronreiter/pytde.git
cd pytde
pip install -e ".[dev]"

Running Tests

pytest

Running Tests with Coverage

pytest --cov=pytde --cov-report=html

License

MIT License - see LICENSE for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Acknowledgments

This parser was developed through reverse engineering of the TDE file format. Special thanks to the open source community for their work on understanding proprietary file formats.