tidyfinance

Helper functions for empirical research in financial economics, addressing a variety of topics covered in Scheuch, Voigt, Weiss, and Frey (2024). The package is designed to provide shortcuts for issues extensively discussed in the book, facilitating easier application of its concepts. For more information and resources related to the book, visit tidy-finance.org/python.

Installation

You can install the release version from PyPI:

pip install tidyfinance

You can install the development version from GitHub:

pip install "git+https://github.com/tidy-finance/py-tidyfinance"

Choosing a Data Frame Backend

The package is implemented in polars internally. By default, the public tidyfinance API still returns pandas data frames for backward compatibility. If you prefer polars — as used throughout the book — switch the global backend with set_backend():

import tidyfinance as tf

tf.set_backend("polars")

# Returns a polars DataFrame
data = tf.download_data(
  domain="Fama-French",
  dataset="factors_ff_3_monthly",
  start_date="2000-01-01",
  end_date="2020-12-31"
)

# Subsequent calls also return polars
tf.estimate_model(data, "mkt_excess ~ smb + hml")
tf.set_backend("pandas")  # back to the default

The setting applies to the whole public API, so any data-bearing function honors it. Pandas data frames are also accepted as input regardless of the active backend (they are converted to polars internally), so results from one call can be fed straight into the next. Under the polars backend there is no conversion overhead at all, since polars is the native representation. You can check the current setting with tf.get_backend().

Note: The default backend is currently pandas, but we expect to switch the default to polars from version 1.0.0 onwards. To keep your code working across that change, set the backend explicitly via tf.set_backend(...).

Download Open Source Data

The main functionality of the tidyfinance package centers around data download. You can download most of the data that we used in Tidy Finance with R using the download_data() function or its children.

import tidyfinance as tf

The function always requires a domain argument and depending on the domain typically also a dataset. For instance, to download monthly Fama-French factors, you have to provide the dataset name according to pdr.famafrench.get_available_datasets():

tf.download_data(
  domain="Fama-French",
  dataset="Fama/French 5 Factors (2x3) [Daily]",
  start_date="2000-01-01",
  end_date="2020-12-31"
)

For q factors, you provide the relevant file name:

tf.download_data(
  domain="Global Q",
  dataset="q5_factors_monthly",
  start_date="2000-01-01",
  end_date="2020-12-31"
)

To download the Welch and Goyal (2008) macroeconomic predictors for monthly, quarterly, or annual frequency:

tf.download_data(
  domain="Goyal-Welch",
  dataset="monthly",
  start_date="2000-01-01",
  end_date="2020-12-31"
)

To download data from Open Source Asset Pricing (OSAP):

tf.download_data(
  domain="Open Source Asset Pricing",
  start_date="2020-01-01",
  end_date="2020-12-31"
)

To download characteristic-managed portfolio (factor) returns from Global Factor Data, select a region, the factor content (a single factor, a theme, "all_themes", or "all_factors"), a frequency, and a weighting scheme:

tf.download_data(
  domain="Global Factor Data",
  region="usa",
  factors="all_factors",
  frequency="monthly",
  weighting="vw_cap",
  start_date="2020-01-01",
  end_date="2020-12-31"
)

The dataset argument also gives access to the underlying long-short portfolios ("portfolios"), industry returns ("industry"), and the reference files "nyse_cutoffs" and "return_cutoffs":

tf.download_data(
  domain="Global Factor Data",
  dataset="industry",
  region="usa",
  classification="gics",
  start_date="2020-01-01",
  end_date="2020-12-31"
)

Use tf.list_supported_jkp_factors() to see the available regions, or tf.list_supported_jkp_factors("usa") to see the factors available for a region.

To download the liquidity factors of Pastor and Stambaugh (2003) from Lubos Pastor's data library:

tf.download_data(
  domain="Pastor-Stambaugh",
  start_date="2020-01-01",
  end_date="2020-12-31"
)

To download the mispricing factors of Stambaugh and Yuan (2017) from Robert Stambaugh's data library, optionally selecting "monthly" (the default) or "daily" data. Note that the source files currently end in December 2016:

tf.download_data(
  domain="Stambaugh-Yuan",
  dataset="monthly",
  start_date="2015-01-01",
  end_date="2016-12-31"
)

To download multiple series from the Federal Reserve Economic Data (FRED):

tf.download_data(
  domain="FRED",
  series=["GDP", "CPIAUCNS"],
  start_date="2020-01-01",
  end_date="2020-12-31"
)

To download the FRED-MD / FRED-QD (McCracken-Ng) macroeconomic databases, a curated, balanced panel of monthly or quarterly macro series with McCracken-Ng stationarity transform codes. Set transform=True to apply the transforms, and vintage to a "YYYY-MM" label or "all" to access historical (real-time) releases:

tf.download_data(
  domain="FRED",
  dataset="FRED-MD",
  transform=True
)

tf.download_data(
  domain="FRED",
  dataset="FRED-QD",
  vintage="2020-03"
)

To download stock prices from Yahoo Finance:

tf.download_data(
  domain="Stock Prices",
  symbols=["AAPL", "MSFT"],
  start_date="2020-01-01",
  end_date="2020-12-31"
)

To download index constituents from selected ETF holdings:

tf.download_data(
  domain="Index Constituents",
  index="S&P 500"
)

Download WRDS Data

To access data from the Wharton Research Data Services (WRDS), you need to set your credentials first:

tf.set_wrds_credentials()

To download monthly CRSP data:

tf.download_data(
  domain="WRDS",
  dataset="crsp_monthly",
  start_date="2020-01-01",
  end_date="2020-12-31"
)

To download annual (or quaterly) Compustat data:

tf.download_data(
  domain="WRDS",
  dataset="compustat_annual",
  start_date="2020-01-01",
  end_date="2020-12-31"
)

To download the CRSP-Compustat linking table:

tf.download_data(
  domain="WRDS",
  dataset="ccm_links"
)

To download bond characteristics from Mergent FISD:

tf.download_data(
  domain="WRDS",
  dataset="fisd"
)

To download Enhanced TRACE data for selected bonds:

tf.download_data(
  domain="WRDS",
  dataset="trace_enhanced",
  cusips=["00101JAH9"],
  start_date="2019-01-01",
  end_date="2021-12-31"
)

To download high-frequency S&P 500 data or factor library data from Hugging Face:

tf.download_data(
  domain="Tidy Finance",
  dataset="high_frequency_sp500",
  start_date="2007-07-26",
  end_date="2007-07-27"
)

tf.download_data(
  domain="Tidy Finance",
  dataset="factor_library",
  sorting_variable="me"
)

Other Helpers

We include functions to check out content from tidy-finance.org:

tf.list_tidy_finance_chapters()
tf.open_tidy_finance_website("capital-asset-pricing-model")

We also include (experimental) functions that can be used for different applications, but note that they might heavily change in future package versions as we try to make them more general:

# Create summary statistics
help(tf.create_summary_statistics)

# Assign portfolios
help(tf.assign_portfolio)

# Estimate betas
help(tf.estimate_betas)

# Estimate Fama-MacBeth
help(tf.estimate_fama_macbeth)

# Add lag columns
help(tf.add_lagged_columns)

# Winsorize or trim
help(tf.winsorize)
help(tf.trim)