beaapi 0.1.0

BEA API Python package

beaapi

beaapi: A Python library library to make it easier to retrieve and work with BEA data. For the parallel R-package, see bea.R.

To install the package, use pip install beaapi.

For help, see our documentation.

For development and building information, see CONTRIBUTING.md.

To Get Started

Once you have installed the package, you can load it in your Python script as follows:

import beaapi

To use the package, you must first register for an API key from BEA by providing your name and email address. The key will be emailed to you.

Once you have received your BEA API key, you can save it to a variable to make it easier to use later:

beakey = 'YOUR 36-DIGIT API KEY'

For scripts, an even better method is to create an unversioned text file called .env with the contents beakey=Your36DigitAPiKey and then use the package python-dotenv to load it automatically.

from dotenv import dotenv_values
beakey = dotenv_values()["beakey"]

Get metadata

The BEA data is distributed across various datasets that each contain many tables. To get a list of the datasets available on the API, use get_data_set_list():

list_of_sets = beaapi.get_data_set_list(beakey)
display(list_of_sets)  # Note the last dataset is only for speeding up metadata queries

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	DatasetName	DatasetDescription
0	NIPA	Standard NIPA tables
1	NIUnderlyingDetail	Standard NI underlying detail tables
2	MNE	Multinational Enterprises
3	FixedAssets	Standard Fixed Assets tables
4	ITA	International Transactions Accounts
5	IIP	International Investment Position
6	InputOutput	Input-Output Data
7	IntlServTrade	International Services Trade
8	IntlServSTA	International Services Supplied Through Affili...
9	GDPbyIndustry	GDP by Industry
10	Regional	Regional data sets
11	UnderlyingGDPbyIndustry	Underlying GDP by Industry
12	APIDatasetMetaData	Metadata about other API datasets

Queries to the different datasets take different parameters. To get a list of the paramaters for a given dataset, use get_parameter_list(); for example, to get a list of the parameters for the NIPA dataset, use:

list_of_params = beaapi.get_parameter_list(beakey, 'NIPA')
display(list_of_params)

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	ParameterName	ParameterDataType	ParameterDescription	ParameterIsRequiredFlag	ParameterDefaultValue	MultipleAcceptedFlag	AllValue
0	Frequency	string	A - Annual, Q-Quarterly, M-Monthly	1		1
1	ShowMillions	string	A flag indicating that million-dollar data sho...	0	N	0
2	TableID	integer	The standard NIPA table identifier	0	<NA>	0
3	TableName	string	The new NIPA table identifier	0	<NA>	0
4	Year	integer	List of year(s) of data to retrieve (X for All)	1		1	X

To get a list of the values for a given parameter, use get_parameter_values(); for example, to get a list of the parameter values for the Frequency parameter of the NIPA dataset, use:

list_of_param_vals = beaapi.get_parameter_values(beakey, 'NIPA', 'Frequency')
display(list_of_param_vals)

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	FrequencyID	Description
0	A	Annual
1	Q	Quarterly
2	M	Monthly

The API documentation is a good place to also understand allowed values.

A few datasets (ITA, IIP, InputOutput, IntlServTrade, GDPbyIndustry, Regional, and UnderlyingGDPbyIndustry) allow you to filter the values based on what will be passed in for other parameters.

tbl = beaapi.get_parameter_values_filtered(beakey, 'ITA', targetparameter='Indicator', AreaOrCountry="China",Frequency="A", Year="2011")
display(tbl.head())

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	Key	Desc
0	BalCapAcct	Balance on capital account
1	BalCurrAcct	Balance on current account
2	BalGds	Balance on goods
3	BalGdsServ	Balance on goods and services
4	BalPrimInc	Balance on primary income

A few of the datasets publish metadata tables that can be queried for particular strings using search_metadata(). This method allows you to search for BEA data by keyword. For example, to find all datasets in which the term "personal consumption" appears, use the following:

search_data = beaapi.search_metadata('Gross domestic', beakey)
search_data.head(2)

Created directory: beaapi_data
Creating first-time local copy of metadata for all datasets - only done once in working directory.
Datasets will be updated only if timestamps indicate metadata obsolete in future searches, and only obsolete metadata sets will be updated.

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	SeriesCode	RowNumber	LineDescription	LineNumber	ParentLineNumber	Tier	Path	TableId	Datasetname	TableName	ReleaseDate	NextReleaseDate
0	A191RL	10	Gross domestic product	1		0	1	T10101	NIPA	Table 1.1.1. Percent Change From Preceding Per...	Feb 28 2019 8:30AM	Mar 28 2019 8:30AM
26	A191RP	280	Gross domestic product, current dollars	27		0	27	T10101	NIPA	Table 1.1.1. Percent Change From Preceding Per...	Feb 28 2019 8:30AM	Mar 28 2019 8:30AM

Please note that that search_metadata currently searches only national data.

The contents of this function are automatically updated using a new metadata component of BEA's API; as such, we recommend that you use it with your API key, and the first use of this function requires that you use your key or it will be unable to extract the metadata.

If you do not wish to automatically update the metadata (e.g., you have conducted a study using the search function), simply searching for the term without also passing your key to the function will do a search only using your locally stored version.

Get data

Once you have identified the TableId number and other information, you can use get_data() to access the data. The following code, for example, returns the NIPA table with 2015 data for Table T20305.

bea_tbl = beaapi.get_data(beakey, datasetname='NIPA', TableName='T20305', Frequency='Q', Year='2015')
display(bea_tbl.head(2))

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	TableName	SeriesCode	LineNumber	LineDescription	TimePeriod	METRIC_NAME	CL_UNIT	UNIT_MULT	DataValue	NoteRef
0	T20305	DPCERC	1	Personal consumption expenditures (PCE)	2015Q1	Current Dollars	Level	6	12119763	T20305
1	T20305	DPCERC	1	Personal consumption expenditures (PCE)	2015Q2	Current Dollars	Level	6	12264140	T20305

We store in the meta-data the index columns so that you can create a unique index on the data-frame.

display(bea_tbl.set_index(bea_tbl.attrs['index_cols']).head(2))

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

		TableName	SeriesCode	LineDescription	METRIC_NAME	CL_UNIT	UNIT_MULT	DataValue	NoteRef
LineNumber	TimePeriod
1	2015Q1	T20305	DPCERC	Personal consumption expenditures (PCE)	Current Dollars	Level	6	12119763	T20305
2015Q2	T20305	DPCERC	Personal consumption expenditures (PCE)	Current Dollars	Level	6	12264140	T20305

Extra meta-data from the API is returned in a dictionary in the attributes called detail and can vary based on the dataset.

print('Extra detail keys:' + str(bea_tbl.attrs['detail'].keys()))
print("Let's look at some interesting ones.")
print('Statistic: ' + bea_tbl.attrs['detail']['Statistic'])
print('UTCProductionTime (to know the vintage): ' + bea_tbl.attrs['detail']['UTCProductionTime'])  # some datasets use 'TsLastUpdated'
print("Notes corresponding to NoteRef:")
display(bea_tbl.attrs['detail']['Notes'].head())

Extra detail keys:dict_keys(['Statistic', 'UTCProductionTime', 'Dimensions', 'Notes'])
Let's look at some interesting ones.
Statistic: NIPA Table
UTCProductionTime (to know the vintage): 2024-03-28T13:07:19.130
Notes corresponding to NoteRef:

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	NoteRef	NoteText
0	T20305	Table 2.3.5. Personal Consumption Expenditures...
1	T20305.1	1. Net expenses of NPISHs, defined as their gr...
2	T20305.2	2. Gross output is net of unrelated sales, sec...
3	T20305.3	3. Excludes unrelated sales, secondary sales, ...
4	T20305.4	4. Food consists of food and beverages purchas...

To retrieve a limited selection of multiple years, list all the years you want to retrieve. For example, to retrieve data for 2011-2015, use Year="2011,2012,2013,2014,2015"

The API documentation includes information about the specific parameters required by get_data().

If you would like to format the data in a wide format so that different variables (in this case LineNumbers) are different rows, you can use to_wide_vars_to_cols().

display(beaapi.to_wide_vars_in_cols(bea_tbl).head(3))

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

LineNumber	1	2	3	4	5	6	7	8	9	10	...	22	23	24	25	26	27	28	29	30	31
TimePeriod
2015Q1	12119763	3896791	1291044	478782	284726	335481	192055	2605746	935572	376595	...	352308	1360579	1008271	10625316	8771780	558876	6123874	1853536	10600376	9106564
2015Q2	12264140	3958930	1317766	496967	291226	336325	193249	2641163	937029	379874	...	356465	1369721	1013256	10769675	8899946	557436	6211388	1869729	10718003	9224176
2015Q3	12382494	3993596	1326900	497705	295659	340178	193359	2666696	946571	383041	...	367356	1380892	1013536	10881970	8994136	553953	6278542	1887834	10821082	9321182

3 rows × 31 columns

Advanced topics

Throttling: The BEA api limits requests to a maximum of 100/minute and 100MB/minute (as well as 30 errors/minute). If the user exceeds this, they will be denied access for 1 hour. This package will automatically self-throttle, so in general, the user does not have to worry about that.

See the docs for additional information on:

• How to construct API queries to pull data from existing iTables.

Quick Links

• CODE_OF_CONDUCT.md
• SECURITY
• LICENSE
• CONTRIBUTORS.md
• CONTRIBUTING.md
• SUPPORT.md
• CHANGELOG.md

Disclaimer

The United States Department of Commerce (DOC) GitHub project code is provided on an ‘as is’ basis and the user assumes responsibility for its use. DOC has relinquished control of the information and no longer has responsibility to protect the integrity, confidentiality, or availability of the information. Any claims against the Department of Commerce stemming from the use of its GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

Use of this library may result in data being stored on users' local machines. Specifically, local copies of BEA API metadata may be stored and automatically updated in the beaapi_data directory (unless otherwise changed) in order to improve performance of search_metadata().