Gets data from Federal Reserve (FED) by connecting to its API.
Project description
install code: pip install datapungi_fed
datapungi_fed
datapungi_fed is a python package used to extract data from the API of Federal Reserve (FED). Overall it:
- provides a quick access to a FED's time series data and access to all other datasets in the FED's API
- provides both a cleaned up output (in pandas format) and a full output of the request result
- provides code snippets that can be used to access the FED API independently of datapungi_fed
- can read a saved API key (as an environment variables (default) or from json/yaml files) to avoid having a copy of it on a script
- can automatically test:
- the connectivity to all datasets,
- the quality of the cleaned up data, and
- if the provided requests code snippet returns the correct result.
Sections
- Short sample runs
- Short sample runs of all drivers
- Description of a full return
- Setting up datapungi_fed
- Testing the package
Sample runs
Quick Setup
For a quick setup (see set the package up for more options), first get an API key from the FED, then save it as an environment variable called API_KEY_FED by, for example, typing on a termninal:
-
In windows:
> setx API_KEY_FED "your api key"
-
In mac:
$ touch ~/.bash_profile $ open -a TextEdit.app ~/.bash_profile
add the following text at the end and save it:
export API_KEY_FED=yourKey
Close the terminal (in mac, restart the computer) after saving the variable.
Short runs:
datapungi_fed is designed to quickly access FED time series data. Given one of its time series symbol (eg, 'gdp') it can be fetched by typing:
import datapungi_fed as dpf
dpf('gdp')
The FRED API has 5 main groups of databases, datapungi_fed includes a 6th group (datasetlist):
database group | description |
---|---|
dataselist | datapungi_fed database listing all FED API databases and the parameters used to query them |
categories | Catagories of datasets - 8 top categories (eg, National Accounts, Prices) that break down into subgroups |
releases | Release groups of data - about 300 (eg, Western Hemisphere Regional Economic Outlook, Penn World Table) |
series | About 600,000 time series provided by various sources |
sources | List of data sources - about 90 data providers (eg, IMF and Bank of Mexico) |
tags | Tags applied to time series (eg, location, data source, frequency) - about 5,000 tags |
These groups of databases are broken down into sets of databases. datapungi_fed access all of them, but for each group it defaults to a specific case. Below is a run sample of each default search.
import datapungi_fed as dpf
data = dpf.data()
data.datasetlist()
data.categories(125)
data.releases()
data.series('GDP')
data.sources('1')
data.tags(tag_names='monetary+aggregates;weekly')
#Query a database, return all information:
full = data.series('gnp',verbose=true)
full['dataFrame'] #pandas table, as above
full['request'] #full request run, see section below
full['code'] #code snippet of a request that reproduces the query.
data._clipcode() #copy ccode to clipboard (Windows only).
Sample run of all drivers
Notice that all panda tables include a "meta" section listing units, short table description, revision date etc. For more detailed metadata, use the verbose = True option (see, Description of a full return).
import datapungi_fed as dpf
data = dpf.data()
v = data.series('gdp')
v._meta
#or
v = dpf('gdp')
v._meta
Also, "meta" is not a pandas official attribute; slight changes to the dataframe (say, merging, or multiplying it by a number) will remove meta.
import datapungi_fed as dpf
#start the drivers:
data = dpf.data()
#FRED tags dataset:
data.tags()
data.tags(api='related_tags',tag_names='monetary+aggregates;weekly')
data.tags('tag/series','slovenia;food;oecd')
Full request result
When the verbose option is selected, eg:
tab = data.(,verbose = True)
A query returns a dictionary with three entries: dataFrame, request and code.
- dataFrame is a cleaned up version of the request result in pandas dataframe format
- request is the full output of a request query (see the request python package)
- code is a request code snippet to get the data that can be placed in a script
- (and "metadata" in some cases - listing detailed metadata)
The most intricate entry is the request one. It is an object containing the status of the query:
print(tab['request']) #200 indicates that the query was successfull
and the output:
tab['request'].json()[]
a dictionary. Its entry
tab['request'].json()[]['Results']
is again a dictionary this time with the following entries:
- Statistic: the name of the table (eg, NIPA)
- UTCProductionTime: the time when you downloaded the data
- Dimensions: the dimensions (unit of measurement) of each entry of the dataset
- Data: the dataset
- Notes: A quick description of the dataset with the date it was last revised.
Setting up datapungi_fed
To use the FED API, the first step is to get an API key from:
There are three main options to pass the key to datapungi_fed:
(Option 1) Pass the key directly:
import datapungi_fed as dpf
data = dpf.data("API KEY")
data.series('gdp')
(Option 2) Save the key in either a json or yaml file and let datapungi_fed know its location:
sample json file :
{
"FED": {"key": "**PLACE YOUR KEY HERE**", "url": ""},
(...Other API keys...)
}
sample yaml file:
FED:
key: PLACE API KEY HERE
description: FED data
url:
api2:
key:
description:
url:
Now can either always point to the API location on a run, such as:
import datapungi_fed as dpf
userSettings = {
'ApiKeysPath':'**C:/MyFolder/myApiKey.yaml**', #or .json
'ApiKeyLabel':'FED',
'ResultFormat':'JSON'
}
data = dpf.data(userSettings = userSettings)
data.series('gdp')
Or, save the path to your FED API key on the package's user settings (only need to run the utils once, datapungi_fed will remember it in future runs):
import datapungi_fed as dpf
dpf.utils.setUserSettings('C:/Path/myKeys.yaml') #or .json
data = dpf.data()
data.series('gdp')
(Option 3) Save the key in an environment variable
Finally, you can also save the key as an environment variable (eg, windows shell and in anaconda/conda virtual environment).
For example, on a command prompt (cmd, powershell etc, or in a virtual environment)
> setx FED=APIKey
Then start python and run:
import datapungi_fed as dpf
dpf('gpd')
Notice: searching for an environment variable named 'FED' is the default option. If changed to some other option and want to return to the default, run:
import datapungi_fed as dpf
dpf.utils.setUserSettings('env')
If you want to save the url of the API in the environment, call it FED_url. datapungi_fed will use the provided http address instead of the default
Changing the API key name
By default, datapungi_fed searches for an API key called 'FED' (in either json/yaml file or in the environment). In some cases, it's preferable to call it something else (in conda, use FED_Secret to encript it). To change the name of the key, run
import datapungi_fed as dpf
dpf.utils.setKeyName('FED_Secret') #or anyother prefered key name
When using environment variables, if saving the API url in the environment as well, call it KeyLabel_url (for example, 'FED_Secret_url'). Else, datapungi_fed will use the default one.
Running Tests
datapungi_fed comes with a family of tests to check its access to the API and the quality of the retrieved data. They check if:
- the connection to the API is working,
- the data cleaning step worked,
- the code snippet is executing,
- the code snippet produces the same data as the datapungi_fed query.
Other tests check if the data has being updated of if new data is available. Most of these tests are run every night on python 3.5, 3.6 and 3.7 (see the code build tag on the top of the document). However, these test runs are not currently checking the code snippet quality to check if its output is the same as the driver's. To run the tests, including the one that checks code snippet quality, type:
import datapungi_fed as dpf
dpf.tests.runTests(outputPath = 'C:/Your Path/')
This will save an html file in the path specified called datapungi_fed_Tests.html
You can save your test output folder in the user settings as well (need / at the end):
import datapungi_fed as dpf
dpf.utils.setTestFolder('C:/mytestFolder/')
References
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for datapungi_fed-0.1.3-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4895ad4e50a1c77d5abe0396039e7f6597715a6fec7aa18a1a5821f4d57b79c3 |
|
MD5 | 8c02836e1abea45a00b7fbda97061afd |
|
BLAKE2b-256 | e82e892f976fcd029b8b9a81250cd6ac9de1315310af2553f75e05225116abf7 |