brazilian-data 0.1.2

This repository is part of the creation of a package for collecting Brazilian data

Links do projeto

GitHub Statistics

Brazilian Data

This repository is part of the development of a Python package designed for collecting data from Brazilian sources. It retrieves data from institutions such as the Banco Central do Brasil, IBGE, IPEA, and FRED. To search for series from the sources, use the following websites.

• Banco Central
• IBGE
• IPEA
• FRED

Table of Contents

• Installation
• How to import
• EconomicData Class Documentation
• Brazilian_Data Usage
• Contributing
• License

Installation

To install the package, execute the following command:

pip install brazilian-data

How to import

from brazilian_data import EconomicData

EconomicData Class Documentation (Version sync)

Overview

The EconomicData class is designed to facilitate the fetching, processing, and saving of economic data from various sources such as Banco Central, IBGE, IPEADATA, and FRED.For examples, see economic_data_examples.ipynb.

Initialization

__init__(self, codes_banco_central=None, codes_ibge=None, codes_ibge_link=None, codes_ipeadata=None, codes_fred=None, start_date=None)

Initializes the EconomicData class with optional parameters for data codes and start date.

• Parameters:
  • codes_banco_central (dict): Dictionary of Banco Central codes.
  • codes_ibge (dict): Dictionary of IBGE codes.
  • codes_ibge_link (dict): Dictionary of IBGE links.
  • codes_ipeadata (dict): Dictionary of IPEADATA codes.
  • codes_fred (dict): Dictionary of FRED codes.
  • start_date (str): Start date for data fetching.

Methods

fetch_data_for_code(self, link, column)

Fetches data from an IBGE link for a specific column.

• Parameters:
  • link (str): URL link to fetch data from.
  • column (str): Column name to fetch data for.
• Returns: Data fetched from the specified link and column.

data_index(self)

Generates a DataFrame with a date range starting from start_date.

• Returns: DataFrame with a date range as the index.

datas_banco_central(self, save=None, diretory=None, data_format=None)

Fetches data from Banco Central and handles exceptions.

• Parameters:
  • save (bool): Whether to save the data.
  • diretory (str): Directory where the data will be saved.
  • data_format (str): Format to save the data ('csv', 'excel', 'json', 'pickle').
• Returns: DataFrame with Banco Central data if not saving.

datas_ibge(self, save=False, diretory=None, data_format=None)

Fetches IBGE data and handles exceptions.

• Parameters:
  • save (bool): Whether to save the data.
  • diretory (str): Directory where the data will be saved.
  • data_format (str): Format to save the data ('csv', 'excel', 'json', 'pickle').
• Returns: DataFrame with IBGE data if not saving.

datas_ibge_link(self, save=None, diretory=None, data_format=None)

Fetches data from IBGE links and handles exceptions.

• Parameters:
  • save (bool): Whether to save the data.
  • diretory (str): Directory where the data will be saved.
  • data_format (str): Format to save the data ('csv', 'excel', 'json', 'pickle').
• Returns: DataFrame with IBGE link data if not saving.

datas_ipeadata(self, salve=None, diretory=None, data_format=None)

Fetches IPEADATA data and handles exceptions.

• Parameters:
  • salve (bool): Whether to save the data.
  • diretory (str): Directory where the data will be saved.
  • data_format (str): Format to save the data ('csv', 'excel', 'json', 'pickle').
• Returns: DataFrame with IPEADATA data if not saving.

datas_fred(self, save=None, diretory=None, data_format=None)

Fetches data from FRED and handles exceptions.

• Parameters:
  • save (bool): Whether to save the data.
  • diretory (str): Directory where the data will be saved.
  • data_format (str): Format to save the data ('csv', 'excel', 'json', 'pickle').
• Returns: DataFrame with FRED data if not saving.

datas_brazil(self, datas_bcb=True, datas_ibge_codigos=True, datas_ibge_link=True, datas_ipeadata=True, datas_fred=False, missing_data=True, fill_method=None, save=None, directory=None, data_format=None)

Fetches all data based on specified options.

• Parameters:
  • datas_bcb (bool): Whether to fetch Banco Central data.
  • datas_ibge_codigos (bool): Whether to fetch IBGE data by codes.
  • datas_ibge_link (bool): Whether to fetch IBGE data by links.
  • datas_ipeadata (bool): Whether to fetch IPEADATA data.
  • datas_fred (bool): Whether to fetch FRED data.
  • missing_data (bool): Whether to handle missing data.
  • fill_method (str): Method to handle missing data ('ffill' or 'bfill').
  • save (bool): Whether to save the data.
  • directory (str): Directory where the data will be saved.
  • data_format (str): Format to save the data ('csv', 'excel', 'json', 'pickle').
• Returns: DataFrame with all requested data if not saving.

save_datas(self, dados, diretory=None, data_format="csv")

Saves the data to the specified directory and format.

• Parameters:
  • dados (DataFrame): DataFrame to be saved.
  • diretory (str): Directory where the data will be saved.
  • data_format (str): Format to save the data ('csv', 'excel', 'json', 'pickle').

help(self)

Prints out information about the available methods and their usage.

• Usage: instance.help()

Brazilian_Data Usage

Initialization of the class

• Parameters: start_date

The variable start_date defines the initial date for data collection.

Example:

start_date = "2020-01-01"

• Parameters: codes_banco_central

The codes_banco_central should be a dictionary with the column name that the datas_banco_central function will recognize as the column name and the series code.

Example:

bcb_codes = {
    "selic": 4189,
    "cambio": 3698,
    "pib_mensal": 4380,
    "igp_m": 189,
    "igp_di": 190,
    "m1": 27788,
}

• Parameters: codes_ibge

The codes_ibge input should be a dictionary where each key represents the name of a variable (e.g., "ipca"), which the function will use as the column name. The value associated with each key should be another dictionary containing the following fields:

• "codigo": An integer representing the code of the data series to be collected. This code is specific to the variable and identifies the series in the IBGE database.

• "territorial_level": A string indicating the territorial level of the data, such as "1" for Brazil, "2" for regions, "3" for states, etc.

• "ibge_territorial_code": A code defining the specific geographical area of the data. The value "all" indicates that data for all areas corresponding to the territorial level should be collected.

• "variable": A string or number that identifies the specific variable within the data series. This may represent a specific category or indicator to be collected.

Example:

variaveis_ibge = {
    "ipca": {
        "codigo": 1737,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "63",
    }
}

• Parameters: codes_ibge_link

The codes_ibge_link input should be a dictionary where each key represents the name of an economic indicator or specific variable (e.g., "pib", "soja"), which the function will use as the column name. The value associated with each key is a URL that points to an Excel file available on the IBGE website, containing the data corresponding to that indicator.

These URLs are dynamically generated from the IBGE SIDRA system and can be used to download the tables directly. Each link contains specific parameters defining the selection of variables, periods, and territorial levels relevant to the query.

Example:

    indicadores_ibge_link = {
    "capital_fixo": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela5932.xlsx&terr=N&rank=-&query=t/5932/n1/all/v/6561/p/all/c11255/93406/d/v6561%201/l/v,p%2Bc11255,t",

    "soja": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela6588.xlsx&terr=N&rank=-&query=t/6588/n1/all/v/35/p/all/c48/0,39443/l/v,p%2Bc48,t",}

These URLs allow for the direct download of the data in Excel format, which can then be processed by your code.

• Parameters: codes_ipeadata

The codes_ipeadata should be a dictionary with the column name that the datas_ipeadata function will recognize as the column name and the series code.

codigos_ipeadata= {
    "taja_juros_ltn": "ANBIMA12_TJTLN1212",
    "imposto_renda": "SRF12_IR12",
    "ibovespa": "ANBIMA12_IBVSP12",
    "consumo_energia": "ELETRO12_CEET12",
    "brent_fob": "EIA366_PBRENT366",}

• Parameters: codes_fred

The codes fred should be a dictionary with the column name that the datas_fred function will recognize as the column name and the series code.

codigos_fred = {
    "nasdaq100": "NASDAQ100",
    "taxa_cambio_efetiva": "RBBRBIS",
    "cboe_nasdaq": "VXNCLS",
    "taxa_juros_interbancaria": "IRSTCI01BRM156N",
    "atividade_economica_eua": "USPHCI",}

Method datas_brazil

For collecting data from different sources, you can use the datas_brazil method:

## Import the `EconomicData` class
from brazilian_data import EconomicData

## Define parameters that will be used in the `datas_brazil` method
DATA_INICIO = "2000-01-01"

data_bcb = True
data_ibge = True
data_ibge_link = True
data_ipeadata = True
data_fred = False

## Initialize the `EconomicData` class
economic_brazil = EconomicData(codes_banco_central=variaveis_banco_central, 
                                 codes_ibge=variaveis_ibge, 
                                 codes_ipeadata=codigos_ipeadata, 
                                 codes_ibge_link=indicadores_ibge_link,
                                 start_date=DATA_INICIO)

## Call the `datas_brazil` method
dados = economic_brazil.datas_brazil(datas_bcb= data_bcb,
                                     datas_ibge_codigos=data_ibge, 
                                     datas_ibge_link=data_ibge_link, 
                                     datas_ipeadata=data_ipeadata,
                                     missing_data=True)

dados.head()

	selic	cambio	pib_mensal	igp_m	igp_di	m1	ipca	custo_m2	pesquisa_industrial_mensal	pib	...	capital_fixo	producao_industrial_manufatureira	soja	milho_1	milho_2	taja_juros_ltn	imposto_renda	ibovespa	consumo_energia	brent_fob
2000-01-01	18.94	1.8037	92576.6	1.24	1.02	79015372.0	0.62	5.51	69.71441	1.4	...	-5.0	57.43586	52381672.0	31477232.0	10595844.0	19.465261	5043.680936	-4.113276	25060.0	25.511000
2000-02-01	18.87	1.7753	91770.4	0.35	0.19	79015372.0	0.13	5.51	69.71441	1.4	...	-5.0	57.43586	52381672.0	31477232.0	10595844.0	19.465261	4120.602582	7.761777	25057.0	27.775714
2000-03-01	18.85	1.7420	92579.9	0.15	0.18	79015372.0	0.22	5.51	69.71441	1.1	...	-0.3	57.43586	52381672.0	31477232.0	10595844.0	19.465261	5606.185192	0.906002	25662.0	27.486087
2000-04-01	18.62	1.7682	91376.2	0.23	0.13	79015372.0	0.42	5.51	69.71441	1.1	...	-0.3	57.43586	52381672.0	31477232.0	10595844.0	19.465261	4634.431697	-12.811448	25598.0	22.764444
2000-05-01	18.51	1.8279	98727.0	0.31	0.67	79015372.0	0.01	5.51	69.71441	1.1	...	-0.3	57.43586	52381672.0	31477232.0	10595844.0	21.681500	4047.302075	-3.739461	25448.0	27.737619

5 rows × 21 columns

The missing_data parameter indicates whether missing values will be replaced using the ffill and bfill methods. By default, missing_data will be True, and missing values will be replaced. If you prefer to keep the missing data, set the missing_data parameter to False.

## Import the `EconomicData` class
from brazilian_data import EconomicData

## Define parameters that will be used in the `datas_brazil` method
data_bcb = True
data_ibge = True
data_ibge_link = True
data_ipeadata = True
data_fred = False

## Initialize the `EconomicData` class
economic_brazil = EconomicData(codes_banco_central=variaveis_banco_central, 
                                 codes_ibge=variaveis_ibge, 
                                 codes_ipeadata=codigos_ipeadata, 
                                 codes_ibge_link=indicadores_ibge_link,
                                 start_date=DATA_INICIO)

## Call the `datas_brazil` method
dados = economic_brazil.datas_brazil(datas_bcb= data_bcb,
                                     datas_ibge_codigos=data_ibge, 
                                     datas_ibge_link=data_ibge_link, 
                                     datas_ipeadata=data_ipeadata,
                                     missing_data=False)
dados.head()

	cambio	pib_mensal	igp_m	m1	ipca	custo_m2	pesquisa_industrial_mensal	pib	despesas_publica	capital_fixo	producao_industrial_manufatureira	soja	milho_1	milho_2	taja_juros_ltn	imposto_renda	ibovespa	consumo_energia	brent_fob
2000-01-01	1.8037	92576.6	1.24	NaN	0.62	NaN	NaN	1.4	3.9	-5.0	NaN	NaN	NaN	NaN	NaN	5043.680936	-4.113276	25060.0	25.511000
2000-02-01	1.7753	91770.4	0.35	NaN	0.13	NaN	NaN	1.4	3.9	-5.0	NaN	NaN	NaN	NaN	NaN	4120.602582	7.761777	25057.0	27.775714
2000-03-01	1.7420	92579.9	0.15	NaN	0.22	NaN	NaN	1.1	3.6	-0.3	NaN	NaN	NaN	NaN	NaN	5606.185192	0.906002	25662.0	27.486087
2000-04-01	1.7682	91376.2	0.23	NaN	0.42	NaN	NaN	1.1	3.6	-0.3	NaN	NaN	NaN	NaN	19.465261	4634.431697	-12.811448	25598.0	22.764444
2000-05-01	1.8279	98727.0	0.31	NaN	0.01	NaN	NaN	1.1	3.6	-0.3	NaN	NaN	NaN	NaN	21.681500	4047.302075	-3.739461	25448.0	27.737619

With the parameter save=True, you can download data. Define the directory parameter to save the file. If you don't define the directory parameter, an error will be returned. You can save in four formats: csv, json, pickle, and excel.

## Import the `EconomicData` class
from brazilian_data import EconomicData

## Define parameters that will be used in the `datas_brazil` method
data_bcb = True
data_ibge = True
data_ibge_link = True
data_ipeadata = True
data_fred = False

## Initialize the `EconomicData` class
economic_brazil = EconomicData(codes_banco_central=variaveis_banco_central, 
                                 codes_ibge=variaveis_ibge, 
                                 codes_ipeadata=codigos_ipeadata, 
                                 codes_ibge_link=indicadores_ibge_link,
                                 start_date=DATA_INICIO)

## Call the `datas_brazil` method
dados = economic_brazil.datas_brazil(datas_bcb= data_bcb,
                                     datas_ibge_codigos=data_ibge, 
                                     datas_ibge_link=data_ibge_link, 
                                     datas_ipeadata=data_ipeadata,
                                     save=True,
                                     directory="../dados/economicos_brazil",
                                     data_format="csv")

This parameter will not return a dataframe, only the saved file. This parameter can be used for other methods in the class.

Method datas_banco_central

If you want to download the data from the Banco Central, you can use the method datas_banco_central.

## Imported library
from brazilian_data import EconomicData

## define the start date
DATA_INICIO = "2000-01-01"

## define the dictionary of codes
variaveis_banco_central= {
    "selic": 4189,
    "cambio": 3698,
    "pib_mensal": 4380,
    "igp_m": 189,
    "igp_di": 190,
    "m1": 27788,
}

## define a new object
economic_brazil = EconomicData(codes_banco_central=variaveis_banco_central, 
                                start_date=DATA_INICIO)

## download the data
dados = economic_brazil.datas_banco_central()

dados.head()

	selic	pib_mensal	igp_m	igp_di	m1
Date
2000-01-01	18.94	92576.6	1.24	1.02	NaN
2000-02-01	18.87	91770.4	0.35	0.19	NaN
2000-03-01	18.85	92579.9	0.15	0.18	NaN
2000-04-01	18.62	91376.2	0.23	0.13	NaN
2000-05-01	18.51	98727.0	0.31	0.67	NaN

Method datas_ibge

If you want to download the data from the IBGE, you can use the method datas_ibge.

## Imported library
from brazilian_data import EconomicData

## define the start date
DATA_INICIO = "2010-01-01"

## define the dictionary of codes
variaveis_ibge = {
    "ipca": {
        "codigo": 1737,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "63",
    },
    "custo_m2": {
        "codigo": 2296,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "1198",
    },
    "pesquisa_industrial_mensal": {
        "codigo": 8159,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "11599",
    },
    "pmc_volume": {
        "codigo": 8186,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "11709",
    },
}

## define a new object
economic_brazil = EconomicData(codes_ibge=variaveis_ibge, start_date=DATA_INICIO)

## download the data
dados = economic_brazil.datas_ibge()

dados.head()

	ipca	custo_m2	pesquisa_industrial_mensal
2010-01-01	0.75	NaN	91.20876
2010-02-01	0.78	NaN	88.95149
2010-03-01	0.52	NaN	105.07767
2010-04-01	0.57	NaN	99.30561
2010-05-01	0.43	NaN	104.27978

Method datas_ibge_link

Some codes in codes_ibge do not work, resulting in errors and no files being returned. To address this, codes_ibge_link was created, where you obtain the link to the file from the IBGE. A example of how to get the link is shown here.

## Imported library
from brazilian_data import EconomicData

## define the start date
DATA_INICIO = "2010-01-01"

## define the dictionary of codes
indicadores_ibge_link = {
    "capital_fixo": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela5932.xlsx&terr=N&rank=-&query=t/5932/n1/all/v/6561/p/all/c11255/93406/d/v6561%201/l/v,p%2Bc11255,t",
    "producao_industrial_manufatureira": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela8158.xlsx&terr=N&rank=-&query=t/8158/n1/all/v/11599/p/all/c543/129278/d/v11599%205/l/v,p%2Bc543,t",
    "soja": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela6588.xlsx&terr=N&rank=-&query=t/6588/n1/all/v/35/p/all/c48/0,39443/l/v,p%2Bc48,t",
    "milho_1": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela6588.xlsx&terr=N&rank=-&query=t/6588/n1/all/v/35/p/all/c48/0,39441/l/v,p%2Bc48,t",
    "milho_2": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela6588.xlsx&terr=N&rank=-&query=t/6588/n1/all/v/35/p/all/c48/0,39442/l/v,p%2Bc48,t",
}

## define a new object
economic_brazil = EconomicData(codes_ibge_link=indicadores_ibge_link, 
                     start_date=DATA_INICIO)

## download the data
dados = economic_brazil.datas_ibge_link()

dados.head()

	capital_fixo	producao_industrial_manufatureira	soja	milho_1	milho_2
2010-01-01	12.9	89.55269	66137344.0	33424815.0	17996575.0
2010-02-01	12.9	91.15047	66941524.0	33735243.0	18620733.0
2010-03-01	29.0	114.92197	67350136.0	33486684.0	19095737.0
2010-04-01	29.0	104.58762	67913643.0	33830625.0	19457468.0
2010-05-01	29.0	110.27518	68131230.0	33577605.0	19569483.0

Method datas_ipeadata()

If you want to download data from IPEA, you can use the datas_ipeadata() method.

## Import the `EconomicData` class
from brazilian_data import EconomicData

## Define parameters that will be used in the `datas_brazil` method
DATA_INICIO = "2010-01-01"

codigos_ipeadata= {
    "taja_juros_ltn": "ANBIMA12_TJTLN1212",
    "imposto_renda": "SRF12_IR12",
    "ibovespa": "ANBIMA12_IBVSP12",
    "consumo_energia": "ELETRO12_CEET12",
    "brent_fob": "EIA366_PBRENT366",
}

## define a new object
economic_brazil = EconomicData(codes_ipeadata=codigos_ipeadata, 
                                start_date=DATA_INICIO)

## download the data
dados = economic_brazil.datas_ipeadata()

dados.head()

	taja_juros_ltn	imposto_renda	ibovespa	consumo_energia	brent_fob
2010-01-01	10.454116	22598.711556	-4.65	33360.0	76.600323
2010-02-01	10.525189	11801.791225	1.68	33730.0	73.642143
2010-03-01	10.822181	15204.637209	5.82	35117.0	78.636452
2010-04-01	11.119986	21267.690569	-4.04	35026.0	84.191667
2010-05-01	11.696400	14772.309694	-6.64	34297.0	77.267742

Method datas_fred()

If you want to download data from FRED, you can use the datas_fred() method. On the first use of datas_fred() , a prompt will appear for setting up the FRED API key. To generate the key, follow the steps described here.

After that, you can run the code again and the data will be collected.

## Import the `EconomicData` class
from brazilian_data import EconomicData

## Define parameters that will be used in the `datas_brazil` method
DATA_INICIO = "2010-01-01"

codigos_fred = {
    "nasdaq100": "NASDAQ100",
    "taxa_cambio_efetiva": "RBBRBIS",
    "cboe_nasdaq": "VXNCLS",
    "taxa_juros_interbancaria": "IRSTCI01BRM156N",
    "atividade_economica_eua": "USPHCI",}

## Initialize the `EconomicData` class
economic_brazil = EconomicData(codes_fred=codigos_fred, start_date=DATA_INICIO)

## Download the data
dados = economic_brazil.datas_fred()

dados.head()

	nasdaq100	taxa_cambio_efetiva	cboe_nasdaq	taxa_juros_interbancaria	atividade_economica_eua
2010-01-01	NaN	160.61	NaN	8.75	97.29
2010-02-01	1760.72	157.74	24.33	8.75	97.37
2010-03-01	1846.40	162.82	19.59	8.75	97.50
2010-04-01	1959.56	166.02	18.76	8.80	97.73
2010-05-01	NaN	165.54	NaN	9.50	98.19

EconomicDataAsync Class Documentation

Overview

The EconomicDataAsync class is an asynchronous version of EconomicData, designed for more efficient data collection using async operations. For examples, see economic_data_async_examples.ipynb.

Initialization

__init__(self, codes_banco_central=None, codes_ibge=None, codes_ibge_link=None, codes_ipeadata=None, codes_fred=None, start_date=None)

• Parameters:
  • codes_banco_central (dict): Central Bank codes
  • codes_ibge (dict): IBGE codes
  • codes_ibge_link (dict): IBGE links
  • codes_ipeadata (dict): IPEADATA codes
  • codes_fred (dict): FRED codes
  • start_date (str): Start date for collection

Main Methods

async def datas_brazil_async(self, **kwargs)

Asynchronously collects all configured data.

data_bcb = True
data_ibge = True
data_ibge_link = True
data_ipeadata = True
data_fred = False

economic_brazil = EconomicDataAsync(codes_banco_central=variaveis_banco_central, 
                                 codes_ibge=variaveis_ibge, 
                                 codes_ipeadata=codigos_ipeadata, 
                                 codes_ibge_link=indicadores_ibge_link,
                                 start_date=DATA_INICIO)

dados = economic_brazil.datas_brazil_async(datas_bcb= data_bcb,
                                     datas_ibge_codigos=data_ibge, 
                                     datas_ibge_link=data_ibge_link, 
                                     datas_ipeadata=data_ipeadata,
                                     missing_data=True)

async def datas_banco_central_async(self, save=False, directory=None, data_format="csv")

Asynchronously collects data from the Central Bank.

DATA_INICIO = "2000-01-01"
variaveis_banco_central= {
    "selic": 4189,
    "cambio": 3698,
    "pib_mensal": 4380,
    "igp_m": 189,
    "igp_di": 190,
    "m1": 27788,
}

economic_brazil = EconomicDataAsync(codes_banco_central=variaveis_banco_central, 
                                start_date=DATA_INICIO)

dados = economic_brazil.datas_banco_central_async()

async def datas_ibge_async(self, save=False, directory=None, data_format="csv")

Asynchronously collects data from IBGE.

variaveis_ibge = {
    "ipca": {
        "codigo": 1737,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "63",
    },
    "custo_m2": {
        "codigo": 2296,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "1198",
    },
    "pesquisa_industrial_mensal": {
        "codigo": 8159,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "11599",
    },
    "pmc_volume": {
        "codigo": 8186,
        "territorial_level": "1",
        "ibge_territorial_code": "all",
        "variable": "11709",
    },
}

economic_brazil = EconomicDataAsync(codes_ibge=variaveis_ibge, 
                                start_date=DATA_INICIO)

dados = economic_brazil.datas_ibge_async()

async def datas_ibge_link_async(self, save=False, directory=None, data_format="csv")

Asynchronously collects data from IBGE via links.

DATA_INICIO = "2010-01-01"

indicadores_ibge_link = {
    "capital_fixo": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela5932.xlsx&terr=N&rank=-&query=t/5932/n1/all/v/6561/p/all/c11255/93406/d/v6561%201/l/v,p%2Bc11255,t",
    "producao_industrial_manufatureira": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela8158.xlsx&terr=N&rank=-&query=t/8158/n1/all/v/11599/p/all/c543/129278/d/v11599%205/l/v,p%2Bc543,t",
    "soja": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela6588.xlsx&terr=N&rank=-&query=t/6588/n1/all/v/35/p/all/c48/0,39443/l/v,p%2Bc48,t",
    "milho_1": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela6588.xlsx&terr=N&rank=-&query=t/6588/n1/all/v/35/p/all/c48/0,39441/l/v,p%2Bc48,t",
    "milho_2": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela6588.xlsx&terr=N&rank=-&query=t/6588/n1/all/v/35/p/all/c48/0,39442/l/v,p%2Bc48,t",
    "pib": "https://sidra.ibge.gov.br/geratabela?format=xlsx&name=tabela5932.xlsx&terr=N&rank=-&query=t/5932/n1/all/v/6564/p/all/c11255/90707/d/v6564%201/l/v,p,t%2Bc11255&verUFs=false&verComplementos2=false&verComplementos1=false&omitirIndentacao=false&abreviarRotulos=false&exibirNotas=false&agruparNoCabecalho=false",
}

economic_brazil = EconomicDataAsync(codes_ibge_link=indicadores_ibge_link, 
                     start_date=DATA_INICIO)

dados = economic_brazil.datas_ibge_link_async()

async def datas_ipeadata_async(self, save=False, directory=None, data_format="csv")

Asynchronously collects data from IPEADATA.

DATA_INICIO = "2010-01-01"

codigos_ipeadata= {
    "taja_juros_ltn": "ANBIMA12_TJTLN1212",
    "imposto_renda": "SRF12_IR12",
    "ibovespa": "ANBIMA12_IBVSP12",
    "consumo_energia": "ELETRO12_CEET12",
    "brent_fob": "EIA366_PBRENT366",
}


economic_brazil = EconomicDataAsync(codes_ipeadata=codigos_ipeadata, 
                                start_date=DATA_INICIO)

dados = economic_brazil.datas_ipeadata_async()

async def datas_fred_async(self, save=False, directory=None, data_format="csv")

Asynchronously collects data from FRED.

DATA_INICIO = "2010-01-01"
codigos_fred = {
    "nasdaq100": "NASDAQ100",
    "taxa_cambio_efetiva": "RBBRBIS",
    "cboe_nasdaq": "VXNCLS",
    "taxa_juros_interbancaria": "IRSTCI01BRM156N",
    "atividade_economica_eua": "USPHCI",}

economic_brazil = EconomicDataAsync(codes_fred=codigos_fred, start_date=DATA_INICIO)

dados = economic_brazil.datas_fred_async()

Usage Example with Jupyter Notebook

import nest_asyncio
nest_asyncio.apply()

from brazilian_data import EconomicDataAsync

data_bcb = True
data_ibge = True
data_ibge_link = True
data_ipeadata = True
data_fred = False
economic_brazil = EconomicDataAsync(codes_banco_central=variaveis_banco_central, 
                                 codes_ibge=variaveis_ibge, 
                                 codes_ipeadata=codigos_ipeadata, 
                                 codes_ibge_link=indicadores_ibge_link,
                                 start_date=DATA_INICIO)

dados = economic_brazil.datas_brazil_async(datas_bcb= data_bcb,
                                     datas_ibge_codigos=data_ibge, 
                                     datas_ibge_link=data_ibge_link, 
                                     datas_ipeadata=data_ipeadata,
                                     missing_data=True)

Performance Comparison

Using the %%timeit magic command in Jupyter Notebook, we compared the execution time between synchronous and asynchronous methods. The results demonstrate significant performance improvements:

Synchronous Version

%%timeit
economic_brazil = EconomicData(
    codes_banco_central=central_bank_variables, 
    codes_ibge=ibge_variables, 
    codes_ipeadata=ipeadata_codes, 
    codes_ibge_link=ibge_link_indicators,
    start_date=START_DATE
)

data = economic_brazil.datas_brazil(
    datas_bcb=True,
    datas_ibge_codigos=True, 
    datas_ibge_link=True, 
    datas_ipeadata=True,
    missing_data=True
)

1min 18s ± 27.4s per loop (mean ± std. dev. of 7 runs, 1 loop each)

Asynchronous Version

%%timeit
economic_brazil = EconomicDataAsync(
    codes_banco_central=central_bank_variables, 
    codes_ibge=ibge_variables, 
    codes_ipeadata=ipeadata_codes, 
    codes_ibge_link=ibge_link_indicators,
    start_date=START_DATE
)

data = economic_brazil.datas_brazil_async(
    datas_bcb=True,
    datas_ibge_codigos=True, 
    datas_ibge_link=True, 
    datas_ipeadata=True,
    missing_data=True
)

12.6s ± 2.5s per loop (mean ± std. dev. of 7 runs, 1 loop each)

Performance Improvement

• The asynchronous version is approximately 84% faster
• Average time reduction: 65.4 seconds
• More consistent performance (lower standard deviation)
• Better resource utilization through concurrent operations

Notes

• Requires Python 3.10+
• Use nest_asyncio in Jupyter notebooks
• Compatible with all sources from the sync version

Contributing

All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome.

License

MIT License