EIA Bulk Data Collection package.
Project description
EIA Bulk Data Collector
EIA Bulk Data Collector is a Python library designed to interact with the U.S. Energy Information Administration (EIA) API to collect bulk energy-related data for analysis and research purposes. This library simplifies the process of fetching, organizing, and processing large datasets from the EIA API.
Features
- Collect bulk data from the EIA API
- Automatically handle API requests and rate limits
- Parse and store data in JSON(limited), DataFrames, Databases.
- Support for all publicly available EIA data. More information about the EIA API found here: https://www.eia.gov.
- Easy-to-use interface for querying and downloading data
- Extensive error catching for ease of use.
Installation
You can install the eia-bulk package using pip:
pip install eia-bulk
Getting Started
1. Set Up API Key and Database Information (Note: The database set up is not required)
To interact with the EIA API, you will need an API key. You can obtain an API key from the EIA website.
Once you have your API key, store it in a .env file in your project directory:
EIA_KEY=your_api_key_here
HOST=db_host_here
DBNAME=db_name_here
USERNAME=db_username_here
PASSWORD=db_password_here
TABLENAME=db_tablename_here
2. Example Usage
from eia import EIABulk as ebulk
Getting route information
Example with
eia = ebulk(write2db=False, route='electricity/rto/fuel-type-data', from_api=True, verbose=2)
print(eia)
Click to view output.
{
"response": {
"id": "fuel-type-data",
"name": "Hourly Generation by Energy Source",
"description": "Hourly net generation by balancing authority and energy source.
Source: Form EIA-930
Product: Hourly Electric Grid Monitor",
"frequency": [
{
"id": "hourly",
"alias": "hourly (UTC)",
"description": "One data point for each hour in UTC time.",
"query": "H",
"format": "YYYY-MM-DD\"T\"HH24"
},
{
"id": "local-hourly",
"alias": "hourly (Local Time Zone)",
"description": "One data point for each hour in local time.",
"query": "LH",
"format": "YYYY-MM-DD\"T\"HH24TZH"
}
],
"facets": [
{
"id": "respondent",
"description": "Balancing Authority / Region"
},
{
"id": "fueltype",
"description": "Energy Source"
}
],
"data": {
"value": {
"aggregation-method": "SUM",
"alias": "Net Generation",
"units": "megawatthours"
}
},
"startPeriod": "2019-01-01T00",
"endPeriod": "2024-12-16T06",
"defaultDateFormat": "YYYY-MM-DD\"T\"HH24",
"defaultFrequency": "hourly"
},
"request": {
"command": "/v2/electricity/rto/fuel-type-data/",
"params": {
"api_key": "api_key_here_1234567890"
}
},
"apiVersion": "2.1.8",
"ExcelAddInVersion": "2.1.0"
}
eia = ebulk(write2db=False, route='electricity/rto', facet_types=[('respondent', 'US48')], frequency='hourly', data_types=['value'], override=True, verbose=2, from_api=False)
print(eia)
Click to view output.
{
"region-data": {
"facets": {
"respondent": [
"GWA",
"TEX",
"BANC",
"GVL",
"FPL",
"..."
],
"type": [
"CO2.IMPR",
"CO2.EM",
"TI",
"CO2.EXPR",
"NG",
"..."
]
},
"frequency": [
"hourly",
"local-hourly"
],
"data": [
"value"
],
"sort_by": [
"period",
"respondent",
"type",
"value"
]
},
"fuel-type-data": {
"facets": {
"respondent": [
"CAR",
"CAL",
"NYIS",
"NWMT",
"PSEI",
"..."
],
"fueltype": [
"UES",
"PS",
"NG",
"OTH",
"WND",
"..."
]
},
"frequency": [
"hourly",
"local-hourly"
],
"data": [
"value"
],
"sort_by": [
"period",
"respondent",
"fueltype",
"value"
]
},
"region-sub-ba-data": {
"facets": {
"subba": [
"SPRM",
"VEA",
"PS",
"4001",
"4007",
"..."
],
"parent": [
"PJM",
"NYIS",
"MISO",
"SWPP",
"CISO",
"..."
]
},
"frequency": [
"hourly",
"local-hourly"
],
"data": [
"value"
],
"sort_by": [
"period",
"subba",
"parent",
"value"
]
},
"interchange-data": {
"facets": {
"fromba": [
"CAL",
"GVL",
"FPL",
"TEX",
"PJM",
"..."
],
"toba": [
"IESO",
"FPL",
"GVL",
"TEX",
"PJM",
"..."
]
},
"frequency": [
"hourly",
"local-hourly"
],
"data": [
"value"
],
"sort_by": [
"period",
"fromba",
"toba",
"value"
]
},
"daily-region-data": {
"facets": {
"respondent": [
"PJM",
"FPL",
"NY",
"TEX",
"EPE",
"..."
],
"type": [
"NG",
"CO2.IMPR",
"CO2.CON",
"CO2.EM",
"CO2.GER",
"..."
],
"timezone": [
"Eastern",
"Mountain",
"Central",
"Pacific",
"Arizona"
]
},
"frequency": [
"daily"
],
"data": [
"value"
],
"sort_by": [
"period",
"respondent",
"type",
"timezone",
"value"
]
},
"daily-region-sub-ba-data": {
"facets": {
"subba": [
"4001",
"4003",
"4005",
"4006",
"4008",
"..."
],
"parent": [
"PJM",
"MISO",
"SWPP",
"ISNE",
"ERCO",
"..."
],
"timezone": [
"Pacific",
"Central",
"Mountain",
"Eastern",
"Arizona"
]
},
"frequency": [
"daily"
],
"data": [
"value"
],
"sort_by": [
"period",
"subba",
"parent",
"timezone",
"value"
]
},
"daily-fuel-type-data": {
"facets": {
"respondent": [
"BANC",
"FPL",
"CAL",
"PJM",
"WWA",
"..."
],
"fueltype": [
"UES",
"PS",
"NG",
"NUC",
"OTH",
"..."
],
"timezone": [
"Mountain",
"Central",
"Arizona",
"Eastern",
"Pacific"
]
},
"frequency": [
"daily"
],
"data": [
"value"
],
"sort_by": [
"period",
"respondent",
"fueltype",
"timezone",
"value"
]
},
"daily-interchange-data": {
"facets": {
"fromba": [
"FPL",
"GVL",
"GWA",
"TEX",
"PJM",
"..."
],
"toba": [
"CEN",
"BANC",
"FPL",
"GVL",
"GWA",
"..."
],
"timezone": [
"Central",
"Pacific",
"Arizona",
"Mountain",
"Eastern"
]
},
"frequency": [
"daily"
],
"data": [
"value"
],
"sort_by": [
"period",
"fromba",
"toba",
"timezone",
"value"
]
},
"emissions-data-by-fuel-type": {
"facets": {
"respondent": [
"SPA",
"IPCO",
"NW",
"MISO",
"SOCO",
"..."
],
"datatype": [
"CO2.EM"
],
"fueltype": [
"OTH",
"OIL",
"NG",
"COL"
]
},
"frequency": [
"hourly",
"local-hourly"
],
"data": [
"value"
],
"sort_by": [
"period",
"respondent",
"datatype",
"fueltype",
"value"
]
},
"daily-emissions-by-fuel-type-data": {
"facets": {
"respondent": [
"GVL",
"FMPP",
"CENT",
"SWPP",
"IPCO",
"..."
],
"datatype": [
"CO2.EM"
],
"fueltype": [
"COL",
"OTH",
"OIL",
"NG"
],
"timezone": [
"Central",
"Eastern",
"Pacific",
"Mountain",
"Arizona"
]
},
"frequency": [
"daily"
],
"data": [
"value"
],
"sort_by": [
"period",
"respondent",
"datatype",
"fueltype",
"timezone",
"..."
]
}
}
response = ebulk(write2db=False, route='electricity/rto/fuel-type-data', facet_types=[('respondent', 'US48')], frequency='hourly', data_types=['value'], override=True, verbose=2, from_api=False).route_endpoints()
print(response)
Click to view output.
The following are the potential routes/features one can pick from given endpoint: electricity/rto/fuel-type-data
{
"frequency": [
"hourly",
"local-hourly"
],
"data": [
"value"
],
"facets": {
"respondent": [
"CAR",
"CAL",
"NYIS",
"NWMT",
"PSEI",
"NY",
"FMPP",
"MIDA",
"GVL",
"TEPC",
"LDWP",
"SCEG",
"CISO",
"NEVP",
"DUK",
"SC",
"WAUW",
"GCPD",
"IID",
"PSCO",
"SW",
"JEA",
"PNM",
"PACE",
"TVA",
"FPL",
"US48",
"TIDC",
"MISO",
"CENT",
"ERCO",
"NE",
"TEC",
"PGE",
"HST",
"BANC",
"TPWR",
"PJM",
"WALC",
"PACW",
"CHPD",
"YAD",
"EPE",
"SEC",
"DEAA",
"FPC",
"AVRN",
"ISNE",
"NW",
"SWPP",
"WWA",
"FLA",
"WACM",
"CPLE",
"MIDW",
"GRID",
"AECI",
"SEPA",
"TEN",
"AVA",
"TEX",
"AZPS",
"IPCO",
"SPA",
"AEC",
"SE",
"CPLW",
"TAL",
"GRIF",
"SCL",
"HGMA",
"EEI",
"GLHB",
"SRP",
"DOPD",
"BPAT",
"LGEE",
"GWA",
"NSB",
"SOCO"
],
"fueltype": [
"UES",
"PS",
"NG",
"OTH",
"WND",
"NUC",
"BAT",
"SUN",
"OIL",
"UES",
"SNB",
"WAT",
"BAT",
"COL",
"UNK",
"SNB",
"PS"
]
},
"sort_by": [
"period",
"respondent",
"fueltype",
"value"
]
}
Getting dataframe
eia = ebulk(write2db=False, route='electricity/rto/fuel-type-data', frequency='hourly', facet_types = ('respondent', 'US48'), data_types='value', override=True, verbose=2, multiplier=1/2)
for df in eia.collect_months_data(2019, [1,2]):
print(df.head(25).to_string(index=False))
Click to view output.
Collecting from 2019-01-01 to 2019-01-16
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-01-01T00&end=2019-01-16T00&offset=0&length=5000&api_key=api_key_here_1234567890
Collecting from 2019-01-16 to 2019-02-01
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-01-16T00&end=2019-02-01T00&offset=0&length=5000&api_key=api_key_here_1234567890
Collecting from 2019-02-01 to 2019-02-15
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-02-01T00&end=2019-02-15T00&offset=0&length=5000&api_key=api_key_here_1234567890
Collecting from 2019-02-15 to 2019-03-01
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-02-15T00&end=2019-03-01T00&offset=0&length=5000&api_key=api_key_here_1234567890
| period | respondent | respondent-name | fueltype | type-name | value | value-units |
|---|---|---|---|---|---|---|
| 2019-01-16T00 | US48 | United States Lower 48 | COL | Coal | 151102 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | NG | Natural Gas | 171401 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | NUC | Nuclear | 103615 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | OIL | Petroleum | 363 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | OTH | Other | 9439 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | SUN | Solar | 1424 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | WAT | Hydro | 41590 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | WND | Wind | 24323 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | COL | Coal | 146397 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | NG | Natural Gas | 165141 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | NUC | Nuclear | 103696 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | OIL | Petroleum | 370 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | OTH | Other | 9250 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | SUN | Solar | 3577 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | WAT | Hydro | 37472 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | WND | Wind | 23474 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | COL | Coal | 141832 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | NG | Natural Gas | 158337 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | NUC | Nuclear | 103679 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | OIL | Petroleum | 362 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | OTH | Other | 9119 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | SUN | Solar | 5945 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | WAT | Hydro | 35440 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | WND | Wind | 23511 | megawatthours |
| 2019-01-15T21 | US48 | United States Lower 48 | COL | Coal | 140873 | megawatthours |
eia = ebulk(write2db=False, route='electricity/operating-generator-capacity', facet_types=[('balancing_authority_code', 'CPLE'),('balancing_authority_code', 'AEC')], data_types=['nameplate-capacity-mw'], override=True, verbose=0, multiplier=4)
for df in eia.collect_years_data(start_year=2019, end_year=2020):
print(df.head(25).to_string(index=False))
Click to view output.
| period | stateid | stateName | sector | sectorName | entityid | entityName | plantid | plantName | generatorid | technology | energy_source_code | energy-source-desc | prime_mover_code | balancing_authority_code | balancing-authority-name | status | statusDescription | nameplate-capacity-mw | unit | nameplate-capacity-mw-units |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2019-02 | NC | North Carolina | ipp-non-chp | IPP Non-CHP | 58921 | Bailey Farm LLC | 59127 | Bailey Farm LLC | 1 | Solar Photovoltaic | SUN | Solar | PV | CPLE | Duke Energy Progress East | OP | Operating | 5 | None | MW |
| 2019-01 | NC | North Carolina | industrial-chp | Industrial CHP | 58536 | Smithfield Packaged Meats Corp | 58575 | Smithfield Packaged Meats Corp. | 4 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | OP | Operating | 2.3 | None | MW |
| 2019-01 | NC | North Carolina | ipp-chp | IPP CHP | 18065 | Smithfield Packaged Meats Corp. | 56035 | Smithfield Packaged Meats Corp. | 1 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.4 | None | MW |
| 2019-01 | NC | North Carolina | ipp-chp | IPP CHP | 18065 | Smithfield Packaged Meats Corp. | 56035 | Smithfield Packaged Meats Corp. | 2 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.4 | None | MW |
| 2019-01 | NC | North Carolina | ipp-non-chp | IPP Non-CHP | 58927 | Snow Hill Solar 2 LLC | 59106 | Snow Hill Solar 2 LLC | 1 | Solar Photovoltaic | SUN | Solar | PV | CPLE | Duke Energy Progress East | OP | Operating | 2 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN2 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN3 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN4 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN5 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN6 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN7 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN8 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN9 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN10 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN11 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-04 | SC | South Carolina | industrial-chp | Industrial CHP | 50006 | Invista | 10795 | Camden South Carolina | GEN1 | Natural Gas Steam Turbine | NG | Natural Gas | ST | CPLE | Duke Energy Progress East | OP | Operating | 5.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GEN8 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GEN9 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GN10 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GN11 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN12 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN13 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN14 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN15 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN16 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
Writing to database
eia = ebulk(write2db=True, route='electricity/rto/fuel-type-data', frequency='hourly', facet_types = ('respondent', 'US48'), data_types='value', override=True, verbose=2, multiplier=1/2)
for df in eia.collect_months_data(2019, [1,2]):
print(df.head(25).to_string(index=False))
Click to view output.
Collecting from 2019-01-01 to 2019-01-16
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-01-01T00&end=2019-01-16T00&offset=0&length=5000&api_key=api_key_here_1234567890
Collecting from 2019-01-16 to 2019-02-01
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-01-16T00&end=2019-02-01T00&offset=0&length=5000&api_key=api_key_here_1234567890
Collecting from 2019-02-01 to 2019-02-15
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-02-01T00&end=2019-02-15T00&offset=0&length=5000&api_key=api_key_here_1234567890
Collecting from 2019-02-15 to 2019-03-01
API URL:https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/?frequency=hourly&data[0]=value&facets[respondent][]=US48&start=2019-02-15T00&end=2019-03-01T00&offset=0&length=5000&api_key=api_key_here_1234567890
| period | respondent | respondent-name | fueltype | type-name | value | value-units |
|---|---|---|---|---|---|---|
| 2019-01-16T00 | US48 | United States Lower 48 | COL | Coal | 151102 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | NG | Natural Gas | 171401 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | NUC | Nuclear | 103615 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | OIL | Petroleum | 363 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | OTH | Other | 9439 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | SUN | Solar | 1424 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | WAT | Hydro | 41590 | megawatthours |
| 2019-01-16T00 | US48 | United States Lower 48 | WND | Wind | 24323 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | COL | Coal | 146397 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | NG | Natural Gas | 165141 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | NUC | Nuclear | 103696 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | OIL | Petroleum | 370 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | OTH | Other | 9250 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | SUN | Solar | 3577 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | WAT | Hydro | 37472 | megawatthours |
| 2019-01-15T23 | US48 | United States Lower 48 | WND | Wind | 23474 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | COL | Coal | 141832 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | NG | Natural Gas | 158337 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | NUC | Nuclear | 103679 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | OIL | Petroleum | 362 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | OTH | Other | 9119 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | SUN | Solar | 5945 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | WAT | Hydro | 35440 | megawatthours |
| 2019-01-15T22 | US48 | United States Lower 48 | WND | Wind | 23511 | megawatthours |
| 2019-01-15T21 | US48 | United States Lower 48 | COL | Coal | 140873 | megawatthours |
eia = ebulk(route='electricity/operating-generator-capacity', facet_types=[('balancing_authority_code', 'CPLE'),('balancing_authority_code', 'AEC')], data_types=['nameplate-capacity-mw'], override=True, verbose=0, multiplier=4)
for df in eia.collect_years_data(start_year=2019, end_year=2020):
print(df.head(25).to_string(index=False))
Click to view output.
| period | stateid | stateName | sector | sectorName | entityid | entityName | plantid | plantName | generatorid | technology | energy_source_code | energy-source-desc | prime_mover_code | balancing_authority_code | balancing-authority-name | status | statusDescription | nameplate-capacity-mw | unit | nameplate-capacity-mw-units |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2019-02 | NC | North Carolina | ipp-non-chp | IPP Non-CHP | 58921 | Bailey Farm LLC | 59127 | Bailey Farm LLC | 1 | Solar Photovoltaic | SUN | Solar | PV | CPLE | Duke Energy Progress East | OP | Operating | 5 | None | MW |
| 2019-01 | NC | North Carolina | industrial-chp | Industrial CHP | 58536 | Smithfield Packaged Meats Corp | 58575 | Smithfield Packaged Meats Corp. | 4 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | OP | Operating | 2.3 | None | MW |
| 2019-01 | NC | North Carolina | ipp-chp | IPP CHP | 18065 | Smithfield Packaged Meats Corp. | 56035 | Smithfield Packaged Meats Corp. | 1 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.4 | None | MW |
| 2019-01 | NC | North Carolina | ipp-chp | IPP CHP | 18065 | Smithfield Packaged Meats Corp. | 56035 | Smithfield Packaged Meats Corp. | 2 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.4 | None | MW |
| 2019-01 | NC | North Carolina | ipp-non-chp | IPP Non-CHP | 58927 | Snow Hill Solar 2 LLC | 59106 | Snow Hill Solar 2 LLC | 1 | Solar Photovoltaic | SUN | Solar | PV | CPLE | Duke Energy Progress East | OP | Operating | 2 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN2 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN3 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN4 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN5 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN6 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN7 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN8 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GEN9 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN10 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN11 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-04 | SC | South Carolina | industrial-chp | Industrial CHP | 50006 | Invista | 10795 | Camden South Carolina | GEN1 | Natural Gas Steam Turbine | NG | Natural Gas | ST | CPLE | Duke Energy Progress East | OP | Operating | 5.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GEN8 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GEN9 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GN10 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-02 | NC | North Carolina | industrial-chp | Industrial CHP | 23815 | Blue Ridge Paper Products Inc | 50244 | Canton North Carolina | GN11 | Conventional Steam Coal | BIT | Bituminous Coal | ST | CPLE | Duke Energy Progress East | OP | Operating | 7.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN12 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN13 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN14 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN15 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
| 2019-05 | NC | North Carolina | industrial-non-chp | Industrial Non-CHP | 17449 | Smithfield Fresh Meats Corp. | 54823 | Smithfield Farmland Corp Bladen | GN16 | Petroleum Liquids | DFO | Disillate Fuel Oil | IC | CPLE | Duke Energy Progress East | SB | Standby/Backup: available for service but not normally used | 1.5 | None | MW |
Key Instance Variables
api_key: Stores the EIA API authentication key route: Constructed API route for data retrieval facets: Filtered categories or facets data: Specific data types to retrieve frequency: Data frequency (hourly, daily, monthly, etc.) write2db: Boolean to determine if data should be written to database verbose: Controls the level of logging and output multiplier: Adjusts time range for data retrieval based on frequency
Key Methods and Their Purposes
1. route_endpoints() or str()
Provides information about available routes, facets, and data types Can fetch information from API or from local route tree Purpose: Explore and understand available API endpoints
Other Relevant Variables:
from_api(Default True): If True retrieves infromation from the API, if false retrieves data from one of the tree json files.
2. request_data(start, end, as_df=True)
Retrieves data from the EIA API for a specified date range Parameters:
start: Start date for data retrieval end: End date for data retrieval as_df: Whether to return data as a pandas DataFrame
Purpose: Fetch data from EIA API with flexible output options
3. collect_year_small_data(year, engine, months=None)
Collects data for a specific year, typically for hourly or daily frequencies Parameters:
year: Year to collect data for engine: Database connection engine months(Optional): specific months to collect
Purpose: Retrieve and optionally store small-granularity time series data
4. collect_year_large_data(the_range, engine)
Collects data for larger time frequencies (monthly, quarterly, yearly) Parameters: the_range: Range of years to collect engine: Database connection engine
Purpose: Retrieve and store data for less frequent time series
5. collect_years_data(years=None, start_year=None, end_year=None)
Flexible method to collect data across multiple years Can specify exact years or a range Handles different data frequencies Purpose: Comprehensive data collection across multiple years
6. update_tree()
Updates the internal route and API structure information Builds a comprehensive dictionary of available API routes, facets, and data types Purpose: Maintain an up-to-date understanding of the EIA API structure
7. url_constructor(start, end)
Builds the URL for API data retrieval Incorporates route, frequency, data types, facets, and date range Purpose: Construct precise API request URLs
8. collect_month_of_small_data(year, month)
Purpose: Collects the data for exactly one month generally with smaller time frequencies.
9. collect_months_of_large(dates)
Parameters:
dates: list or touple that contains two date strings
Purpose: Collects larger time frequency data within the given date range.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
eia_bulk-0.1.1.tar.gz
(3.2 MB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file eia_bulk-0.1.1.tar.gz.
File metadata
- Download URL: eia_bulk-0.1.1.tar.gz
- Upload date:
- Size: 3.2 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.0.1 CPython/3.12.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2743b4b4484fca8b687b0dab27b973f937fb944bafa78bd2e0048abb08d4d4e8
|
|
| MD5 |
2b4af774e5af3a4c92c6135c86d845a7
|
|
| BLAKE2b-256 |
dc4c1f6d8090354c26e94f166c7dc7f67c4bb6bc7bea89a2deb9394c5655b33b
|
File details
Details for the file eia_bulk-0.1.1-py3-none-any.whl.
File metadata
- Download URL: eia_bulk-0.1.1-py3-none-any.whl
- Upload date:
- Size: 3.4 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.0.1 CPython/3.12.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a6a809ed8d745066a103e6bf63d147a3fe10e04e92a3cac799300dee33e3c9bc
|
|
| MD5 |
2bca0b26d87bf0012d0194512ce08cb2
|
|
| BLAKE2b-256 |
78789c552c7a53f94884a8da2b578ecd0a12d07469c17b23a6cf5ae72842c68f
|
