ElexonDataPortal 2.0.16

No project description provided

Elexon Data Portal

The ElexonDataPortal library is a Python Client for retrieving data from the Elexon/BMRS API. The library significantly reduces the complexity of interfacing with the Elexon/BMRS API through the standardisation of parameter names and orchestration of multiple queries when making requests over a date range. To use the ElexonDataPortal you will have to register for an Elexon API key which can be done here.

Installation

The library can be easily installed from PyPi, this can be done using:

pip install ElexonDataPortal

Getting Started

We'll begin by initialising the API Client. The key parameter to pass here is the api_key, alternatively this can be set by specifying the environment variable BMRS_API_KEY which will then be loaded automatically.

from ElexonDataPortal import api

client = api.Client('your_api_key_here')

Now that the client has been initialised we can make a request!

One of the key abstractions within the ElexonDataPortal library is the handling of multiple requests over a date range specified through the start_date and end_date parameters. Each response will be automatically cleaned and parsed, then concatenated into a single Pandas DataFrame. If a settlement period and date column can be identified in the returned data then a new column will be added with the local datetime for each data-point. N.b. that if passed as a string the start and end datetimes will be assumed to be in the local timezone for the UK

start_date = '2020-01-01'
end_date = '2020-01-01 1:30'

df_B1610 = client.get_B1610(start_date, end_date)

df_B1610.head(3)

	documentType	businessType	processType	timeSeriesID	curveType	settlementDate	powerSystemResourceType	registeredResourceEICCode	marketGenerationUnitEICCode	marketGenerationBMUId	marketGenerationNGCBMUId	bMUnitID	nGCBMUnitID	activeFlag	documentID	documentRevNum	resolution	start	end	settlementPeriod	quantity	local_datetime
0	Actual generation	Production	Realised	ELX-EMFIP-AGOG-TS-212	Sequential fixed size block	2020-01-01	Generation	48W000CAS-BEU01F	48W000CAS-BEU01F	M_CAS-BEU01	CAS-BEU01	M_CAS-BEU01	CAS-BEU01	Y	ELX-EMFIP-AGOG-22495386	1	PT30M	2020-01-01	2020-01-01	1	18.508	2020-01-01 00:00:00+00:00
1	Actual generation	Production	Realised	ELX-EMFIP-AGOG-TS-355	Sequential fixed size block	2020-01-01	Generation	48W00000STLGW-3A	48W00000STLGW-3A	T_STLGW-3	STLGW-3	T_STLGW-3	STLGW-3	Y	ELX-EMFIP-AGOG-22495386	1	PT30M	2020-01-01	2020-01-01	1	28.218	2020-01-01 00:00:00+00:00
2	Actual generation	Production	Realised	ELX-EMFIP-AGOG-TS-278	Sequential fixed size block	2020-01-01	Generation	48W00000GNFSW-1H	48W00000GNFSW-1H	T_GNFSW-1	GNFSW-1	T_GNFSW-1	GNFSW-1	Y	ELX-EMFIP-AGOG-22495386	1	PT30M	2020-01-01	2020-01-01	1	29.44	2020-01-01 00:00:00+00:00

If you've previously written your own code for extracting data from the Elexon/BMRS API then you may be wondering where some of the normal parameters you pass have gone. The reduction in the parameters passed are due to 4 core drivers:

• Standardisation of date range parameter names
• Removal of the need to specify ServiceType
• Automatic passing of APIKey after client initialisation
• Shipped with sensible defaults for all remaining parameters

The full list of data streams that are able to be requested can be found here. If you wish to make requests using the raw methods these are available through the ElexonDataportal.dev.raw module.

Further information can be found in the Quick Start guide.

What's Changed in v2

The latest release of the library includes a full rewrite of the code-base. We have endeavoured to make the new API as intuitive as possible but that has required breaking changes from v1, if you wish to continue using the historic library use pip install ElexonDataPortal==1.0.4. N.b v1 will not be maintained going forward, you are advised to change over to v2.0.0+.

The key feature changes are:

• Coverage of more BMRS streams
• Automated default values
• Cleaner client API
• A larger range of request types are compatible with the date range orchestrator

Programmatic Library Generation

One of the core features within the ElexonDataPortal library is that it is self-generating, by which we mean it can rebuild itself (including any new API request methods) from scratch using only the endpoints.csv spreadsheet. As well as generating the Python Client library a BMRS_API.yaml file is created, this provides an OpenAPI specification representation of the Elexon/BMRS API. In turn this allows us to automatically generate documentation, as well as run tests on the API itself to ensure that everything is working as expected - during this process we identified and corrected several small errors in the API documentation provided by Elexon.

To rebuild the library simply run the following in the root directory:

python -m ElexonDataPortal.rebuild

N.b. If you wish to develop the library further or use any of the programmatic library generation functionality then please install the development version of the library using:

pip install ElexonDataPortal[dev]

If you are not installing into a fresh environment it is recommended you install pyyaml and geopandas using conda to avoid any dependency conflicts. In future we are looking to release ElexonDataPortal as a conda package to avoid these issues.

Data Stream Descriptions

The following table describes the data streams that are currently retreivable through the API. The client method to retrieve data from a given stream follows the naming convention get_{stream-name}.

Stream	Description
B0610	Actual Total Load per Bidding Zone
B0620	Day-Ahead Total Load Forecast per Bidding Zone
B0630	Week-Ahead Total Load Forecast per Bidding Zone
B0640	Month-Ahead Total Load Forecast Per Bidding Zone
B0650	Year Ahead Total Load Forecast per Bidding Zone
B0710	Planned Unavailability of Consumption Units
B0720	Changes In Actual Availability Of Consumption Units
B0810	Year Ahead Forecast Margin
B0910	Expansion and Dismantling Projects
B1010	Planned Unavailability In The Transmission Grid
B1020	Changes In Actual Availability In The Transmission Grid
B1030	Changes In Actual Availability of Offshore Grid Infrastructure
B1320	Congestion Management Measures Countertrading
B1330	Congestion Management Measures Costs of Congestion Management
B1410	Installed Generation Capacity Aggregated
B1420	Installed Generation Capacity per Unit
B1430	Day-Ahead Aggregated Generation
B1440	Generation forecasts for Wind and Solar
B1510	Planned Unavailability of Generation Units
B1520	Changes In Actual Availability of Generation Units
B1530	Planned Unavailability of Production Units
B1540	Changes In Actual Availability of Production Units
B1610	Actual Generation Output per Generation Unit
B1620	Actual Aggregated Generation per Type
B1630	Actual Or Estimated Wind and Solar Power Generation
B1720	Amount Of Balancing Reserves Under Contract Service
B1730	Prices Of Procured Balancing Reserves Service
B1740	Accepted Aggregated Offers
B1750	Activated Balancing Energy
B1760	Prices Of Activated Balancing Energy
B1770	Imbalance Prices
B1780	Aggregated Imbalance Volumes
B1790	Financial Expenses and Income For Balancing
B1810	Cross-Border Balancing Volumes of Exchanged Bids and Offers
B1820	Cross-Border Balancing Prices
B1830	Cross-border Balancing Energy Activated
BOD	Bid Offer Level Data
CDN	Credit Default Notice Data
DERSYSDATA	Derived System Data
DETSYSPRICES	Detailed System Prices
DEVINDOD	Daily Energy Volume Data
DISBSAD	Balancing Services Adjustment Action Data
FORDAYDEM	Forecast Day and Day Ahead Demand Data
FREQ	Rolling System Frequency
FUELHH	Half Hourly Outturn Generation by Fuel Type
MELIMBALNGC	Forecast Day and Day Ahead Margin and Imbalance Data
MID	Market Index Data
MessageDetailRetrieval	REMIT Flow - Message List Retrieval
MessageListRetrieval	REMIT Flow - Message List Retrieval
NETBSAD	Balancing Service Adjustment Data
NONBM	Non BM STOR Instructed Volume Data
PHYBMDATA	Physical Data
SYSDEM	System Demand
SYSWARN	System Warnings
TEMP	Temperature Data
WINDFORFUELHH	Wind Generation Forecast and Out-turn Data