entsoe-client 0.1.0

Formulate human-readable queries and retrieve data from ENTSO-E into pandas.DataFrame format.

ENTSO-E Client enables straight-forward access to all of the data at ENTSO-E Transparency Platform.

• Query templates abstract the API specifics through Enumerated types.

• Parse responses into Pandas DataFrames without loss of any information.

The separation of Queries, Client and Parser with their hierarchical abstractions keep the package extensible and maintainable. A pipeline from Query to DataFrame is trivial, preserving the ability to customize steps in between.

The implementation relies primarily on the Transparency Platform restful API - user guide. The Manual of Procedures (MoP) documents provide further insight on the business requirements specification. Further information can be found in the Electronic Data Interchange (EDI) Library.

---

Main contributions

• Exhaustive List of ParameterTypes.

  These allow mapping between natural language and the codes required for GET requests, e.g. DocumentType.A85 == DocumentType("Imbalance price"). This feature allows keeping track of queries without jumping between documents or adding comments.

• Exhaustive List of Pre-defined Queries from ENTSO-E API Guide.

  ENTSO-E API Guide is a minial set for any API connector to implement and reflects all dashboards on ENTSO-E Transparency Platform.

• Parsers

  Response Documents come in XML schema which can be parsed into pandas DataFrames.

  Implemented: GL_MarketDocuments, TransmissionNetwork_MarketDocuments, Publication_MarketDocuments and Balancing_MarketDocuments.

  Missing: Outages, Congestion Management and System Operations.

Nevertheless, ENTSO-E Client seeks to be minimal to go from Query to DataFrame and requires domain- knowledge on how to formulate queries and interpret various columns of a parsed response.

---

ENTSO-E relies on many codes (Type) to map to desired queries. Types are encoded in Enum classes with a .help() function to list the all. They can be addressed through Type[code] or Type(string), making interaction easy. HTTP requests and responses usually require the code, whereas we want to formulate the query as a human-readable string.

from entsoe_client import Queries
from entsoe_client.ParameterTypes import *

Queries.Transmission.CapacityAllocatedOutsideEU(
        out_Domain=Area.SK,
        in_Domain=Area.UA_BEI,
        marketAgreementType=MarketAgreementType('Daily'), # Original code: A01
        auctionType=AuctionType('Explicit'), # Original code: A02
        auctionCategory=AuctionCategory('Hourly'), # Original code: A04
        classificationSequence_AttributeInstanceComponent_Position=1,
        periodStart=201601012300,
        periodEnd=201601022300)

>>> ParameterTypes.DocumentType['A25'] == ParameterTypes.DocumentType('Allocation result document')
True
>>> ec.ParameterTypes.DocumentType.help()
--- DocumentType ---
API_PARAMETER: DESCRIPTION
[...]
A25: Allocation result document
A71: Generation forecast
A72: Reservoir filling information
A73: Actual generation
A85: Imbalance prices
A86: Imbalance volume
[...]
API_PARAMETER: DESCRIPTION
--- DocumentType ---
>>> ec.ParameterTypes.BusinessType.help()
--- BusinessType ---
API_PARAMETER: DESCRIPTION
[...]
A25: General Capacity Information
A29: Already allocated capacity(AAC)
A97: Manual frequency restoration reserve
B08: Total nominated capacity
C22: Shared Balancing Reserve Capacity
C24: Actual reserve capacity
[...]
API_PARAMETER: DESCRIPTION
--- BusinessType ---

#shortened from sample_plot.py
import entsoe_client as ec
from settings import api_key

# Instantiate Client, Parser and Query.
client = ec.Client(api_key)
parser = ec.Parser()
query = ec.Queries.Generation.AggregatedGenerationPerType(
    in_Domain=ec.ParameterTypes.Area.DE_LU,
    periodStart=202109050200,
    periodEnd=202109070200)

# Extract data.
response = client(query)
df = parser(response)
[...]

# Transform data.
production = df[~consumption_mask][['quantity', 'TimeSeries.MktPSRType.psrType']]
## PsrType, e.g. `B01` := `Biomass`.
production['GenerationType'] = production['TimeSeries.MktPSRType.psrType']. \
    apply(lambda x: ParameterTypes.PsrType[x].value) # Map ENTSO-E PsrTypes into human-readable string.
production_by_type = pd.pivot_table(production,
                                    index=production.index,
                                    columns='GenerationType',
                                    values='quantity')
[...]
# Plot.
production_by_type.plot.bar(title="Production by Generation Type in DE-LU",
                            xlabel="UTC",
                            ylabel='MWh',
                            ax=ax,
                            **plot_params)
[...]