snecs_functions

Python package for working with the Sub-National Energy Consumption Statistics dataset

Overview

The Sub-National Energy Consumption Statistics (SNECS) dataset is published by the UK government here:

This Python package contains a series of Python functions which can be used to:

Download the CSV files which make up the SNECS dataset.
Import the data into a SQLite database.
Access the data in the database, for data analysis and visualisation.

A description of the CSV files in the SNECS dataset, along with instructions for downloading and importing in SQLite, has been created using the format of a CSVW metadata Table Group object (saved as a JSON file), which is available here: https://raw.githubusercontent.com/building-energy/snecs_functions/main/snecs_tables-metadata.json

Installation

pip install snecs_functions

Quick Start

import snecs_functions

# downloads all CSV files to folder '_data' and import all the data to SQLite database 'snecs_data.sqlite'
snecs_functions.download_and_import_all_data() 

# view gas consumption in the East Midlands in 2019 and 2020
result = snecs_functions.get_government_office_region_elec(region_code='E12000004' year=[2019,2020])
print(result)

[
 {'year': 2019, 'gor': 'E12000004', 'region': 'East Midlands', 'e7_meters': 598.736, 'standard_meters': 1494.085, 'domestic_meters': 2092.821, 'non_domestic_meters': 172.576, 'total_meters': 2265.397, 'e7_sales_gwh': 2607.405573, 'standard_sales_gwh': 4911.897206, 'domestic_sales_gwh': 7519.302779, 'non_domestic_sales_gwh': 12759.67612, 'total_gwh': 20278.9789, 'e7_mean_kwh': 4354.850173, 'e7_median_kwh': 3271.2, 'standard_mean_kwh': 3287.56209, 'standard_median_kwh': 2709.8, 'domestic_mean_kwh': 3592.902966, 'domestic_median_kwh': 2846.6, 'non_domestic_mean_kwh': 73936.56196, 'non_domestic_median_kwh': 8378.45, 'all_mean_kwh': 8951.622561, 'all_median_kwh': 2928.1, 'avg_per_hhld_kwh': 3719.672351}, 
 {'year': 2020, 'gor': 'E12000004', 'region': 'East Midlands', 'e7_meters': 579.852, 'standard_meters': 1532.956, 'domestic_meters': 2112.808, 'non_domestic_meters': 172.466, 'total_meters': 2285.274, 'e7_sales_gwh': 2661.295432, 'standard_sales_gwh': 5337.930295, 'domestic_sales_gwh': 7999.225726, 'non_domestic_sales_gwh': 11468.72202, 'total_gwh': 19467.94775, 'e7_mean_kwh': 4589.611542, 'e7_median_kwh': 3379.3, 'standard_mean_kwh': 3482.115791, 'standard_median_kwh': 2808.8, 'domestic_mean_kwh': 3786.063725, 'domestic_median_kwh': 2941.2, 'non_domestic_mean_kwh': 66498.45199, 'non_domestic_median_kwh': 6556.95, 'all_mean_kwh': 8518.868087, 'all_median_kwh': 3002.1, 'avg_per_hhld_kwh': 3923.50087}
]

API

get_available_csv_file_names

Description: Returns the CSV file names of all tables in the CSVW metadata file.

snecs_functions.get_available_csv_file_names()

Returns: A list of the https://purl.org/berg/csvw_functions_extra/vocab/csv_file_name value in each table.

download_and_import_data

Description: Downloads all the SNECS data and imports all data into a SQLite database.

snecs_functions.download_and_import_data(
        csv_file_names = None,
        data_folder='_data',
        database_name='snecs_data.sqlite',
        verbose=False,
        )

The data to be downloaded is described in the CSVW metadata file here: https://raw.githubusercontent.com/building-energy/snecs_functions/main/snecs_tables-metadata.json

Running this function will:

create the data_folder if it does not already exist.
download the CSV files to the data folder.
download the CSVW metadata file to the data folder. This is named 'snecs_tables-metadata.json'.
create a SQLite database named database_name in the data folder if it does not already exist.
import the CSV data into the SQLite database.

Arguments:

csv_file_names (str, list or None): The CSV file name(s) to download and import (see get_available_csv_file_names). None will download the entire dataset.
data_folder (str): The filepath of a local folder where the downloaded CSV data is saved to and the SQLite database is stored.
database_name (str): The name of the SQLite database, relative to the data_folder.
verbose (bool): If True, then this function prints intermediate variables and other useful information.

Returns: None

get_snecs_table_names_in_database

Description: Returns the table names of all SNECS tables in the SQLite database.

snecs_functions.get_snecs_table_names_in_database(
        data_folder = '_data',
        database_name = 'snecs_data.sqlite',
        )

Returns: A list of table names.

get_government_office_region_elec

Description: Returns the electricity statistics for government office regions.

snecs_functions.get_government_office_region_elec(
        year=None,
        region_code=None,
        fields = None,
        data_folder='_data',
        database_name='snecs_data.sqlite',
        verbose=False
        )

Arguments:

year (int or list): Year(s) to filter by (2015 - 2021)
region_code (str or list): ONS region code(s) to filter by (e.g. 'E12000004')