gla-data

Shared tools for loading, transforming, and visualising the GLA datasets used in City Data analysis.

Installation

CLI

To install gla-data globally so the gla-data CLI is available anywhere in your terminal:

uv tool install gla-data

This requires uv to be installed.

pip

pip install gla-data

pixi

pixi add gla-data

Pixi pulls geospatial dependencies (geopandas, pyproj, etc.) from conda-forge, which provides reliable pre-built binaries across all platforms and Python versions.

Usage

Python API

import gla_data

# Load 2021 LSOA boundaries for London
lsoa_gdf = gla_data.load_boundaries(geography='lsoa', year=2021)

# Load census dwelling counts
dwellings_2021 = gla_data.load_census_dwellings(year=2021)
dwellings_2011 = gla_data.load_census_dwellings(year=2011)

# Crosswalk 2011 dwelling counts onto 2021 LSOA boundaries
dwellings_2011_xw = gla_data.crosswalk(
    dwellings_2011,
    from_year=2011,
    to_year=2021,
    value_cols=['dwellings_total'],
)

# Load an OA → LSOA geography lookup
lookup = gla_data.load_geography_lookup(year=2021, smallest_geography='oa')

# Aggregate OA-level data up to LSOA level
lsoa_totals = gla_data.aggregate(
    df,
    from_geography='oa',
    to_geography='lsoa',
    value_cols=['count'],
    year=2021,
)

# OS MasterMap buildings, with heights, in one 1km grid square plus a 100m
# margin (GLA only; needs the database settings below)
buildings = gla_data.load_mastermap_topography(
    bbox=(530000, 180000, 531000, 181000),
    buffer_m=100,
    descriptive_group='Building',
)

# GLA tree canopy in the same square plus a 100m margin. The first call
# downloads the data (about 270MB) and builds a local file from it, which
# takes several minutes; later calls read that file.
canopy = gla_data.load_tree_canopy(
    bbox=(530000, 180000, 531000, 181000),
    buffer_m=100,
)

CLI

The package installs a gla-data command. Run catalogue to list all available datasets:

gla-data catalogue

This prints a table of every dataset, the function used to load it, a description, and its access tier (public, registered, licensed or internal; see Configuration for the last three).

Configuration

Public datasets need no setup. Datasets held inside the GLA (the GIS team's Postgres database and S3 buckets) need settings that are not included in this package: connection details, table names and bucket names. GLA staff can get the values from the City Intelligence Data Science team.

Where to put settings

Every setting is named GLA_DATA_<NAME> and can be set in any of these places. Higher in the list wins:

1. An environment variable, e.g. GLA_DATA_BLUESKY_IMAGERY_BUCKET.
2. A .env file in the folder you run your code from (a project's own settings).
3. Your user-level settings file, ~/.config/gla-data/.env (on Windows, C:\Users\<you>\.config\gla-data\.env). Use this for settings shared by all your projects, such as database connections and bucket names. Set GLA_DATA_USER_ENV_FILE to use a different path.

A settings file holds one setting per line:

GLA_DATA_BLUESKY_IMAGERY_BUCKET=<bucket name>

A misspelt GLA_DATA_ setting in a settings file raises an error rather than being ignored. Other keys in a project's .env are left alone.

Settings for GLA-internal datasets

AWS credentials are never set here. gla-data uses the standard AWS sources: an AWS profile (~/.aws), AWS_* environment variables, or the machine's own role on GLA cloud machines. Keep database passwords in your user-level file, never in a project's repository.

Development

git clone <repo-url>
cd gla-data
pixi install

pixi run test       # run tests
pixi run lint       # check code style
pixi run format     # auto-format
pixi run typecheck  # type checking