HandoverLab

A system-level simulator for studying handover in dense 5G and beyond-5G networks, written in Python.

HandoverLab simulates 3GPP mobility procedures (baseline handover with Event A3, Release-16 conditional handover and DAPS handover, Release-18 L1/L2-triggered mobility) for many users moving through a street grid, an open area, along a highway, in an indoor office or across rural cells, with a received-power-level radio model from TR 38.901 (optionally with SSB beams, beam failure recovery and several carriers), an L3-filtered measurement chain, a radio-link failure model, and an outcome label for every handover attempt.

Status: 1.12.0 (python -m pip install handoverlab). The public surface (configuration and result formats with a schema version, the rule interface, metric names, commands and the documented Python names) follows the stability policy: within 1.x it only grows, and anything removed is deprecated first. New models are options, so the results of earlier configurations do not change.

See it

Run and watch. handoverlab gui opens a dashboard in the browser, on your computer only. Choose a layout, users and rules, press Run, and follow the users. Each user is coloured by its serving cell:

See where things happen. The tab Where marks every handover, unnecessary handover and failure at the place where it happened:

Compare rules fairly. All rules run on the same worlds, one world per seed. The tab Compare shows each mean with its 95 % confidence interval over the seeds, and the paired difference to the first rule:

Run whole studies. handoverlab study tunes every rule, checks gates, runs paired tests with multiple-comparison correction and writes an HTML report:

Indoors and in rural cells. Since 1.7.0: the TR 38.901 indoor office (12 ceiling sites, 20 m apart) and rural macro cells (sites 1732 m apart, users in cars at 120 km/h), both with the IMT-2020 link budgets and the soft LOS state of TR 38.901:

Beams and several carriers. Since 1.10.0: sectors with SSB beams (the antenna of TR 38.901), beam management and beam failure recovery as in TS 38.321 and TS 38.133, and further carriers with inter-frequency measurements in gaps, for example the two-layer dense urban deployment of ITU-R M.2412 (4 GHz macro cells, 30 GHz micro cells):

What it models

• Scenarios: the TR 37.885 urban street grid (configurable number of blocks; 433 m x 250 m pitch, 20 m streets) with sites on the pavement, an open area with a hexagonal site grid, the TR 37.885 highway (since 1.11.0 with wrap-around, LOS-only links and both site layouts of TR 36.885, below and above 6 GHz), the TR 38.901 indoor office, rural macro cells, explicit site lists, and built-in scenario cards (handoverlab scenarios).
• Channel: TR 38.901 UMi street canyon, UMa, RMa and InH-Office path loss (LOS/NLOS); line of sight from the building geometry plus the spatially consistent LOS state, hard or soft; spatially correlated shadowing; a time-varying component; L1 measurement error; beam gain and load-coupled interference; optionally a penetration loss per user (in-car loss of TR 38.901); or, since 1.12.0, your own radio map from a ray tracer or measurements (radio.map_file, the map-based model of TR 38.901 clause 8).
• Beams and carriers (options, 1.10.0): sectors with SSB beams from the TR 38.901 antenna model, cell quality from the beams (TS 38.331 5.5.3.3), the serving beam from beam reports, beam failure detection and recovery (TS 38.321 5.17, TS 38.133 8.5), several carriers with the measurement periods and gaps of TS 38.133, Event A5 between carriers.
• Mobility: street-following users with turn probabilities, random waypoints in the open area, pedestrian and vehicular speed profiles or your own speed groups.
• Traffic: constant service rates per 5QI class, or a traffic model per user: full buffer, periodic packets (as VoIP) or FTP model 3 of TR 36.814. The load of every cell, and so the interference, follows the traffic of its users.
• Measurements: L1 samples, the TS 38.331 L3 filter, Event A3 with time-to-trigger, periodic reporting for conditional handover.
• Handover procedures: baseline handover (report, preparation, command, execution), Rel-16 conditional handover (up to 8 prepared candidates) and DAPS handover, Rel-18 L1/L2-triggered mobility and conditional LTM from the Rel-19 work, behind a small rule interface. Rules that learn from outcomes are supported, among them mobility robustness optimisation of a cell individual offset per cell pair (a3_mro), an epsilon-greedy bandit (a3_bandit, checked against an exhaustive search) and a deep Q-network (dqn, optional PyTorch).
• Failure model: Qout/Qin with windows, N310/N311/T310, lost commands and configurations, execution failures, re-establishment (TR 36.839 / TR 36.842 values by default), with options for the link monitoring (own interferer load, restart on a new cell, ideal random access).
• Metrics: outcome of every attempt (useful, unnecessary, too early, wrong cell, too late, censored), failures by cause, ping-pong, preparation overhead, interruption, delivered rate, signalling (RRC messages, commands), L1 measurements of the device and network energy (EARTH model).
• Reproducibility: one world per seed shared by all compared rules (paired comparison); the engine draws no random numbers; golden event-log fingerprints guard every change. An optional compiled kernel (run.backend: numba) gives the same event logs bit for bit.

Built for beyond-5G

HandoverLab targets the questions of dense beyond-5G networks: small cells 50-300 m apart, the upper mid-band (FR3, 7-24 GHz; the scenario fr3-urban-15ghz, 10 GHz by default), and the mobility procedures beyond baseline handover: conditional handover, DAPS handover, L1/L2-triggered mobility of Release 18 and conditional LTM from the Release 19 work, next to rules that learn from handover outcomes and A3 tuned by mobility robustness optimisation, the fair baseline for a new rule. The cost of each procedure (signalling, prepared cells, interruption, energy) is measured next to its benefit.

Calibrated against 3GPP

The radio model passes the large-scale calibration of TR 38.901 (clause 7.8.1): in the calibration drop of 3GPP, the CDFs of the coupling loss and of the geometry (SINR) lie inside the range of the 21 companies that reported their results in the 3GPP contribution R1-165974 (Huawei, Ericsson, Nokia, Samsung, Qualcomm and others), for UMa, UMi and the indoor office at 6 and 30 GHz. 325 of 342 compared percentiles are inside the range, none is more than about 1 dB from the industry mean, and handoverlab verify repeats the check (details).

Follows the 3GPP texts

Every rule of the handover procedure is listed clause by clause against the official 3GPP texts (TS 38.331, 38.133, 38.300, 38.321 of Releases 17 to 19, TR 36.839, TR 38.901) in standard compliance, with what is followed, what is abstracted and what is not modelled yet. Since 1.9.0 the 3GPP protocol profile (protocol: {cmd_during_t310: receive, t304_s: 1.0, rlm_holdoff_s: 0.2}) makes the handover command and its execution behave as TS 38.331 specifies. Against handover statistics measured in commercial 5G networks (SIGCOMM, IMC and MobiCom measurement studies) the profile brings the handover failure rate from about six times above the measured range to just above it; ping-pong with the Verizon A3 setting and the handover distance in low-band rural cells match. Since 1.10.0 the page also covers beams, beam failure recovery, measurement gaps and Event A5, and the DAPS and LTM interruptions are checked against TS 38.133 and published 3GPP evaluations.

Checked against ns-3

The same 24 Event-A3 settings were simulated in HandoverLab and in the packet-level simulator ns-3 with 5G-LENA. Handover and unnecessary-handover rates agree (median ratio 0.92, rank correlation 0.99). For failure rates, with the in-car loss of ns-3 and the restart of the link monitoring switched on, the order of settings and conditions agrees too, also on ns-3 results that were held back from any tuning (rank correlation 0.82-0.91). The absolute values differ, so compare failure rates between rules within HandoverLab. The causes are traced in docs/calibration.md. On two of them HandoverLab follows the 3GPP texts more closely than ns-3:

• In-car penetration loss. TR 38.901 clause 7.4.3.2 gives it to a user inside a car. ns-3 adds it only while a user moves at 30-120 km/h and drops it above 120 km/h. HandoverLab applies it per user when configured (radio.penetration_db), at any speed.
• Time to detect a radio problem. TS 38.133 clause 8.1 evaluates out-of-sync over about 200 ms. ns-3 counts control receptions instead of time, and at light traffic this took 0.65 to 0.83 s in the reference runs. HandoverLab evaluates over time (protocol.qout_window_s, 0.2 s).

On a third point ns-3 was closer to the 3GPP texts: link monitoring restarts on a newly joined cell. HandoverLab has had this as an option since 1.3.0 (protocol.rlm_holdoff_s). The comparison covers baseline handover at 3.5 GHz only: 5G-LENA has no conditional handover, DAPS or L1/L2-triggered mobility.

What's new

• 1.12.0: radio maps from a file (radio.map_file): the path gain of every site and pixel from a ray tracer (TR 38.901 clause 8, map-based hybrid model) or interpolated from measurements, with the error on held-out points; an exported street-grid map gives the same event log bit for bit.
• 1.11.0: the TR 37.885 highway as specified (wrap-around, LOS-only base-station-to-vehicle links, sites on one side 1732 m apart below 6 GHz, 19 hexagonal sites with 3 sectors above), checked against an independent computation; every scenario card audited against TR 38.901 (also against V19.5.0), TR 37.885, TR 36.885 and ITU-R M.2412, with two cards corrected and the old highway card deprecated; a3_bandit against an exhaustive search (two of three criteria met, reported as is); the rule dqn with optional PyTorch; an optional Numba kernel with identical results.
• 1.10.0: SSB beams per sector with beam management and beam failure recovery (TS 38.321 5.17, TS 38.133 8.5), several carriers with inter-frequency measurements in gaps and Event A5, the measurement and RLM evaluation periods of TS 38.133, and DAPS and LTM checked against TS 38.133 and published 3GPP evaluations. Three sectors per site bring the mid-band handover counts into the range measured in commercial networks; a two-layer deployment alone does not.
• 1.9.0: the 3GPP protocol profile (handover command received during T310 and T304 as in TS 38.331), a clause-by-clause standard-compliance page, a fresh ns-3 hold-out and a comparison with commercial 5G measurements.
• 1.8.0: the radio model calibrated against 3GPP: the large-scale calibration of TR 38.901 clause 7.8.1 against the 21 companies of R1-165974, as the verify group calibration.
• 1.7.0: indoor office (TR 38.901 InH) and rural (RMa) scenarios with the IMT-2020 link budgets, the optional soft LOS state of TR 38.901, and the rule a3_mro (mobility robustness optimisation of a cell individual offset per cell pair).
• 1.6.0: conditional LTM (rule cltm), from the Release 19 work on L1/L2-triggered mobility.
• 1.5.0: signalling and energy metrics (RRC messages, handover commands, L1 measurements, site power and energy per bit); why failure rates differ from ns-3, stated in the README.
• 1.4.0: traffic models per user (traffic.model: full buffer, periodic, FTP model 3); a dashboard with confidence intervals, a map of where handovers and failures happen, and the status of a running study; the ns-3 link-monitoring finding in the calibration report.
• 1.3.0: optional restart of the link monitoring on a newly joined cell (TS 38.133).
• 1.2.0: optional penetration loss per user; the ns-3 comparison with the in-car loss of ns-3.
• 1.1.0 / 1.1.1: optional link-monitoring model; held-back ns-3 results; issue forms, CI on Windows for every test.

Earlier versions and every detail: the changelog.

Install

HandoverLab needs Python 3.9 or later. NumPy, SciPy and PyYAML are installed with it; matplotlib (reports, extra report), pyarrow (Parquet tables, extra parquet), Streamlit (the dashboard, extra gui) and MkDocs (the documentation site, extra docs) are optional.

From PyPI:

python -m pip install handoverlab                  # the core package
python -m pip install "handoverlab[report,gui]"    # with reports and the dashboard
handoverlab verify --only pathloss fingerprints    # optional: a quick check of the installation

From source (for development, the tests and the examples):

git clone https://github.com/mjs19971997/handoverlab.git
cd handoverlab
python -m pip install -e ".[test,report]"    # add gui for the dashboard: ".[test,report,gui]"
python -m pytest            # optional: run the tests (a few minutes)
handoverlab verify          # optional: verification report (about two minutes)

Quick start

Write an example configuration and run it:

handoverlab init config.yaml
handoverlab run config.yaml -o results/metrics.csv

run writes one row per seed and rule with all metrics and the fingerprint of the event log, to CSV or (with a .json output name) to JSON together with the resolved scenario and the package versions. handoverlab rules lists the available rules and handoverlab scenarios the built-in scenarios (scenario: smoke in a configuration uses one). Configurations can be YAML, TOML or JSON; see examples/.

A whole study, with tuning, gates, paired tests and a report (a few minutes):

handoverlab study examples/study_smoke.yaml --workers 2 --report

The report lands in results/study-smoke/report/ (report.html, report.md, figures); re-running the command skips everything that is already done, and handoverlab status results/study-smoke shows what is left and how long it will take.

Or explore in the browser:

python -m pip install "handoverlab[gui]"
handoverlab gui                      # http://localhost:8501, on your computer only

Example

import handoverlab as hl

scenario = hl.Scenario.from_dict({
    "layout": {"kind": "street_grid", "blocks": [3, 3], "site_spacing_m": 150.0},
    "mobility": {"users": 20, "profile": "veh16"},
    "run": {"duration_s": 60.0, "warmup_s": 10.0},
})
rules = [
    {"rule": "a3", "name": "A3", "off_db": 3.0, "ttt_s": 0.16},
    {"rule": "cho", "name": "CHO", "o_prep_db": -2.0, "o_exec_db": 3.0, "ttt_exec_s": 0.08},
]
for r in hl.simulate(scenario, rules, seeds=[1]):      # both rules run on the same world
    print(r.rule, r.metrics["handovers_pm"], r.metrics["unnecessary_pm"], r.metrics["failures_pm"])

Sample output of examples/traffic_models.py, the same rule under the four traffic models (one seed, 30 users, 60 s on the urban grid at 10 GHz):

model        cell load  rate [Mbit/s]  delivered  SINR [dB]  failures/UE-min
fixed             0.28          14.25       0.98       24.8             0.00
full_buffer       0.51         707.08       0.48       22.4             0.04
periodic          0.28           0.04       0.96       24.9             0.00
ftp3              0.28           3.93       0.93       24.9             0.00

Busier cells interfere more: under full buffer the SINR drops by more than 2 dB and failures appear.

The full script is examples/compare_a3_cho.py. Writing your own handover rule takes a few lines; see the user guide.

Documentation

The documentation site is at https://mjs19971997.github.io/handoverlab/. It is a MkDocs site (python -m pip install -e ".[docs]", then mkdocs serve); the pages are plain Markdown in docs/ and readable on GitHub as well.

• Tutorials: your first run, writing your own rule, a fair tuning study, reading the report.
• User guide: concepts, scenarios, configuration keys, command line, Python API, rule plug-ins and their contract checks, event logs and their levels, verification, metrics, studies (tuning, gates, storage, workers, status, reports), paired statistics, limitations.
• How-to guides: dashboard, add a scenario, run on many cores, export event logs, reproduce and check results, DAPS and LTM, a learning-based rule, the upper mid-band, a highway.
• Reference: command line, metric names and the API (generated from the docstrings, in the site); stability policy.
• Scenario cards: the built-in scenarios with the source of every parameter.
• Verification, failure-model calibration and performance.
• Changelog, contribution guide and releasing.

Roadmap

1. Stage 1 (0.1.0a1): core simulator, configuration, rule interface, command line, tests.
2. Stage 2 (0.2.0a1): verification in depth (invariant checks, golden event-log fingerprints, radio statistics, rule contract test kit, wider bh6 regression, ns-3 reference case), event logs as tables, the verify command, scenario cards, a faster world build.
3. Stage 3 (0.3.0a1): studies: equal-budget tuning with automatic gates, the selection rule, paired statistics, job records and Parquet tables, resumable studies with reused runs, worker processes, study and report commands, batched engines.
4. Stage 4 (0.4.0a1): browser dashboard (extra gui), tutorials, API and command-line reference with a documentation site, event-log levels, status with the time left, the grid-edge policy extend, failure-model calibration against ns-3 / 5G-LENA.
5. Stage 5 (1.0.0rc1): the public interfaces frozen and documented with a schema version (configuration, results, rule interface, metric names) and a stability policy; how-to guides; a lint job and a coverage threshold in CI; a nightly comparison of fingerprints between Linux and Windows; release plumbing for PyPI and Zenodo.
6. Stage 6 (1.0.0rc2): DAPS handover, L1/L2-triggered mobility, a learning-based example rule and an upper mid-band scenario, all additive.
7. Stage 7 (1.0.0rc3): a highway layout with lane-keeping vehicular mobility and its scenario card, site lists from CSV files, all additive.
8. Stage 8 (1.0.0rc4): the 3GPP citations and the documentation checked and corrected; no change of results.
9. 1.0.0: the first stable release, published on PyPI.
10. 1.0.1: the dashboard image shows on the PyPI project page; no change of results.
11. 1.0.2: README links work on PyPI, the documentation site is linked, text corrections; no change of results.
12. 1.1.0-1.3.0: optional link-monitoring and penetration-loss models traced against ns-3; issue forms and CI for every test on Windows.
13. 1.4.0: traffic models per user, a richer dashboard, pictures in the README.
14. 1.5.0: signalling and energy metrics.
15. 1.6.0: conditional LTM.
16. 1.7.0: indoor and rural scenarios; mobility robustness optimisation.
17. 1.8.0: the radio model calibrated against 3GPP (TR 38.901 clause 7.8.1).
18. 1.9.0: the 3GPP protocol profile and the standard-compliance page.
19. 1.10.0: SSB beams and beam failure recovery, several carriers with measurement gaps, DAPS and LTM against TS 38.133 and published 3GPP evaluations.
20. 1.11.0: the TR 37.885 highway, the card audit, the bandit against an exhaustive search, the dqn rule and the optional Numba kernel.
21. 1.12.0 (this release): radio maps from ray tracing or measurements.
22. Later (planned, not done): NTN mobility (CHO events D1/T1), 2.0.0.

Citation

If you use HandoverLab, please cite it as described in CITATION.cff (GitHub shows it under Cite this repository). The releases are archived on Zenodo: doi:10.5281/zenodo.23409131 cites all versions (it resolves to the latest); version 1.0.0 is doi:10.5281/zenodo.23409132 and version 1.0.1 doi:10.5281/zenodo.23409402.

Licence

MIT. See LICENSE.

Authors

Mostafa Jalili Sanjarani and Yasser Elmi Sola