Skip to main content

Information-theoretic outlier detection for spatial networks

Project description

elwood-spatial

Information-theoretic outlier detection for spatial networks. Includes functionality for instant outlier detection, as well as characterizing the agreement/volatility of antecedant measurements at a node or within the network.

Orignally developed to identify outliers in air-sensor networks during wildland fire smoke events, the functionality provided within this package can be broadly applied to outlier problems in general, although much of it is relevant to environmental monitoring networks where space/time are relevant concepts.

This code will be updated/adjusted throughout peer-review of a paper where these concepts were used. This package may be expaded to explore mutual/conditional information between nodes within a network, and quantify how their relationships change under various external forcings.

Installation

pip install elwood-spatial

Quick Start

import elwood_spatial as es

# 1. Define discrete bins for your measurements (domain specific)
bins = es.BinSpec.from_tuples([(0, 50), (51, 100), (101, 150), (151, 200)])

# 2. Sensor readings at a single timestep
values = {"sensor_1": 45, "sensor_2": 120, "sensor_3": 48, "sensor_4": 52}
bin_indices = {k: bins.bin_index(v) for k, v in values.items()}
# => {'sensor_1': 0, 'sensor_2': 2, 'sensor_3': 0, 'sensor_4': 1}

Inspecting Network Entropy

Inspect the entropy of the network to see how much agreement exists among measurements:

all_indices = list(bin_indices.values())

# Shannon entropy — low values mean most sensors agree
entropy = es.shannon_entropy(all_indices)
print(f"Network entropy: {entropy:.3f} bits")

# Probability distribution across bins
probs = es.bin_probabilities(all_indices)
print(f"Bin probabilities: {probs}")
# => {0: 0.5, 2: 0.25, 1: 0.25}

Viewing Information Content per Sensor

Information content tells you how "surprising" each measurements is relative to its neighbours. A rare bin assignment produces high information:

for sensor_id, bi in bin_indices.items():
    ic = es.information_content(bi, all_indices)
    bd = es.bin_deviation(bi, [v for k, v in bin_indices.items() if k != sensor_id])
    print(f"{sensor_id}: value={values[sensor_id]}, bin={bi}, "
          f"information={ic:.3f} bits, bin_deviation={bd:.3f}")

Detecting Outliers

The rule-based detector flags a measurement when all three conditions hold:

  1. Information content >= θ (default 1.75)
  2. Network entropy < S (default 1.75)
  3. Bin deviation >= n_bins / β (default 3.5)
from elwood_spatial.detect import detect_outliers, PARAMS_OPERATIONAL

network = {
    "sensor_1": {"neighbors": ["sensor_2", "sensor_3", "sensor_4"], "weights": [1.0, 0.9, 0.8]},
    "sensor_2": {"neighbors": ["sensor_1", "sensor_3", "sensor_4"], "weights": [1.0, 0.9, 0.7]},
    "sensor_3": {"neighbors": ["sensor_1", "sensor_2", "sensor_4"], "weights": [0.9, 0.9, 0.8]},
    "sensor_4": {"neighbors": ["sensor_1", "sensor_2", "sensor_3"], "weights": [0.8, 0.7, 0.8]},
}

results = detect_outliers(values, bins, network, PARAMS_OPERATIONAL)
for sid, flagged in results.items():
    print(f"{sid}: {'OUTLIER' if flagged else 'normal'}")

See the `examples/` notebooks for some use cases.

## Documentation

Docs, interactive testbed, and guides:
**https://elwood-spatial.com/**

## Development

```bash
pip install -e ".[dev]"
pytest tests/ -v
ruff check src/

License

BSD-3-Clause

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

elwood_spatial-0.1.1.tar.gz (13.6 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

elwood_spatial-0.1.1-py3-none-any.whl (14.9 kB view details)

Uploaded Python 3

File details

Details for the file elwood_spatial-0.1.1.tar.gz.

File metadata

  • Download URL: elwood_spatial-0.1.1.tar.gz
  • Upload date:
  • Size: 13.6 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.12

File hashes

Hashes for elwood_spatial-0.1.1.tar.gz
Algorithm Hash digest
SHA256 95f6586ca1996680cfb59fd472abad577f281087f8133612386070a81e4844f3
MD5 801d047febd13ffbddadee97e82f5541
BLAKE2b-256 f83c7351702eea4a6720a4a06afb009fae1f6c8c95cfec8b84e75b2588765241

See more details on using hashes here.

File details

Details for the file elwood_spatial-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: elwood_spatial-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 14.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.12

File hashes

Hashes for elwood_spatial-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d31e03985b3bf21d35ac1d16d068304ce13e0400b3d6d0539f887649bc19cb94
MD5 edf78535ec329efe509503cf8a6c4589
BLAKE2b-256 20c0cd359e955a214279de4e6af0fd5f5adc47c374f650d4843b0f4e3fc228f8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page