disease-ews 1.0.0

Automated epidemiological surveillance system that processes WHO outbreak data (1996-2020) to generate risk scores, early warning flags, and analytical reports across 8 infectious diseases.

Disease Outbreak Early Warning System

A Python-based epidemiological surveillance platform that processes WHO outbreak data (1996-2020), computes multi-signal risk indicators, generates analytical visualizations, and produces automated early-warning reports. Includes a static web dashboard for interactive data exploration.

---

Table of Contents

• Project Overview
• Diseases Covered
• Architecture
• Technology Stack
• Folder Structure
• Installation
• Usage
• Pipeline Outputs
• Methodology
• Alert Level System
• Web Dashboard
• Future Development
• Data Sources
• License

---

Project Overview

The Disease Outbreak Early Warning System (Disease EWS) is designed to:

1. Ingest and clean real WHO outbreak records spanning 1996 to 2020
2. Enrich the data with epidemiological metrics (incidence rate, CFR, rolling averages, growth rates)
3. Compute epidemic thresholds using historical baselines (pre-2019)
4. Classify outbreak severity into a four-tier alert system (GREEN, YELLOW, ORANGE, RED)
5. Calculate composite risk scores and early warning flags for each region-disease pair
6. Generate publication-quality charts and a consolidated monitoring dashboard
7. Produce a multi-page PDF early-warning report with actionable recommendations

The system is fully automated -- a single command executes the entire pipeline from raw data to finished reports and visualizations.

---

Diseases Covered

The system currently monitors 8 infectious diseases present in the WHO outbreak dataset:

Disease	Pathogen Type	Lethality Classification
COVID-19	Coronavirus	Normal
SARS	Coronavirus	Brutal
Avian Influenza	Influenza A (H5N1)	Normal
Ebola	Filovirus	Brutal
Cholera	Vibrio cholerae	Normal
MERS-CoV	Coronavirus	Brutal
Dengue	Flavivirus	Normal
Influenza	Influenza virus	Normal

Disease selection is configured in src/data_loader.py via the SELECTED_DISEASES list. Additional diseases can be added by extending this list and providing matching entries in the country-to-region mapping.

---

Architecture

+-------------------+     +------------------+     +------------------+
|   Raw WHO Data    | --> |  Data Loader     | --> |  Preprocessing   |
|  (Outbreaks.csv)  |     |  (data_loader)   |     |  (preprocessing) |
+-------------------+     +------------------+     +------------------+
                                                           |
                                                           v
                          +------------------+     +------------------+
                          |   PDF Report     | <-- |  Risk Analyzer   |
                          | (report_gen)     |     |  (risk_analyzer) |
                          +------------------+     +------------------+
                                                           |
                                                           v
                          +------------------+     +------------------+
                          |  Web Dashboard   | <-- |   Visualizer     |
                          |  (frontend/)     |     |  (visualizer)    |
                          +------------------+     +------------------+

Data flows left-to-right through the pipeline. The visualizer and report generator consume processed data to produce the final outputs.

---

Technology Stack

Component	Technology
Language	Python 3.10+
Data Processing	pandas, NumPy
Statistical	SciPy (linear regression for trend analysis)
Visualization	Matplotlib, Seaborn
PDF Generation	ReportLab (Platypus high-level API)
Web Dashboard	Vanilla HTML5, CSS3, JavaScript
Typography	Inter (Google Fonts)

---

Folder Structure

disease_ews/
|-- data/
|   |-- raw/
|       |-- Outbreaks.csv          # WHO outbreak records (1996-2020)
|       |-- diseases.csv           # Disease metadata (pathogens, lethality, incubation)
|-- src/
|   |-- __init__.py
|   |-- data_loader.py            # CSV ingestion, country-to-region mapping, cleaning
|   |-- data_generator.py         # Synthetic data generator (for testing/demo)
|   |-- preprocessing.py          # Feature engineering, alert classification, aggregation
|   |-- risk_analyzer.py          # Rt estimation, doubling time, EW flags, risk scores
|   |-- visualizer.py             # Matplotlib chart generation (6 chart types)
|   |-- report_generator.py       # ReportLab PDF report builder
|-- frontend/
|   |-- index.html                # Dashboard entry point
|   |-- css/
|   |   |-- style.css             # Design system (dark theme, glassmorphism, responsive)
|   |-- js/
|       |-- app.js                # Interactivity (lightbox, scroll reveal, counters)
|-- outputs/
|   |-- charts/                   # Generated PNG charts (6 files)
|   |-- data/                     # Processed CSV exports
|   |-- reports/                  # PDF early-warning report
|-- main.py                       # Pipeline orchestrator (single entry point)
|-- requirements.txt              # Python dependencies
|-- README.md                     # This file
|-- .gitignore

---

Installation

Prerequisites

• Python 3.10 or higher
• pip or uv package manager

Steps

1. Clone the repository:

   git clone <repository-url>
   cd disease_ews
   

2. Create and activate a virtual environment (recommended):

   python -m venv .venv
   
   # Windows
   .venv\Scripts\activate
   
   # macOS / Linux
   source .venv/bin/activate
   

3. Install dependencies:

   pip install -r requirements.txt
   

   Or using uv:

   uv pip install -r requirements.txt
   

---

Usage

Run the Full Pipeline

Execute the entire analysis pipeline with a single command:

python main.py

This runs all 6 stages sequentially:

1. Load -- Reads and filters the WHO outbreak dataset
2. Preprocess -- Engineers features, computes thresholds, assigns alert levels
3. Risk Analysis -- Calculates Rt, doubling time, EW flags, composite risk scores
4. Charts -- Generates 5 individual visualization charts
5. Dashboard -- Renders a composite multi-panel monitoring dashboard
6. Report -- Builds a multi-page PDF early-warning report

View the Web Dashboard

After running the pipeline, open the dashboard in any browser:

# Direct file open
start frontend/index.html        # Windows
open frontend/index.html          # macOS
xdg-open frontend/index.html     # Linux

Or use a local server for best results:

python -m http.server 8000 --directory .
# Then navigate to http://localhost:8000/frontend/index.html

---

Pipeline Outputs

Charts (outputs/charts/)

File	Description
01_disease_trends.png	Global monthly case trends per disease with rolling avg
02_risk_heatmap.png	Region x Disease alert level heatmap
03_regional_comparison.png	Stacked bar + horizontal bar of regional disease burden
04_growth_rates.png	Rolling 3-month growth rate with alert thresholds
05_alert_timeline.png	Stacked area chart of alert level distribution
06_dashboard.png	Composite 4-row multi-panel monitoring dashboard

Processed Data (outputs/data/)

File	Description
raw_who_outbreaks.csv	Filtered and cleaned WHO records
enriched_who_data.csv	Full enriched dataset with all computed features
disease_metadata.csv	Disease reference data (pathogens, incubation, lethality)

Report (outputs/reports/)

File	Description
early_warning_report.pdf	Multi-page PDF with executive summary, charts, tables

---

Methodology

Epidemic Threshold

Calculated from pre-2019 historical baseline for each country-disease pair:

threshold = historical_mean + 1.96 * historical_std

Cases are compared against this threshold to compute exceedance ratios.

Alert Classification

Level	Condition
GREEN	Exceedance < 1.0x AND growth rate < 10%
YELLOW	Exceedance 1.0-2.0x OR growth rate 10-25%
ORANGE	Exceedance 2.0-3.0x OR growth rate 25-50%
RED	Exceedance >= 3.0x OR growth rate >= 50%

Composite Risk Score (0-100)

Weighted combination of four normalized indicators:

risk_score = 0.30 * growth_rate
           + 0.25 * incidence_rate
           + 0.25 * threshold_exceedance
           + 0.20 * case_fatality_rate

Early Warning Flags

Multi-signal detection triggered when at least 2 of 4 conditions are met:

1. Effective reproduction number Rt >= 1.5
2. Month-over-month growth rate >= 20%
3. Threshold exceedance >= 1.5x baseline
4. Growth rate velocity (acceleration) >= 10 percentage points per month

Rt Estimation

Simple proxy using the ratio of rolling average cases in the current window versus the previous window (default: 3-month windows).

Doubling Time

Estimated from monthly growth rate:

doubling_time = ln(2) / ln(1 + growth_rate)

---

Alert Level System

The four-tier system provides graduated response guidance:

• GREEN (Normal): Routine monitoring. Cases within expected seasonal range.
• YELLOW (Watch): Heightened surveillance. Investigate potential clusters.
• ORANGE (Warning): Activate preparedness protocols. Increase reporting frequency.
• RED (Critical): Immediate response. Deploy rapid response teams and emergency resources.

---

Web Dashboard

The frontend provides a professional dark-themed monitoring interface built with vanilla HTML, CSS, and JavaScript. Features include:

• Animated KPI summary cards with key system metrics
• Interactive chart panels with click-to-expand lightbox
• Disease coverage grid showing all 8 monitored diseases
• Alert level classification reference
• Methodology breakdown with formulas
• Fully responsive layout for desktop, tablet, and mobile
• Scroll-reveal animations and smooth navigation
• No external framework dependencies -- opens directly in any browser

---

Future Development

The following enhancements are planned for future iterations:

Data and Coverage Expansion

• Expand disease coverage beyond the current 8 diseases to include Measles, Malaria, Tuberculosis, Zika, Yellow Fever, and other WHO-tracked diseases
• Integrate additional data sources (ECDC, CDC, ProMED) for cross-validation
• Add support for sub-national (state/province) level outbreak data
• Incorporate real-time data feeds via WHO API integration

Analytical Enhancements

• Implement machine learning models (LSTM, Prophet) for case forecasting
• Add spatial clustering analysis to detect cross-border outbreak corridors
• Integrate climate and environmental data (temperature, rainfall) as predictive features
• Develop a contact network-based Rt estimation replacing the current proxy method
• Add seasonal decomposition (STL) for better baseline modeling

Platform and Interface

• Migrate the static dashboard to a live server-based application (Flask or FastAPI)
• Add real-time data refresh and automated scheduled pipeline execution
• Implement user authentication and role-based access control
• Build an interactive map visualization with drill-down by region and country
• Add email/SMS alerting when early warning flags are triggered
• Develop a REST API for programmatic access to risk scores and alert data

Reporting

• Add automated report scheduling (daily, weekly, monthly)
• Generate region-specific and disease-specific sub-reports
• Include confidence intervals and uncertainty quantification in all metrics
• Export dashboard panels as individual high-resolution figures for publications

---

Data Sources

• WHO Disease Outbreak News (DON): Historical outbreak records from 1996 to 2020, sourced from the World Health Organization public reporting system.
• Disease Metadata: Pathogen classification, lethality ratings, transmission medium, and incubation period data compiled from WHO and CDC reference materials.

All data used in this project is from publicly available WHO records. This system is intended for analytical and research purposes only. It does not constitute official public health guidance.

---

License

This project is provided for educational and research purposes. Please refer to the LICENSE file for terms of use.