pattern-fill 0.1.4

Time series gap-filling using daily diurnal patterns

Pattern Fill

A metadata-aware implementation of pattern fill algorithms for gap-filling time series data using daily diurnal patterns.

🕹️ 📊 Try it Online

Launch Interactive Pattern Designer →

Design and test patterns directly in your browser using our WASM-powered marimo notebook. No installation required!

Features

• Dual pattern modes: Spline-based (traditional) and Sine wave-based (new)
• Metadata awareness: Track processing steps, preserve data provenance
• Weekday/weekend patterns: Different patterns for different day types
• FFT-based auto-fitting: Automatically extract sine components from data
• Flexible API: Easy-to-use factory methods and serialization

Installation

uv pip install pattern-fill

Or with pip:

pip install pattern-fill

Quick Start

Sine Wave Patterns

Define patterns using intuitive sine wave parameters:

from pattern_fill import DailyPattern, pattern_fill
import pandas as pd
import numpy as np

# Create a simple daily pattern peaking at 8 AM
pattern = DailyPattern.from_simple_sine(
    amplitude=0.4,    # Variation strength (0-1)
    frequency=1.0,    # Cycles per day
    phase=8.0,        # Peak time (hours)
    baseline=0.5      # Center value (0-1)
)

# Create time series with gaps
index = pd.date_range("2024-01-01", periods=96, freq="15min")
series = pd.Series(np.random.randn(96) + 10, index=index)
series.iloc[20:30] = np.nan

# Fill gaps using the pattern
results = pattern_fill([series], pattern=pattern)
filled_series, processing_steps = results[0]

Complex Multi-Component Patterns

For wastewater treatment or environmental monitoring:

# Complex pattern: daily + twice-daily variations
pattern = DailyPattern.from_sine_waves(
    components=[
        (0.35, 1.0, 8.0),   # Daily cycle, peak at 8 AM
        (0.15, 2.0, 13.0),  # Twice-daily, peaks at 1 PM and 1 AM
        (0.05, 1/7, 0.0),   # Weekly variation
    ],
    baseline=0.45,
    name="nh4_pattern"
)

Auto-Fit from Data

from pattern_fill import fit_sine_pattern

# Automatically extract sine components using FFT
pattern = fit_sine_pattern(
    clean_series,
    n_components=2,  # Number of components to fit
)

# Or specify frequencies explicitly
pattern = fit_sine_pattern(
    clean_series,
    frequencies=[1.0, 2.0],  # Daily + twice-daily
)

Spline-Based Patterns (Traditional)

The traditional approach using control points:

pattern = DailyPattern(
    hours=[0, 6, 12, 18],
    values=[0.2, 0.8, 0.9, 0.5],
    name="flow_pattern"
)

Sine Wave Parameters

• amplitude: Controls the strength of variation (0-1 range)
• frequency: Cycles per day
  • 1.0 = once per day (24-hour cycle)
  • 2.0 = twice per day (12-hour cycle)
  • 0.5 = once every 2 days (48-hour cycle)
• phase: Time of peak in hours (0-24)
  • 0.0 = peak at midnight
  • 6.0 = peak at 6 AM
  • 12.0 = peak at noon
• baseline: Center value around which the sine oscillates (0-1 range)

Weekday/Weekend Patterns

patterns = {
    "weekday": DailyPattern.from_sine_waves(
        [(0.35, 1.0, 8.0)],
        baseline=0.5,
        day_type="weekday"
    ),
    "weekend": DailyPattern.from_sine_waves(
        [(0.25, 1.0, 10.0)],
        baseline=0.45,
        day_type="weekend"
    ),
}

results = pattern_fill([series], pattern=patterns)

Serialization

Save and load patterns:

# To JSON
json_str = pattern.to_json()

# From JSON
pattern2 = DailyPattern.from_json(json_str)

# To dict
d = pattern.to_dict()

# From dict
pattern3 = DailyPattern.from_dict(d)

Benefits of Sine Patterns over Splines

• Intuitive: Parameters directly map to physical phenomena
• Readable: DailyPattern.from_sine_waves([(0.35, 1.0, 8.0)]) immediately communicates "daily cycle peaking at 8 AM"
• Composable: Easily combine multiple periodicities
• Natural: Perfect for wastewater treatment and environmental monitoring patterns

API Reference

Main Functions

• pattern_fill(input_series, pattern, ...) - Fill gaps in time series
• fit_pattern(series, n_control_points, ...) - Auto-fit spline pattern from data
• fit_sine_pattern(series, n_components, ...) - Auto-fit sine pattern using FFT

Classes

• DailyPattern - Main pattern class supporting both spline and sine modes
• SineComponent - Individual sine wave component

Factory Methods

• DailyPattern.from_sine_waves(components, baseline, ...) - Create from sine components
• DailyPattern.from_simple_sine(amplitude, frequency, phase, baseline) - Create simple sine pattern

Examples

See the notebooks/ directory for interactive examples using the pattern designer.

Development

# Install dependencies
uv sync

# Run tests
pytest tests/

# Run specific test file
pytest tests/test_sine_component.py -v

References

• Spline interpolation: Scipy
• FFT-based sine fitting: Scipy
• AR model fitting:L Statsmodel
• Stochastic gap filling: in Influent generator: Towards realistic modelling of wastewater flowrate and water quality using machine-learning methods. PhD Thesis. Département de génie civil et de génie des eaux, Université Laval, Québec, QC, Canada. 161pp.
• Smooth daily pattern insertion in wastewater time series: in Suivi, compréhension et modélisation d’une technologie à biofilm pour l’augmentation de la capacité des étangs aérés PhD Thesis. Département de génie civil et de génie des eaux, Université Laval, Québec, Canada. 214pp.

License

MIT License

Contributing

Contributions welcome! Please open an issue or pull request.