Example Usage

from pathlib import Path
from mango_autoencoder import AutoEncoder

# Load a trained model
model = AutoEncoder.load_from_pickle("path/to/model.pkl")

# Set up output directory
reconstruct_output_dir = Path("autoencoder_output/reconstruction")
reconstruct_output_dir.mkdir(parents=True, exist_ok=True)

# Perform reconstruction on new data
reconstructed_results = model.reconstruct_new_data(
    id_columns="source_file",
    data=data,
    iterations=3,
    save_path=str(reconstruct_output_dir),
    reconstruction_diagnostic=True
)

Parameters

• ``data``: Input data (numpy array, pandas DataFrame, or polars DataFrame)

• ``iterations``: Number of reconstruction iterations (default: 1) - Higher iterations can improve reconstruction quality for data with many missing values - Each iteration uses the previous reconstruction to improve the next one

• ``id_columns``: Column(s) that define IDs to process reconstruction separately - Useful when data contains multiple time series (e.g., different sensors, locations) - Can be a string, integer, or list of strings/integers

• ``save_path``: Path to save reconstruction results and diagnostics

• ``reconstruction_diagnostic``: If True, generates error analysis and visualization files

How It Works

1. Data Validation: Checks that the new data has the same features as the training data

2. ID Processing: Separates data by ID columns if specified

3. Iterative Reconstruction: - For each iteration, the model reconstructs the data - Missing values (NaN) are filled with reconstructed values - The process repeats to improve reconstruction quality

4. Result Generation: Returns reconstructed data and optionally saves diagnostic files

Training Phase

When you train a model with build_and_train(), the following files are created in the specified save_path:

Model Files

• ``models/model.pkl``: Main model file containing the trained Keras model and training parameters

• ``models/{epoch}.pkl``: Checkpoint files saved every checkpoint epochs (e.g., 10.pkl, 20.pkl)

Visualization Files

• ``loss_history.html``: Interactive plot showing training and validation loss over epochs

Reconstruction Files (if reconstruction_diagnostic=True)

• ``actual_vs_reconstructed.html``: Interactive plot comparing original vs reconstructed data

• ``reconstruction_error.csv``: Detailed reconstruction error data

• ``reconstruction_error_summary.csv``: Summary statistics of reconstruction errors

• ``reconstruction_error_boxplot.html``: Box plot visualization of reconstruction errors by feature and data split

Reconstruction Phase (reconstruct_new_data)

When using reconstruct_new_data(), the following files are created in the specified save_path:

Reconstruction Results

• ``reconstruct_new_data/{id}_reconstruction_results.csv``: Reconstructed data for each ID (or “global” if no IDs)

Diagnostic Files (if reconstruction_diagnostic=True)

• ``reconstruct_new_data/{id}_reconstruction_error.csv``: Reconstruction error data for each ID

• ``reconstruct_new_data/{id}_reconstruction_error_summary.csv``: Summary statistics for each ID

• ``reconstruct_new_data/{id}_reconstruction_error_boxplot.html``: Box plot of reconstruction errors for each ID

File Structure Example

autoencoder_output/
├── models/
│   ├── model.pkl
│   ├── 10.pkl
│   ├── 20.pkl
│   └── ...
├── loss_history.html
├── actual_vs_reconstructed.html
├── reconstruction_error.csv
├── reconstruction_error_summary.csv
├── reconstruction_error_boxplot.html
└── reconstruct_new_data/
    ├── global_reconstruction_results.csv
    ├── global_reconstruction_error.csv
    ├── global_reconstruction_error_summary.csv
    └── global_reconstruction_error_boxplot.html