🚀 Advanced Neural Network CPD Learning for Bayesian Networks

🏠 Homepage: https://github.com/rudzanimulaudzi/DeepParameters

DeepParameters is a comprehensive Python package for learning Conditional Probability Distributions (CPDs) using state-of-the-art neural network architectures. It provides a unified interface for experimenting with various deep learning approaches to probabilistic modeling.

🚀 Key Features

• 9 Neural Network Architectures: Simple NN, Advanced NN, LSTM, Autoencoder, VAE, BNN, Normalizing Flow, Ultra, Mega
• 12 Sampling Methods: Gibbs, Metropolis-Hastings, Importance, BPE, Variational, HMC, SMC, Adaptive KDE, Weighted, Stratified, KDE, and Dirichlet Bayesian
• Configurable Parallel Learning: Choose between 'topological' and 'parent_child' parallel execution styles
• Parallel CPD Learning: Multi-threaded parameter learning with factor group decomposition
• Comprehensive Evaluation: 7 performance metrics including MAE, KL divergence, and probability consistency
• Simple Interface: Unified learn_cpd_for_node() function for all architectures
• Command-line Interface (CLI): Learn CPDs, compare results, and inspect architectures directly from the terminal via deepparameters / dp

🔧 Data Preprocessing Requirements

⚠️ IMPORTANT: DeepParameters requires discrete data for proper functioning. All variables in your dataset must be categorical/discrete rather than continuous.

Data Discretization Steps

Before using DeepParameters, ensure your data is properly discretized:

import pandas as pd
import numpy as np
from sklearn.preprocessing import KBinsDiscretizer, LabelEncoder

# Example: Converting continuous data to discrete
def preprocess_data_for_deepparameters(data, continuous_columns=None, n_bins=3):
    """
    Prepare data for DeepParameters by discretizing continuous variables.
    
    Args:
        data (pd.DataFrame): Input dataset
        continuous_columns (list): List of continuous columns to discretize
        n_bins (int): Number of bins for discretization
    
    Returns:
        pd.DataFrame: Discretized dataset ready for DeepParameters
    """
    processed_data = data.copy()
    
    # Auto-detect continuous columns if not specified
    if continuous_columns is None:
        continuous_columns = data.select_dtypes(include=[np.number]).columns.tolist()
    
    # Discretize continuous variables
    for col in continuous_columns:
        if col in processed_data.columns:
            discretizer = KBinsDiscretizer(n_bins=n_bins, encode='ordinal', strategy='uniform')
            processed_data[col] = discretizer.fit_transform(processed_data[[col]]).astype(int)
    
    # Ensure categorical variables are properly encoded
    categorical_columns = processed_data.select_dtypes(include=['object', 'category']).columns
    for col in categorical_columns:
        le = LabelEncoder()
        processed_data[col] = le.fit_transform(processed_data[col])
    
    # Convert all columns to integer type (required for DeepParameters)
    for col in processed_data.columns:
        processed_data[col] = processed_data[col].astype(int)
    
    return processed_data

# Example usage
# Load your raw data
raw_data = pd.read_csv('your_raw_data.csv')

# Preprocess data
discretized_data = preprocess_data_for_deepparameters(raw_data, n_bins=3)

# Verify discretization
print("Data ready for DeepParameters!")
for col in discretized_data.columns:
    print(f"{col}: {sorted(discretized_data[col].unique())}")

🆕 What's New

📦 Installation

pip install deepparameters
pip install --upgrade deepparameters

🎯 Quick Start

from deepparameters import learn_cpd_for_node
import pandas as pd
# Import BayesianNetwork (DiscreteBayesianNetwork for newer pgmpy versions)
try:
    from pgmpy.models import DiscreteBayesianNetwork as BayesianNetwork
except ImportError:
    from pgmpy.models import BayesianNetwork

# Load and preprocess your data (ensure it's discretized!)
data = pd.read_csv('your_discretized_data.csv')

# Define your Bayesian network structures
true_model = BayesianNetwork([('A', 'B'), ('C', 'B')])
learnt_model = BayesianNetwork([('A', 'B'), ('C', 'B')])

# Learn CPD with default settings
cpd = learn_cpd_for_node(
    node='B', 
    data=data, 
    true_model=true_model, 
    learnt_bn_structure=learnt_model,
    num_parameters=10
)

# Advanced configuration with tunable optimizers and early stopping
cpd = learn_cpd_for_node(
    node='B',
    data=data,
    true_model=true_model,
    learnt_bn_structure=learnt_model,
    num_parameters=20,
    network_type='lstm',           # Try: simple, advanced, lstm, autoencoder, vae, bnn
    sampling_method='4',           # Try: 1-12 for different sampling methods
    optimizer='adamw',             # adam, adamw, sgd, rmsprop, nadam
    early_stopping_patience=15,   # Configurable early stopping
    epochs=200,
    verbose=True
)

⚡ Parallel Learning

Learn CPDs for entire networks using configurable parallel execution with two distinct approaches:

Topological Parallel Learning

Groups nodes by dependency levels in the network. Nodes at the same topological level (same distance from root nodes) are learned in parallel. This approach:

• Works well for networks with clear hierarchical structure
• Suitable for data with strong dependency relationships
• Reliable for complex networks with multiple dependency paths

Parent-Child Factor Group Learning

Groups nodes based on shared parent relationships. Nodes with the same parents are learned together. This approach:

• Optimized for networks with many nodes sharing common parents
• Suitable for data with clustered family relationships
• More efficient for hierarchical data structures

📋 Basic Usage

from deepparameters.core import DeepParametersLearner

learner = DeepParametersLearner()

# For hierarchical networks (recommended)
cpds = learner.learn_network_parallel(
    data=your_data,
    network_structure=your_network,
    parallel_style='parent_child'
)

# For complex interconnected networks (recommended)  
cpds = learner.learn_network_parallel(
    data=your_data,
    network_structure=your_network,
    parallel_style='topological'
)

🎯 Style Selection Guide

⚙️ Advanced Configuration

# High-performance configuration
cpds = learner.learn_network_parallel(
    data=data,
    network_structure=network,
    parallel_style='parent_child',
    max_workers=4,           # Optimal for most systems
    epochs=30,               # Good balance of quality/speed
    network_type='advanced', # For complex learning
    max_time_per_group=60   # Prevent timeouts
)

# Performance benchmarking
results = learner.benchmark_parallel_performance(
    data=data,
    network_structure=network,
    parallel_style='parent_child',
    max_workers_list=[1, 2, 4],
    epochs=20
)

🎯 Best Practices

1. Start with parallel_style='topological' if unsure
2. Use 2-4 workers for optimal performance
3. Provide 500+ samples for reliable learning
4. Monitor memory usage for large networks
5. Implement error handling in production code

📊 Complete Examples

from deepparameters.core import DeepParametersLearner

# Initialize learner
learner = DeepParametersLearner()

# Option 1: Topological Level Groups (default)
# Groups nodes by dependency levels - reliable for complex networks
cpds = learner.learn_network_parallel(
    data=data,
    network_structure=bn,
    parallel_style='topological',  # Default
    max_workers=4,
    verbose=True
)

# Option 2: Parent-Child Factor Groups (optimized)
# Groups nodes by parent relationships - better for hierarchical structures
cpds = learner.learn_network_parallel(
    data=data,
    network_structure=bn,
    parallel_style='parent_child',  # Optimized for hierarchical networks
    max_workers=4,
    verbose=True
)

# Advanced parallel configuration
cpds = learner.learn_network_parallel(
    data=data,
    network_structure=bn,
    parallel_style='parent_child',   # Choose decomposition strategy
    network_type='advanced',         # Neural architecture
    sampling_method='3',             # Importance sampling
    epochs=100,
    max_workers=6,                   # Parallel workers
    max_time_per_group=60,           # Time limit per group
    verbose=True
)

🏗️ Architecture Overview

Neural Network Architectures

Sampling Methods

📊 Performance Evaluation

DeepParameters provides comprehensive evaluation metrics:

• Mean Absolute Error (MAE): Primary accuracy metric
• KL Divergence: Distribution similarity measure
• Root Mean Square Error (RMSE): Error magnitude
• Maximum Error: Worst-case performance
• JS Divergence: Symmetric distribution distance
• Cosine Similarity: Directional similarity
• Probability Consistency: Probabilistic validity

from deepparameters import evaluate_cpd_performance

# Evaluate learned CPD against ground truth
results = evaluate_cpd_performance(learned_cpd, true_cpd)
print(f"MAE: {results['mean_absolute_error']:.4f}")
print(f"KL Divergence: {results['kl_divergence']:.4f}")

💻 Command-Line Interface (CLI)

DeepParameters ships with a built-in terminal interface. Two commands are registered on install: deepparameters and the short alias dp.

Quick start

# Inspect available architectures and sampling methods
deepparameters info

# Learn a CPD for one node (outputs JSON)
deepparameters learn \
  --node heart_disease \
  --data medical_data.csv \
  --edges network_edges.csv \
  --num-parameters 20 \
  --network-type vae \
  --sampling-method 4 \
  --epochs 200 \
  --output cpd_heart.json

# Learn all CPDs in parallel
deepparameters learn-network \
  --data medical_data.csv \
  --edges network_edges.csv \
  --parallel-style topological \
  --max-workers 4 \
  --output-dir ./cpds/

# Compare a learned CPD to ground truth
deepparameters compare \
  --learned cpd_heart.json \
  --true    cpd_heart_true.json \
  --output  metrics.json

CLI commands at a glance

Input files: CSV for data, two-column CSV (parent,child) for edges.
Output: JSON files containing serialized TabularCPD objects. These can be reloaded directly into pgmpy.

Use deepparameters --help or deepparameters <command> --help for full flag documentation.

�🔧 Advanced Configuration

# Full parameter configuration with detailed comments
cpd = learn_cpd_for_node(
    node='B',                       # Target node to learn CPD for
    data=data,                      # Training dataset (must be discretized)
    true_model=true_model,          # True Bayesian network structure
    learnt_bn_structure=learnt_model, # Learned network structure
    num_parameters=50,              # Number of parameters for neural network
    network_type='vae',             # Neural architecture type
    sampling_method='8',            # Sampling method (1-12)
    optimizer='adamw',              # Optimizer: adam, adamw, sgd, rmsprop, nadam
    early_stopping_patience=20,    # Early stopping patience (epochs)
    epochs=500,                     # Maximum training epochs
    batch_size=64,                  # Training batch size
    learning_rate=0.001,            # Learning rate for optimizer
    validation_split=0.2,           # Validation data split ratio
    early_stopping=True,            # Enable early stopping
    verbose=True,                   # Enable verbose output
    random_state=42                 # Random seed for reproducibility
)

🔧 Neural Network Optimizer Options

📚 Documentation

• Complete Documentation: Full documentation and API reference
• Performance Analysis: Detailed benchmarks and comparisons
• API Reference: Complete function documentation

🧪 Example Workflows

Complete Workflow Guide: Step-by-step usage examples and comprehensive tutorials

🤝 Contributing

We welcome contributions! For now email rudzani.mulaudzi2@students.wits.ac.za

📄 License

This project is licensed under the MIT License.

🎓 Citation

If you use DeepParameters in your research, please cite:

@software{deepparameters2026,
  title={DeepParameters: Neural Network Bayesian Network CPD Learning},
  author={Rudzani Mulaudzi},
  year={2026},
  version={2.0.11},
  url={https://github.com/rudzanimulaudzi/DeepParameters}
}

🆘 Support

Coming Soon

DeepParameters - Making advanced CPD learning accessible to everyone.