beyondaccuracy-ml 0.2.0

Ethical risk evaluation for machine learning models.

beyond_accuracy

beyond_accuracy is a production-ready Python package for ethical risk evaluation of machine learning models. It accepts a trained model and dataset, computes ethical risk metrics, generates an aggregate ethical risk score, exports JSON and PDF reports, and provides a Streamlit dashboard plus CLI workflow.

Installation

pip install beyond_accuracy

For local development:

pip install -r requirements.txt
pip install -e .

Python Usage

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

from beyond_accuracy import evaluate

df = pd.read_csv("loan_data.csv")

X = df.drop(columns=["loan_status"])
y = df["loan_status"]

numeric_columns = X.select_dtypes(include=["number"]).columns.tolist()
categorical_columns = [col for col in X.columns if col not in numeric_columns]

model = Pipeline(
    steps=[
        (
            "preprocessor",
            ColumnTransformer(
                transformers=[
                    (
                        "num",
                        Pipeline(
                            steps=[
                                ("imputer", SimpleImputer(strategy="median")),
                                ("scaler", StandardScaler()),
                            ]
                        ),
                        numeric_columns,
                    ),
                    (
                        "cat",
                        Pipeline(
                            steps=[
                                ("imputer", SimpleImputer(strategy="most_frequent")),
                                ("encoder", OneHotEncoder(handle_unknown="ignore")),
                            ]
                        ),
                        categorical_columns,
                    ),
                ]
            ),
        ),
        ("classifier", LogisticRegression(max_iter=1000)),
    ]
)

model.fit(X, y)

report = evaluate(
    model=model,
    data=df,
    target="loan_status",
    sensitive=["gender", "age"],
    launch_ui=False,
)

print(report.risk_score)
print(report.decision)
print(report.module_scores)
print(report.to_json())
report.save_json("ethical_report.json")
report.save_pdf("ethical_report.pdf")

To run evaluation and launch the Streamlit dashboard directly from Python:

report = evaluate(
    model=model,
    data=df,
    target="loan_status",
    sensitive=["gender", "age"],
    launch_ui=True,
    json_path="ethical_report.json",
    pdf_path="ethical_report.pdf",
)

CLI Usage

Run evaluation and open the dashboard:

beyond-audit --data data.csv --target loan_status --sensitive gender age --ui

Run evaluation without opening the dashboard:

beyond-audit --data data.csv --target loan_status --sensitive gender age --no-ui

Behavior:

• Loads the CSV dataset
• Uses a supplied pickled model when --model model.pkl is provided
• Otherwise trains a baseline logistic regression pipeline
• Runs the ethical evaluation
• Saves ethical_report.json and ethical_report.pdf
• Launches the Streamlit dashboard with the saved JSON report when UI is enabled

Streamlit Dashboard

The dashboard supports:

• CSV upload
• Optional report JSON upload
• Optional pickled model upload
• Target and sensitive attribute selection
• Ethical Risk Score and decision display
• Gauge, bar, pie, and bias comparison charts
• JSON and PDF report downloads

Run directly:

streamlit run beyond_accuracy/ui/dashboard.py

Load a saved report directly into the dashboard:

streamlit run beyond_accuracy/ui/dashboard.py -- --report-json ethical_report.json

Project Structure

beyond_accuracy/
??? beyond_accuracy/
?   ??? __init__.py
?   ??? cli.py
?   ??? evaluator.py
?   ??? metrics/
?   ?   ??? bias.py
?   ?   ??? dataset.py
?   ?   ??? explainability.py
?   ?   ??? privacy.py
?   ?   ??? robustness.py
?   ??? report/
?   ?   ??? pdf_export.py
?   ?   ??? report_generator.py
?   ??? ui/
?       ??? dashboard.py
??? examples/
?   ??? example_usage.py
??? tests/
?   ??? test_evaluator.py
??? pyproject.toml
??? README.md
??? requirements.txt
??? setup.py

EthicalReport API

The evaluate(...) API returns an EthicalReport object with:

• risk_score
• decision
• module_scores
• to_dict()
• to_json()
• save_json()
• save_pdf()

Publishing To PyPI

python -m pip install --upgrade build twine
python -m build
python -m twine check dist/*
python -m twine upload dist/*

Before publishing:

• Update the version in beyond_accuracy/__init__.py, setup.py, and pyproject.toml
• Verify package metadata
• Run the test suite
• Build and inspect the generated wheel and source distribution

Screenshots

Add dashboard screenshots here:

• docs/images/dashboard-overview.png
• docs/images/report-downloads.png