SMERVisual
SMERVisual is a Python package designed for explainable machine learning using the Self Model Entities Related (SMER) method. It provides tools for explainable classification of images with LLM-generated text descriptions, which are then analyzed using the SMER explanation technique.
The package supports both OpenAI API models and local language models, offering flexibility in model selection.
Installation
Install SMERVisual using pip:
pip install smer-visual
Features
Image Description
The image_description function generates text descriptions for images using either OpenAI models or local language models. Key features include:
- Support for OpenAI models like
gpt-4o-miniand local models. - Customizable prompts for generating concise and informative descriptions.
Description Embeddings
The get_description_embeddings function computes embeddings for image descriptions using OpenAI or local models.
Explainable Classification
The classify_with_logreg function performs logistic regression-based classification on image datasets while computing SMER values for explainability. Key features include:
- Aggregation of embeddings for classification.
- Computation of feature importance for each word in the description.
- Support for AOPC (Area Over the Perturbation Curve) analysis.
Visualization
- Important Words: The
plot_important_wordsfunction visualizes the most important words across images. - AOPC Curves: The
plot_aopcfunction compares the explainability of SMER and LIME methods by plotting AOPC curves. - Bounding Boxes: The
BoundingBoxGeneratorclass overlays bounding boxes on images, highlighting critical words identified in classification.
Usage Example
Image Description and Embeddings
from smer_visual.smer import image_description, get_description_embeddings
# Generate image descriptions
descriptions = image_description(
model="gpt-4o-mini",
data_folder="path/to/images",
api_key="your_openai_api_key"
)
# Generate embeddings for descriptions
embeddings_df = get_description_embeddings(
descriptions=descriptions,
embedding_model="text-embedding-ada-002",
api_key="your_openai_api_key"
)
Explainable Classification
from smer_visual.smer import classify_with_logreg, aggregate_embeddings
from sklearn.linear_model import LogisticRegression
import numpy as np
# Prepare data
embeddings_df["aggregated_embedding"] = embeddings_df["embedding"].apply(aggregate_embeddings)
X_train = np.stack(embeddings_df["aggregated_embedding"].values)
y_train = embeddings_df["label"]
# Train a logistic regression model
logreg_model = LogisticRegression()
logreg_model.fit(X_train, y_train)
# Perform classification and compute feature importance
aopc_df, updated_dataset = classify_with_logreg(embeddings_df, X_train, logreg_model)
Visualization
from smer_visual.smer import plot_important_words, plot_aopc
# Plot important words
plot_important_words(updated_dataset)
# Plot AOPC curves
plot_aopc(aopc_df, logreg_model, max_k=5)
Bounding Box Generation
from smer_visual.smer import save_bounding_box_images
results = save_bounding_box_images(
input_path="data/",
output_folder="output",
df_top_words=top_words_df,
model_id="IDEA-Research/grounding-dino-base",
box_threshold = 0.5,
text_threshold = 0.4
)
Why Use SMERVisual?
- Explainable AI – Provides insight into model decision-making.
- Model-Agnostic – Compatible with OpenAI APIs and open-source models.
- Zero-Shot Detection – No additional training data required.
- Easy Integration – Simple API for seamless use with existing machine learning workflows.
Contributing
Contributions are welcome. If you’d like to contribute, follow these steps:
- Fork the repository.
- Create a new branch (
git checkout -b feature-branch). - Commit changes (
git commit -m "Add new feature"). - Push to the branch (
git push origin feature-branch). - Open a pull request.
License
SMERVisual is released under the MIT License.
Metadata
Release files for smer-visual 1.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| smer_visual-1.1.1.tar.gz | 13.8 kB | Without provenance | Details for smer_visual-1.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| smer_visual-1.1.1-py3-none-any.whl | Python 3 | none | any | Details for smer_visual-1.1.1-py3-none-any.whl |
Total release size: 26.3 kB
Release files / smer_visual-1.1.1.tar.gz
| Download URL | smer_visual-1.1.1.tar.gz |
|---|---|
| Size | 13.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
62e50778ac8cfd1370a4f575ac57b517bb73837a46f08bc543812807baec490c
|
|
BLAKE2b-256 checksum How to use checksums |
e5e6a24bf0c13a1ccc7e5d820bee1310c25200ed9533ce5db3d86a1205318d51
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.7
|
Release files / smer_visual-1.1.1-py3-none-any.whl
| Download URL | smer_visual-1.1.1-py3-none-any.whl |
|---|---|
| Size | 12.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
efadfd40f7d6bdfd8b76556e0a3c3d51a196991f60b094864fbc6aa4399d88aa
|
|
BLAKE2b-256 checksum How to use checksums |
e33ddfc085e748a4f016d218542c089b08624615b67ddf170df857c9ce4f67cd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.7
|
Security for smer-visual 1.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.