meno 0.2.0

Topic modeling toolkit for messy text data

Topic Modeling Toolkit: meno

Overview

This Python package, meno, is designed to streamline topic modeling on free text data, with a special focus on messy datasets such as insurance claims notes and customer correspondence. The package combines classical methods like Latent Dirichlet Allocation (LDA) with modern techniques leveraging large language models (LLMs) via Hugging Face, dimensionality reduction with UMAP, and advanced visualizations. It is built to be primarily used in Jupyter environments while also being flexible enough for other settings.

Key Features

• Unsupervised Topic Modeling:
  • Automatically discover topics when no pre-existing topics are available using LDA and LLM-based embedding and clustering techniques. (See details in Architecture section below)
• Supervised Topic Matching:
  • Match free text against a user-provided list of topics using semantic similarity and classification techniques. (See details in Architecture section below)
• Advanced Visualization:
  • Create interactive and static visualizations including topic distributions, embeddings (UMAP projections), cluster analyses, and topic coherence metrics (e.g., word clouds per topic).
• Interactive HTML Reports:
  • Generate standalone, interactive HTML reports to present topic analysis to less technical stakeholders, with options for customization and data export.
• Robust Data Preprocessing:
  • Tackle messy data challenges (misspellings, unknown acronyms) with integrated cleaning functionalities using NLP libraries (spaCy, fuzzy matching, context-aware spelling correction, and customizable stop words/lemmatization rules.). (See details in Architecture section below)
• Active Learning with Cleanlab:
  • Incorporate active learning loops and fine-tuning of labels using Cleanlab, facilitating hand-labeling and iterative improvements, with multiple sampling strategies (e.g., uncertainty sampling).
• Extensibility & Ease of Use:
  • Designed with modularity in mind so that users can plug in new cleaning, modeling, or visualization techniques without deep customization while still maintaining a simple interface.

Architecture & Design

The package follows a modular design with clear separation of concerns, ensuring that each component can be developed, tested, and extended independently:

Data Preprocessing Module:

Handles cleaning tasks such as:

• Spelling Correction: Utilizes fuzzywuzzy for basic corrections and incorporates context-aware spelling correction using a pre-trained language model (e.g., a smaller, faster model than the main LLMs used for topic modeling).
• Acronym Resolution: Addresses acronyms using a combination of a user-expandable predefined dictionary, rule-based pattern matching (e.g., identifying all-caps words), and, optionally, leveraging LLMs to disambiguate acronyms based on context.
• Normalization: Handles tasks like lowercasing, punctuation removal, and stemming/lemmatization.
• Customizable Stop Words and Lemmatization: Allows users to easily define and apply custom stop word lists and lemmatization rules specific to their domain via configuration files.

Topic Modeling Module:

Unsupervised Modeling:

• LDA: Implements Latent Dirichlet Allocation using gensim.
• LLM-based Topic Extraction:
  • Embedding Generation: Uses pre-trained LLMs from Hugging Face Transformers (e.g., Sentence Transformers like all-MiniLM-L6-v2 as a default, with options for users to specify other models) to generate document embeddings.
  • Clustering: Applies clustering algorithms (e.g., HDBSCAN, K-Means) to the embeddings to identify topic clusters. Provides options for users to adjust clustering parameters.
  • Optional LLM Fine-tuning: Consider adding this as a later feature. It would involve allowing users to fine-tune the LLM on their data.

Supervised Matching:

• Semantic Similarity: Uses cosine similarity between document embeddings (generated by the same LLMs as in unsupervised modeling) and embeddings of user-provided topic descriptions.
• Classification: (Optional, for later development): Trains a classifier (e.g., a Transformer model) on labeled data to directly predict topic labels.
• Topic Input Format: Users provide topics as a list of strings (topic names) and, optionally, short descriptions for each topic. The system will generate embeddings for these descriptions.
• Thresholding and "Other" Category: Implements a similarity threshold (configurable by the user) below which a document is assigned to an "Other" or "Unmatched" category.

Visualization Module:

Provides functions to create:

• Static Plots: Topic distribution histograms.
• Interactive Embedding Plots: UMAP projections of document embeddings, colored by topic (either discovered or assigned). Interactivity includes zooming, panning, hovering for details (document text and topic probabilities), and selecting data points.
• Topic Coherence Visualizations: Word clouds for each topic, showing the most important words. Top words can be ranked by probability (for LDA) or by some measure of relevance to the embedding centroid (for LLM-based topics).
• UMAP Parameter Control: Allows users to customize UMAP parameters (e.g., n_neighbors, min_dist) via configuration files.

Report Generation Module:

Generates interactive, standalone HTML reports using Plotly or Bokeh alongside Jinja2 for templating.

• Customization: Allows users to customize the report's appearance (e.g., colors, fonts) and content (e.g., adding a logo, introductory text, conclusions) via configuration files or a simple API.
• Data Export: Provides options to export the underlying data (topic assignments, probabilities, embeddings) in various formats (e.g., CSV, JSON).

Active Learning Module:

Integrates with Cleanlab to facilitate hand-labeling, iterative label fine-tuning, and quality control over topic assignments.

• Wrapper Functions: Provides simplified wrapper functions around Cleanlab's functionality to streamline the active learning process. These functions handle presenting documents for labeling, updating the model (either LDA or the LLM-based similarity matching), and re-evaluating performance.
• Labeling Interface: Provides a basic labeling interface within Jupyter notebooks using ipywidgets. This allows users to view documents and assign topic labels without leaving the notebook environment. (Consider linking to external labeling tools in the future.)
• Sampling Strategies: Offers different sampling strategies, primarily uncertainty sampling (selecting documents where the model is least confident).

Dependencies & Requirements

• Python: 3.10+
• Key Libraries:
  • Topic Modeling: gensim, Hugging Face Transformers
  • Dimensionality Reduction: umap-learn
  • Data Cleaning & NLP: spaCy, fuzzywuzzy, scikit-learn (for some NLP tasks and potentially classification)
  • Visualization: plotly, bokeh
  • Active Learning: cleanlab
  • HTML Templating: jinja2
  • Configuration: PyYAML or json5
• Additional libraries for testing, logging, and documentation (e.g., pytest, sphinx).

Installation & Setup

• Installation via pip (with a setup.py or pyproject.toml file).
• Instructions for installation via Conda.
• Instructions on setting up a virtual environment.
• Detailed dependency installation guide in the README.

Usage in Jupyter

The package is optimized for use within Jupyter notebooks.

Example notebooks demonstrating:

• Loading and preprocessing messy text data.
• Running both unsupervised and supervised topic models.
• Visualizing topics and generating interactive reports.
• Using Cleanlab for active learning and fine-tuning.

Extensibility & Customization

• Modular API: Each component (cleaning, modeling, visualization, reporting) can be replaced or extended.
• Configuration Files: Uses YAML/JSON configuration files (with validation against a schema) to allow users to customize pipelines without modifying code. Provides example configuration files and clear documentation on the available options.
• Plugin System: Consider a simple plugin architecture for users to add custom processing steps or models.

Testing & Continuous Integration

• Unit Tests: Comprehensive tests for each module using pytest.
• Integration Tests: Test end-to-end pipelines on sample datasets.
• CI Pipeline: Automated testing on GitHub Actions for every push and pull request.

Documentation & Examples

• User Guide: Detailed documentation on installation, API usage, and examples of typical workflows.
• Tutorial Notebooks: Step-by-step Jupyter notebooks demonstrating real-world applications.
• API Reference: Generated from docstrings to ensure up-to-date and detailed reference materials.

Contribution Guidelines

• Code of Conduct & Contribution Guide: Clear instructions for contributing, reporting issues, and submitting pull requests.
• Issue Tracker & Feature Requests: Guidance on how to report bugs or request new features.

Examples

Insurance Complaint Analysis

The package includes an example that demonstrates topic modeling on the Australian Insurance PII Dataset from Hugging Face. This dataset contains over 1,500 insurance complaint letters with various types of insurance issues, making it ideal for testing topic modeling capabilities.

To run the insurance example:

# Install required dependencies
pip install -r requirements_insurance_example.txt

# Run the example script
python examples/insurance_topic_modeling.py

This example demonstrates:

• Loading data from Hugging Face Datasets
• Preprocessing insurance complaint letters
• Discovering topics using unsupervised clustering
• Matching documents to predefined insurance-related topics
• Visualizing topic distributions and document embeddings
• Generating an interactive HTML report

The results will be saved in the output directory.

Roadmap & Future Enhancements

• Enhanced NLP Cleaning: Integration of more advanced text normalization and context-aware corrections.
• Additional Topic Models: Explore integration with other topic modeling techniques (e.g., NMF, neural topic models).
• Expanded Reporting Options: Additional templates and customization options for HTML report generation.
• User Feedback Loop: Streamlined integration with active learning systems for continuous model improvement.
• Scalability: Implement support for larger datasets through techniques like mini-batch processing (for LDA) and potentially distributed processing (e.g., using Dask).
• Multilingual Support: Extend support beyond English to other languages by leveraging multilingual LLMs and language-specific NLP tools.
• Fine-tuning LLMs: Allow users to fine-tune LLMs for specific domains or datasets.
• Additional Datasets: Add more example datasets from different domains to showcase the package's versatility.