crisp-t 0.3.0

Qualitative Research support tools in Python!

🔍 CRISP-T (Sense-making from Text and Numbers!)

TL;DR 🚀 CRISP-T is a qualitative research method and a toolkit to perform textual (e.g. topic modelling) and numeric (e.g. decision trees) analysis of mixed datasets for computational triangulation and sense-making using large language models.

Qualitative research involves the collection and analysis of textual data, such as interview transcripts, open-ended survey responses, and field notes. It is often used in social sciences, humanities, and health research to explore complex phenomena and understand human experiences. In addition to textual data, qualitative researchers may also collect quantitative data, such as survey responses or demographic information, to complement their qualitative findings. Additionally, qualitative researchers use external data sources, such as census data or social media data, to provide context and triangulate their findings. Qualitative research is often characterized by its inductive approach, where researchers aim to generate theories or concepts from the data rather than testing pre-existing hypotheses. It emphasizes the importance of data-driven analysis and theory development.

CRISP-T is a method and corresponding open-source tool set to integrate textual data (as a list of documents) and numeric data (as Pandas DataFrame) into structured classes that retain metadata from various analytical processes, such as topic modeling and decision trees. Researchers, with or without GenAI assistance, can define relationships between textual and numerical datasets based on their chosen theoretical lens. A final analytical phase ensures that proposed relationships actually hold true. 👉 See Demo.

An MCP server exposes all functionality as tools, resources, and prompts, enabling integration with AI agent platforms such as Claude desktop, VSCODE and other MCP-compatible clients.

Installation

pip install crisp-t

Include machine learning features for numeric data analysis:

pip install crisp-t[ml]

Include XGBoost for gradient boosting features:

pip install crisp-t[xg]

• Mac users need to install libomp: brew install libomp for XGBoost to work.

Command Line Scripts

CRISP-T now provides four main command-line scripts:

• crisp — Main CLI for qualitative triangulation and analysis (see below)
• crispviz — Visualization CLI for corpus data (word frequencies, topic charts, wordclouds, etc.)
• crispt — Corpus manipulation CLI (create, edit, query, and manage corpus objects)
• crisp-mcp — Starts the MCP server for AI integration (see MCP section below)

All scripts are installed as entry points and can be run directly from the command line after installation.

crisp (Analytical CLI)

crisp [OPTIONS]

Input/Output Options

• --source, -s PATH|URL: Read source data from a directory (reads .txt, .pdf and a single .csv) or from a URL
• --sources PATH|URL: Provide multiple sources; can be used multiple times
• --inp, -i PATH: Load an existing corpus from a folder containing corpus.json (and optional corpus_df.csv)
• --out, -o PATH: When saving the corpus, provide a folder path; the CLI writes corpus.json (and corpus_df.csv if available) into that folder. When saving analysis results (topics, sentiment, etc.), this acts as a base path: files are written with suffixes, e.g., results_topics.json.
• --unstructured, -t TEXT: Text CSV column(s) to analyze/compare (can be used multiple times). This is useful when you have free-form text data in a DataFrame. If this is provided, those columns are used as documents.
• --ignore TEXT: Comma-separated words to ignore during ingestion (applies to --source/--sources)

Analysis Options

• --codedict: Generate qualitative coding dictionary
• --topics: Generate topic model using LDA
• --assign: Assign documents to topics
• --cat: List categories of entire corpus or individual documents
• --summary: Generate extractive text summary
• --sentiment: Generate sentiment scores using VADER
• --sentence: Generate sentence-level scores when applicable
• --nlp: Generate all NLP reports (combines above text analyses)
• --nnet, --cls, --knn, --kmeans, --cart, --pca, --regression, --ml: Machine learning and clustering options (requires crisp-t[ml])
  • --regression: Perform linear or logistic regression (automatically detects binary outcomes for logistic regression)
• --visualize: Generate visualizations (word clouds, topic charts, etc.)
• --num, -n INTEGER: Number parameter (clusters, topics, epochs, etc.) - default: 3
• --rec, -r INTEGER: Record parameter (top N results, recommendations) - default: 3
• --filters, -f TEXT: Filters to apply as key=value (can be used multiple times); keeps only documents where document.metadata[key] == value. Invalid formats raise an error.
• --verbose, -v: Print verbose messages for debugging

Data Sources

• --source, -s PATH|URL: Read source data from a directory (reads .txt and .pdf) or from a URL
• --sources PATH|URL: Provide multiple sources; can be used multiple times

crispviz (Visualization CLI)

crispviz [OPTIONS]

• --inp, --source, --sources: Input corpus or sources
• --out: Output directory for PNG images
• Visualization flags: --freq, --by-topic, --wordcloud, --top-terms, --corr-heatmap
• Optional params: --bins, --top-n, --columns

crispt (Corpus Manipulation CLI)

crispt [OPTIONS]

• --id, --name, --description: Corpus metadata
• --doc: Add document as id|name|text or id|text (repeatable)
• --remove-doc: Remove document by ID (repeatable)
• --meta: Add/update corpus metadata as key=value (repeatable)
• --add-rel: Add relationship as first|second|relation (repeatable)
• --clear-rel: Clear all relationships
• --out: Save corpus to folder/file as corpus.json
• --inp: Load corpus from folder/file containing corpus.json
• Query options:
  • --df-cols: Print DataFrame column names
  • --df-row-count: Print DataFrame row count
  • --df-row INDEX: Print DataFrame row by index
  • --doc-ids: Print all document IDs
  • --doc-id ID: Print document by ID
  • --relationships: Print all relationships
  • --relationships-for-keyword KEYWORD: Print relationships involving a keyword
• Semantic search (requires chromadb):

  • --semantic QUERY: Perform semantic search with query string

  • --semantic-n N: Number of results to return (default: 5)

  • --metadata-df: Export collection metadata as DataFrame+

  • --metadata-keys KEYS: Comma-separated metadata keys to include+

  • • The above two options can be used to export or add metadata from NLP to the DataFrame. For example, you can extract sentiment scores or topic assignments as additional columns for numerical analysis. This is useful if dataframe and documents are aligned as in a survey response.

Example Usage

When saving the corpus via --out, the CLI writes corpus.json (and corpus_df.csv if present) into the specified folder. If you pass a file path, only its parent directory is used for writing corpus.json.

MCP Server

CRISP-T provides a Model Context Protocol (MCP) server that exposes all functionality as tools, resources, and prompts. This enables integration with AI assistants and other MCP-compatible clients.

Using the MCP Server

Configuring MCP Clients

Claude Desktop

Add to your Claude Desktop configuration file:

MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "crisp-t": {
      "command": "<python-path>crisp-mcp"
    }
  }
}

Using with Other MCP Clients

The server can be used with any MCP-compatible client. Configure your client to run the crisp-mcp command via stdio.

Available Tools

The MCP server provides tools for:

Corpus Management

• load_corpus - Load corpus from folder or source
• save_corpus - Save corpus to folder
• add_document - Add new document
• remove_document - Remove document by ID
• get_document - Get document details
• list_documents - List all document IDs
• add_relationship - Link text keywords with numeric columns
• get_relationships - Get all relationships
• get_relationships_for_keyword - Query relationships by keyword

NLP/Text Analysis

• assign_topics - Assign documents to topics (creates keyword labels)
• extract_categories - Extract common concepts
• generate_summary - Generate extractive summary
• sentiment_analysis - VADER sentiment analysis

Semantic Search (requires chromadb)

• semantic_search - Find documents similar to a query using semantic similarity
• export_metadata_df - Export ChromaDB metadata as DataFrame

DataFrame/CSV Operations

• get_df_columns - Get DataFrame column names
• get_df_row_count - Get number of rows
• get_df_row - Get specific row by index

Machine Learning (requires crisp-t[ml])

• kmeans_clustering - K-Means clustering
• decision_tree_classification - Decision tree with feature importance
• svm_classification - SVM classification
• neural_network_classification - Neural network classification
• regression_analysis - Linear/logistic regression with coefficients
• pca_analysis - Principal Component Analysis
• association_rules - Apriori association rules
• knn_search - K-nearest neighbors search

Resources

The server exposes corpus documents as resources:

• corpus://document/{id} - Access document text by ID

Prompts

• analysis_workflow - Complete step-by-step analysis guide based on INSTRUCTIONS.md
• triangulation_guide - Guide for triangulating qualitative and quantitative findings

Example MCP commands

Role of CRISP-T in research and practice

The workflow enables AI assistants to help conduct comprehensive analyses by combining text analytics, machine learning, and triangulation of qualitative-quantitative findings.

For example, in market research, a company collects:

• Textual feedback from customer support interactions.
• Numerical data on customer retention and sales performance. Using this framework, business analysts can investigate how recurring concerns in feedback correspond to measurable business outcomes.

Framework Documentation

For detailed information about available functions, metadata handling, and theoretical frameworks, see the comprehensive user instructions. For semantic search examples and best practices, see the Semantic Search Guide. Documentation (WIP) is also available here.

Citation

• Released on 10/11/2025 for presentation at ICIS 2025 conference.
• Paper coming soon. Cite this repository in the meantime:

Give us a star ⭐️

If you find this project useful, give us a star. It helps others discover the project.

Contact

• Bell Eapen (UIS) | Contact |