Skip to main content

AskPandas

Ask questions about pandas DataFrames and CSV files in plain English. An LLM writes the pandas code, AskPandas runs it in a restricted sandbox and returns the answer.

PyPI version Python versions Tests Downloads License: MIT Ruff

AskPandas is an open-source Python library that lets you query and transform CSV data using natural language. Run it fully local and free with open-source models via Ollama (no API keys, no cloud), or plug in OpenAI, Mistral, Hugging Face Inference Providers or any OpenAI-compatible API.

✨ Why AskPandas?

  • Plain-English analytics: df.chat("total revenue by region, sorted") returns the printed result of real pandas code.
  • Local or hosted LLMs: OllamaLLM for local models, OpenAICompatibleLLM for OpenAI/Mistral/vLLM/LM Studio/Groq/..., HuggingFaceLLM for HF Inference Providers.
  • Multi-DataFrame questions: pass several DataFrames (df0, df1, ...) and ask for joins and comparisons.
  • Restricted execution: generated code runs without imports, file access or dunder attribute access, with a whitelisted set of builtins.
  • Batteries included: chart helpers, data-quality, statistics and performance utilities, query categorization and validation.
  • Lightweight: depends only on pandas, numpy, matplotlib, seaborn, scipy, psutil and requests (no torch/transformers).

⚙️ How it works

  1. AskPandas builds a prompt with your question plus a summary of each DataFrame (columns, dtypes, null counts, sample rows).
  2. The LLM replies with pandas code; AskPandas extracts the code block and validates it.
  3. The code runs in a restricted namespace where your data is available as df0, df1, ... and its printed output is returned. Invalid replies are retried with correction hints, LLM errors are retried with backoff.

Safety note: the sandbox blocks the common escape routes (imports, open, dunder access, eval/exec), but it is a best-effort guard, not a security boundary. Only run AskPandas with models and data you trust, and do not expose it to untrusted users.

🚀 Quick Start (5 minutes!)

1. Install AskPandas

pip install askpandas

2. Install Ollama (one command)

# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: Download from https://ollama.com/download

3. Pull a lightweight model

ollama pull phi3:mini    # Very small, very fast

4. Start Ollama

ollama serve

5. Start analyzing data!

Prefer a hosted model? See Using hosted LLMs below.

import askpandas as ap

# Set up AI
llm = ap.OllamaLLM(model_name="phi3:mini")
ap.set_llm(llm)

# Load your data
df = ap.DataFrame("your_data.csv")

# Ask questions in plain English!
result = df.chat("What is the total revenue?")
print(result)

🚀 See It In Action!

import askpandas as ap
import pandas as pd

# Create sample data
data = {
    'product': ['Apple', 'Banana', 'Orange'],
    'price': [2.50, 1.00, 1.50],
    'quantity': [100, 200, 150]
}
df = pd.DataFrame(data)
df['revenue'] = df['price'] * df['quantity']

# Create AskPandas DataFrame
sales_df = ap.DataFrame(df)

# AI-powered analysis
result = sales_df.chat("What is the total revenue?")
print(result)
# Output (exact wording depends on the model): Total Revenue: $675.00

# More complex queries
result = sales_df.chat("Show me the top 3 products by revenue")
# Output: Product analysis with rankings

result = sales_df.chat("Calculate average price by product")
# Output: Average Price: $1.67

🎯 What Can You Do? (Everything!)

📊 Data Analysis - Just Ask!

# Basic questions
df.chat("What is the average price?")
# Output: Average Price: $1.67

df.chat("Show me the top 5 customers by revenue")
# Output: Customer rankings with revenue amounts

df.chat("How many sales were made in each region?")
# Output: Regional sales breakdown

# Complex analysis
df.chat("""
    Analyze our sales performance:
    1. Calculate total revenue by month
    2. Show the trend over time
    3. Identify the best performing products
    4. Create a visualization
""")
# Output: Comprehensive analysis with insights

🎨 Visualizations

# Ask for a chart; the generated code saves it with save_plot() to askpandas_plots/
df.chat("Create a bar chart of sales by region")
df.chat("Plot revenue trends over time")
df.chat("Show correlation between price and quantity")
df.chat("Display distribution of customer ages")

🔍 Data Quality & Cleaning

# Automatic data assessment
df.chat("Check for missing values and duplicates")
df.chat("Identify outliers in numeric columns")
df.chat("Clean column names and standardize formats")
df.chat("Validate data types and suggest improvements")

🌐 Multi-Dataset Analysis

# Work with multiple files
customers = ap.DataFrame("customers.csv")
orders = ap.DataFrame("orders.csv")
products = ap.DataFrame("products.csv")

# Cross-dataset insights
ap.chat("""
    Customer analysis:
    1. Join customers with their orders
    2. Calculate lifetime value by segment
    3. Show purchase patterns
    4. Identify high-value customers
""", customers, orders, products)

💡 Real-World Examples

📈 Sales Analysis

import askpandas as ap

# Load sales data
sales = ap.DataFrame("sales_data.csv")

# Comprehensive sales report
sales.chat("What is our total revenue?")
# Output: Total Revenue: $78,586.11

sales.chat("Show me the top 3 products by revenue")
# Output: Product rankings with revenue amounts

sales.chat("Calculate average order value by region")
# Output: Regional performance metrics

sales.chat("How many sales were made in each region?")
# Output: Regional sales breakdown

👥 Customer Analytics

# Customer behavior analysis
customers = ap.DataFrame("customers.csv")
transactions = ap.DataFrame("transactions.csv")

ap.chat("""
    Customer behavior insights:
    1. Customer lifetime value analysis
    2. Purchase frequency patterns
    3. Churn prediction factors
    4. Customer satisfaction metrics
    5. Personalized marketing recommendations
""", customers, transactions)

📊 Financial Analysis

# Financial data processing
financial = ap.DataFrame("financial_data.csv")

financial.chat("""
    Financial performance review:
    1. Profit and loss analysis
    2. Cash flow trends
    3. Expense categorization
    4. Budget vs actual comparison
    5. Financial ratios and KPIs
    6. Risk assessment and recommendations
""")

🔬 Research & Academic

# Research data analysis
research = ap.DataFrame("research_data.csv")

research.chat("""
    Statistical analysis:
    1. Descriptive statistics for all variables
    2. Correlation analysis between key factors
    3. Hypothesis testing results
    4. Outlier detection and treatment
    5. Data distribution visualizations
    6. Statistical significance testing
""")

☁️ Using hosted LLMs (OpenAI, Mistral, Hugging Face)

import askpandas as ap

# OpenAI (reads OPENAI_API_KEY)
ap.set_llm(ap.OpenAICompatibleLLM.openai("gpt-4o-mini"))

# Mistral (reads MISTRAL_API_KEY)
ap.set_llm(ap.OpenAICompatibleLLM.mistral("mistral-small-latest"))

# Hugging Face Inference Providers (reads HF_TOKEN)
ap.set_llm(ap.HuggingFaceLLM("Qwen/Qwen2.5-Coder-7B-Instruct"))

# Any OpenAI-compatible server (vLLM, LM Studio, Ollama's /v1, Groq, Together, ...)
ap.set_llm(ap.OpenAICompatibleLLM("my-model", base_url="http://localhost:8000/v1", api_key="..."))

df = ap.DataFrame("demo_sales.csv")
print(df.chat("What is the total revenue by region? Sort descending."))

Verified output with gpt-4o-mini on the bundled demo_sales.csv (matches a direct pandas groupby):

Total Revenue by Region:
region
South    50557.76
North    40075.88
East     30787.38
West     26978.72
Name: revenue, dtype: float64

Environment variables

Variable Used by
OPENAI_API_KEY OpenAICompatibleLLM.openai() / OpenAICompatibleLLM(...) default
MISTRAL_API_KEY OpenAICompatibleLLM.mistral()
HF_TOKEN (or HF_API_KEY) HuggingFaceLLM

Ollama needs no key; point OllamaLLM(host=...) at a remote server if it is not on localhost:11434.

🧩 API Overview

Object What it does
ap.DataFrame(data) / AskDataFrame Wrap a pandas DataFrame, dict, list of records, or a .csv/.json path. Adds .chat(), .info(), .get_summary_stats(), .get_column_info(), .clean_columns(); other attributes pass through to pandas.
ap.set_llm(llm) / ap.get_llm() Set/get the global LLM used by .chat() and ap.chat().
ap.chat(query, *dfs) Ask a question across several DataFrames (df0, df1, ...).
OllamaLLM, OpenAICompatibleLLM, HuggingFaceLLM, BaseLLM LLM clients; subclass BaseLLM (implement generate and is_available) to add your own.
AskPandasEngine(llm) The engine behind .chat(); keeps execution history (get_execution_history(), get_engine_stats(), export_execution_history()).
ap.analyze_query, ap.validate_query, ap.get_query_examples Query categorization, validation against columns, example prompts.
ap.set_config(...) / ap.get_config() Global settings (verbosity, plot style, output directory, ...).
askpandas.visualization.charts create_bar_chart, create_line_chart, create_scatter_plot, create_histogram, create_correlation_heatmap, create_box_plot, save_plot.
askpandas.utils data_quality, statistical and performance helper modules.

🛠️ Advanced Features

🔧 Custom Configuration

import askpandas as ap

# Set your preferences
ap.set_config(
    verbose=True,                    # See what's happening
    plot_style="seaborn",           # Beautiful charts
    output_dir="my_analysis",       # Save results here
    max_execution_time=120,         # Allow longer analysis
    enable_history=True             # Track all queries
)

🎨 Custom Visualizations

# Create custom charts
from askpandas.visualization.charts import create_bar_chart, save_plot

# Custom bar chart
fig = create_bar_chart(
    df.df,
    x_col="category",
    y_col="value",
    title="My Custom Chart",
    figsize=(12, 8)
)

# Save with high quality
save_plot(fig, "custom_chart.png", dpi=300)

🔍 Query Intelligence

# Get help with your queries
query = "Show me sales trends"
analysis = ap.analyze_query(query)
print(f"Query type: {analysis['primary_category']}")

# Get suggestions
suggestions = ap.get_query_examples('visualization')
print("Try these:", suggestions[:3])

# Validate your query
validation = ap.validate_query(query, df.columns)
if validation['is_valid']:
    print("✅ Query is valid!")

🚀 Performance Tips for Best Results

💪 Optimize Your Queries

# ✅ Good - Specific and clear
df.chat("Calculate total revenue by month for 2024, excluding returns")

# ❌ Avoid - Too vague
df.chat("Analyze this data")

# ✅ Good - Step-by-step analysis
df.chat("""
    1. Filter data for Q4 2024
    2. Group by product category
    3. Calculate sum of revenue
    4. Sort by revenue descending
    5. Show top 10 results
""")

# ✅ Good - Include context
df.chat("Show customer retention rate, considering customers who made purchases in both 2023 and 2024")

🎯 Choose the Right Model

# For speed and basic analysis
llm = ap.OllamaLLM(model_name="phi3:mini")      # Fastest

# For better quality and complex queries
llm = ap.OllamaLLM(model_name="mistral:7b")     # Balanced

# For best results (slower)
llm = ap.OllamaLLM(model_name="llama3.2:13b")   # Highest quality

📊 Data Preparation Tips

# Clean your data first
df = ap.DataFrame("messy_data.csv")

# Ask AskPandas to help clean it
df.chat("""
    Help me clean this data:
    1. Identify and handle missing values
    2. Remove duplicates
    3. Fix data type issues
    4. Standardize column names
    5. Show me what was cleaned
""")

# Then analyze the clean data
df.chat("Now analyze the cleaned data for insights")

🔧 Installation Options

Basic Installation

pip install askpandas
pip install "askpandas[full]"

Development Installation

git clone https://github.com/irfanalidv/AskPandas
cd AskPandas
pip install -e ".[dev]"

📱 Platform Support

  • ✅ macOS, Linux and Windows (pure Python; Python 3.10-3.13)
  • ✅ Notebooks and cloud environments such as Google Colab (use a hosted LLM or a reachable Ollama server)

🆘 Troubleshooting

Common Issues & Solutions

"No LLM configured" error?

# Make sure Ollama is running
ollama serve

# Check if model is downloaded
ollama list

Slow responses?

# Try a smaller model
ollama pull phi3:mini

# Close other applications to free memory

Installation issues?

# Update pip
pip install --upgrade pip

# AskPandas requires Python 3.10+
python3.12 -m pip install askpandas

📚 Learning Resources

Interactive Examples

# Run the interactive demo
python simple_demo.py

# Try the configuration setup
python simple_config.py

Sample Datasets

  • fake_sample.csv - Small sample for testing
  • comprehensive_sample.csv - Larger dataset for practice
  • Create your own CSV files and start analyzing!

🚀 Roadmap

Next

  • Jupyter notebook integration
  • More visualization options (Plotly, Bokeh)
  • SQL query generation
  • Data pipeline automation

Version 1.0.0 (Future)

  • Enterprise features
  • Advanced ML integration
  • Real-time data streaming
  • Community plugins

🤝 Get Help & Contribute

📄 License

MIT License - see LICENSE. Use freely for personal and commercial projects!

🙏 Acknowledgments

  • Ollama Team - Making local AI accessible
  • HuggingFace - Open-source AI models
  • Pandas Community - Amazing data tools
  • Open Source Contributors - Building the future together

🎯 Complete Working Demonstration

Want to see everything in action? Run our comprehensive demo:

# Clone the repository
git clone https://github.com/irfanalidv/AskPandas.git
cd AskPandas

# Run the complete demonstration
python final_working_demo.py

This demo showcases:

  • ✅ DataFrame Creation & Analysis - Real data processing
  • ✅ Data Quality & Cleaning - Automatic column standardization
  • ✅ AI-Powered Queries - Natural language analysis
  • ✅ Multi-Dataset Analysis - Joining and complex queries
  • ✅ Configuration Management - Customizable settings
  • ✅ Query Intelligence - Automatic query categorization

🚀 Quick Test

import askpandas as ap
import pandas as pd

# Create test data
data = {'name': ['Alice', 'Bob'], 'age': [25, 30], 'salary': [50000, 60000]}
df = pd.DataFrame(data)
ap_df = ap.DataFrame(df)

# Test basic methods
print(f"Shape: {ap_df.shape()}")  # Output: Shape: (2, 3)
print(f"Columns: {ap_df.columns()}")  # Output: Columns: ['name', 'age', 'salary']

# Get comprehensive info
print(ap_df.info())  # Output: Detailed DataFrame information

# Statistical description
print(ap_df.describe())  # Output: Statistical summary

🚀 Ready to transform your data analysis? Install AskPandas today!

pip install askpandas

Made with ❤️ by Md Irfan Ali

Metadata

Release files for askpandas 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for askpandas 0.2.0
File Size Uploaded Provenance Actions
askpandas-0.2.0.tar.gz 64.4 kB Without provenance Details for askpandas-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for askpandas 0.2.0
File Interpreter ABI Platform Actions
askpandas-0.2.0-py3-none-any.whl Python 3 none any Details for askpandas-0.2.0-py3-none-any.whl

Total release size: 117.3 kB

Release files / askpandas-0.2.0.tar.gz

Download URL askpandas-0.2.0.tar.gz
Size 64.4 kB
Tags Source
SHA-256 checksum
How to use checksums
93a5a1622f4df25c28ccf160a44591175ee199dfa4d6785e9ac7d24e82478e33
BLAKE2b-256 checksum
How to use checksums
7f898b3bb61b1f782e6ad228924319c6439207c74a03cea0abc432dab962effa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.7

Release files / askpandas-0.2.0-py3-none-any.whl

Download URL askpandas-0.2.0-py3-none-any.whl
Size 52.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
78eea8193938908c5f39ad7e33610a65a96e514f5024b573e35fdb8522c489ca
BLAKE2b-256 checksum
How to use checksums
77e318e4f045bc59114b054e8044d11b5e106ea66cdab89d8a365f141c7870b1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.7

Security for askpandas 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page