AskPandas
Ask questions about pandas DataFrames and CSV files in plain English: an LLM writes the pandas code, AskPandas runs it in a restricted sandbox and returns the answer.
Why AskPandas
Writing a one-off groupby / merge / chart for every ad-hoc question is slow, and pasting a
dataset into a chat window loses the exact numbers. AskPandas keeps your data local: the LLM sees
only a compact summary of each DataFrame (columns, dtypes, null counts, three sample rows) and
replies with pandas code, which AskPandas validates and executes against the real data. You get
results computed by pandas, not numbers guessed by a model, and you can use a free local model
via Ollama or any OpenAI-compatible API.
Features
| Plain-English analytics | df.chat("total revenue by region, sorted") returns what the generated pandas code printed. |
| Local or hosted LLMs | OllamaLLM (local), OpenAICompatibleLLM (OpenAI, Mistral, Groq, Together, vLLM, LM Studio, Ollama /v1), HuggingFaceLLM (HF Inference Providers), or your own two-method class. |
| Several DataFrames at once | ap.chat(question, orders, customers) exposes them as df0, df1, ... for joins and comparisons. |
| Restricted execution | Generated code runs without imports, function/class definitions, dunder access or file builtins, with a whitelist of builtins. |
| Self-correcting loop | Invalid code is retried with correction hints; LLM errors are retried with exponential backoff. |
| Charts | Chart helpers are available to generated code; figures are saved to askpandas_plots/ (configurable). |
| Offline helpers | Data-quality scoring and auto-cleaning, statistical tests (t-test, ANOVA, chi-square, Mann-Whitney, Kruskal-Wallis), dtype/memory optimisation. No LLM needed. |
| Lightweight and typed | Depends on pandas, numpy, matplotlib, seaborn, scipy, psutil and requests only (no torch). Ships py.typed; mypy-clean. |
Install
pip install askpandas
# or
uv pip install askpandas
Extras: askpandas[full] adds Jupyter, ipywidgets and Plotly; askpandas[dev] adds the test and
lint toolchain. Requires Python 3.10+.
Quickstart
With a hosted model (needs network and OPENAI_API_KEY):
import askpandas as ap
ap.set_llm(ap.OpenAICompatibleLLM.openai("gpt-4o-mini")) # reads OPENAI_API_KEY
# Local and free instead: ap.set_llm(ap.OllamaLLM("llama3.1")) # needs `ollama serve`
sales = ap.DataFrame("examples/data/demo_sales.csv")
print(sales.chat("What is the total revenue by region? Sort descending."))
Output from a run with gpt-4o-mini on the bundled sample file (the same numbers as a direct
pandas groupby; the wording of the header line depends on the model):
Total Revenue by Region:
region
South 50557.76
North 40075.88
East 30787.38
West 26978.72
Name: revenue, dtype: float64
Without any model or key, plug in an object with generate and is_available to see the
pipeline run end to end:
import askpandas as ap
class FixedLLM(ap.BaseLLM):
"""Stands in for a model: always answers with the same pandas code."""
def generate(self, prompt: str) -> str:
return "```python\nprint(df0.groupby('team')['tickets'].sum().idxmax())\n```"
def is_available(self) -> bool:
return True
ap.set_llm(FixedLLM())
tickets = ap.DataFrame({"team": ["billing", "shipping", "shipping"], "tickets": [12, 30, 9]})
print(tickets.chat("Which team handled the most tickets?")) # shipping
How it works
flowchart TD
U["df.chat(q) / ap.chat(q, df0, df1, ...)"] --> E[AskPandasEngine.process_query]
E --> C["Context per DataFrame<br/>columns, dtypes, nulls, 3 sample rows,<br/>dtype hints, IQR outlier counts"]
E --> K["Classify query (regex)<br/>analysis / visualization / data_quality /<br/>statistical / multi_dataframe"]
C --> P[Prompt = template + sandbox rules + context + question]
K --> P
P --> L{"BaseLLM.generate<br/>OllamaLLM / OpenAICompatibleLLM /<br/>HuggingFaceLLM / custom"}
L -->|error| B[Exponential backoff, retry up to 3x] --> L
L --> X["Extract first fenced code block,<br/>drop import lines, dedent"]
X --> V{"compile() + pattern check"}
V -->|invalid| H[Add correction hints, retry] --> L
V -->|still invalid after 3 tries| F[Fallback: describe df0]
V -->|valid| S["SafeExecutor<br/>AST check: no import / def / class / dunder<br/>whitelisted builtins; df0..dfN, pd, np, plt, sns,<br/>save_plot and chart helpers"]
F --> S
S --> O["Captured stdout (+ stderr)<br/>unsaved figures saved and closed"]
O --> R[Execution history + return text]
Design decisions:
- The model writes code, pandas computes. Only a summary of each DataFrame goes into the prompt, so large tables do not hit context limits and the numbers come from pandas.
- One namespace for
exec. Globals and locals are the same dict so comprehensions can seedf0,df1, ... on every supported Python version. - Errors are values inside the sandbox. A failing snippet returns a message (and is recorded
as a failure in the engine history) instead of raising, so interactive sessions keep going.
Problems outside the sandbox raise typed exceptions from
askpandas.exceptions. - Any backend with two methods. The engine needs only
generate(prompt) -> strandis_available() -> bool; the bundled clients use plainrequestswith timeouts.
Safety note: the sandbox blocks the common escape routes (imports,
open, dunder access,eval/exec, function definitions), but it is a best-effort guard, not a security boundary: pandas itself can read and write files. Only run AskPandas with models and data you trust, and do not expose it to untrusted users.
Use cases and recipes
All examples run offline with no API key (the LLM examples use a small scripted backend,
offline_llm.py).
| Script | What it shows |
|---|---|
sales_kpis.py |
Revenue by region, top products and a saved monthly-revenue chart from a CSV. |
customer_segments_multi_df.py |
One question that joins orders and customers (df0/df1) to compare segments and find inactive customers. |
data_quality_audit.py |
Quality score, auto-cleaning, ANOVA and Spearman correlations on 1,000 rows, no LLM at all. |
custom_llm_backend.py |
A custom BaseLLM that logs prompts, the sandbox rejecting a dunder escape, execution history. --openai runs it against OpenAI. |
git clone https://github.com/irfanalidv/AskPandas && cd AskPandas
pip install -e .
python examples/sales_kpis.py
More things to ask
df.chat("Create a bar chart of revenue by region") # saved to askpandas_plots/
df.chat("Check for missing values and duplicates")
df.chat("Identify outliers in numeric columns")
df.chat("Show the correlation between price and quantity")
ap.chat("Join customers with their orders and compute revenue per segment", customers, orders)
Specific questions work best ("total revenue by month for 2024, excluding returns" rather than
"analyze this data"). Small local models (for example phi3:mini) are fast but follow the code
format less reliably than larger ones.
Configuration
Environment variables
| Variable | Used by |
|---|---|
OPENAI_API_KEY |
OpenAICompatibleLLM.openai() and the OpenAICompatibleLLM(...) default |
MISTRAL_API_KEY |
OpenAICompatibleLLM.mistral() |
HF_TOKEN (or HF_API_KEY) |
HuggingFaceLLM |
MPLBACKEND=Agg |
Recommended on headless machines so charts render without a display |
Ollama needs no key; point OllamaLLM(host=...) at a remote server if it is not on
localhost:11434.
ap.set_config(...) keys
| Key | Default | Effect |
|---|---|---|
output_dir |
"askpandas_plots" |
Directory where save_plot and auto-saved figures go. |
plot_style |
"seaborn" |
Applied to matplotlib when set via set_config (seaborn, minimal, dark, default). |
verbose |
False |
Logs LLM and config changes on the askpandas logger at INFO level. |
model_name, temperature, max_tokens, enable_plotting, safe_mode, max_execution_time, enable_history |
see ap.get_config() |
Stored for forward compatibility; not enforced yet. Set temperature, tokens and timeouts on the LLM client instead. |
Client and engine options
| Option | Where | Default |
|---|---|---|
temperature, timeout |
OllamaLLM(...) |
0.1, 120 s |
base_url, api_key, temperature, max_tokens, timeout, system_prompt |
OpenAICompatibleLLM(...) |
OpenAI URL, env var, 0.1, 1000, 60 s, code-only prompt |
retry_backoff |
AskPandasEngine(llm, config={"retry_backoff": 1.0}) |
1.0 s, doubled per retry |
API overview
| Object | What it does |
|---|---|
ap.DataFrame(data) / AskDataFrame |
Wrap a DataFrame, dict, list of records, or a .csv/.json path (str or Path). Adds .chat(), .info(), .get_summary_stats(), .get_column_info(), .clean_columns(); other attributes pass through to pandas. |
ap.set_llm(llm) / ap.get_llm() |
Set or get the global LLM used by .chat() and ap.chat(). |
ap.chat(query, *dfs) |
Ask one question across several DataFrames (df0, df1, ...). |
OllamaLLM, OpenAICompatibleLLM, HuggingFaceLLM, BaseLLM |
LLM clients; subclass BaseLLM (or duck-type generate and is_available) to add your own. |
AskPandasEngine(llm) |
The engine behind .chat(); get_execution_history(), get_engine_stats(), get_performance_metrics(), export_execution_history("json" | "csv"). |
ap.analyze_query, ap.validate_query, ap.get_query_examples |
Regex-based query categorisation, column-reference checks, example prompts. |
ap.set_config / ap.get_config |
Global settings (see Configuration). |
create_bar_chart, create_line_chart, create_scatter_plot, create_histogram, create_correlation_heatmap, create_box_plot, save_plot |
Chart helpers (also available inside generated code). |
askpandas.utils.data_quality |
assess_data_quality, clean_dataframe, generate_quality_report, DataQualityAnalyzer, DataCleaner. |
askpandas.utils.statistical |
StatisticalAnalyzer (descriptives, correlations with p-values, outliers, normality, hypothesis tests), quick_stats, detect_outliers. |
askpandas.utils.performance |
optimize_dataframe, PerformanceOptimizer, PerformanceBenchmark, benchmark_function. |
askpandas.exceptions |
AskPandasError and subclasses DataLoadError, LLMNotConfiguredError, LLMError, CodeGenerationError, QueryProcessingError (each also a ValueError or RuntimeError, as in earlier releases). |
Statistical methods used
The offline helpers wrap standard methods from SciPy:
- Outlier fences at 1.5 x IQR: Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.
- Normality: Shapiro, S. S. & Wilk, M. B. (1965). An analysis of variance test for normality. Biometrika, 52(3/4), 591-611; plus a Kolmogorov-Smirnov test against a fitted normal.
- Rank tests: Mann, H. B. & Whitney, D. R. (1947). Annals of Mathematical Statistics, 18(1), 50-60; Kruskal, W. H. & Wallis, W. A. (1952). Journal of the American Statistical Association, 47(260), 583-621.
- Effect sizes: Cohen's d for t-tests, eta squared for ANOVA, Cramer's V for chi-square.
When to use it
AskPandas fits exploratory analysis in scripts and notebooks where you trust the model and the
data and want answers computed by pandas. It is deliberately small: the core is a prompt builder,
a code extractor and a sandboxed exec. If you need a larger feature set (for example connectors
to many data sources or a hosted UI), look at bigger projects in this space such as
PandasAI. For untrusted users or untrusted models, run
generated code in a real isolation layer (container or separate process) rather than relying on
an in-process sandbox.
Roadmap
- Enforce
max_execution_timewith a subprocess-based executor. - Return structured results (the last DataFrame or value) alongside printed text.
- Async LLM clients and streaming.
Contributing, security, citation, license
- Contributing: see CONTRIBUTING.md
(
ruff,mypyandpytest --covmust pass; all tests run offline). - Security: please report vulnerabilities as described in SECURITY.md.
- Changelog: CHANGELOG.md.
- Citation: if you use AskPandas in research, cite it using CITATION.cff (GitHub's "Cite this repository" button reads it).
- License: MIT, see LICENSE.
Metadata
Release files for askpandas 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| askpandas-0.3.0.tar.gz | 117.4 kB | Without provenance | Details for askpandas-0.3.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| askpandas-0.3.0-py3-none-any.whl | Python 3 | none | any | Details for askpandas-0.3.0-py3-none-any.whl |
Total release size: 174.2 kB
Release files / askpandas-0.3.0.tar.gz
| Download URL | askpandas-0.3.0.tar.gz |
|---|---|
| Size | 117.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
054d0b25aad9ef3dc6815af77e93423c0460b3785211b289bb9f582f4a643a02
|
|
BLAKE2b-256 checksum How to use checksums |
bac92c93fcd3d85dbf12bf25cdcc01a30a8993b539c527ba0c523bbc27580e02
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.7
|
Release files / askpandas-0.3.0-py3-none-any.whl
| Download URL | askpandas-0.3.0-py3-none-any.whl |
|---|---|
| Size | 56.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
cd088a165b39fd0cd112d96dc5eda10baca0b2e22ec2c33617ff52aade5e57bf
|
|
BLAKE2b-256 checksum How to use checksums |
bcd134e5c0035ad843ad1e8dbe10246701048489836c66dd3eb23bf3ae476c60
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.7
|
Security for askpandas 0.3.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.