Skip to main content

AK-SCI

A unified, AI-assisted data science toolkit. Author: Amanullah Khan

AK-SCI does four concrete things:

  1. Diagnoses runtime errors using a small, embedded, offline machine-learning model (TF-IDF + Logistic Regression, trained on 130+ examples across 16 categories of common Python / data-science / automation errors) — no API key or internet connection needed. An optional cloud AI-assist mode (needs your own API key) adds deeper natural-language explanations for trickier cases.
  2. Optionally auto-fixes a narrow, whitelisted set of safe recoveries — fuzzy-matching a mistyped column name, filling a blank numeric field, guarding a division by zero, coercing a string to a number — entirely opt-in and always logged. See Auto-fix below.
  3. Streams large datasets through memory-bounded chunks, so a pipeline built with MicroPipeline can process CSV files far bigger than available RAM without loading the whole thing at once.
  4. Wraps pandas, Polars, NumPy, SciPy, and scikit-learn behind one consistent API (UnifiedFrame, ml, stats), so common operations — standardizing a dataframe, fitting a regression, running a t-test — read the same way no matter which backend your data happens to be in.

What this library is not

It doesn't change how Python itself works (the GIL, dynamic typing, etc. are unaffected — no pure-Python library can do that), and it doesn't silently rewrite your code. The error handler explains and suggests by default; it only returns a fallback value if you explicitly opt in with reraise=False, and it only auto-fixes anything if you explicitly opt in with auto_fix=True and supply the context a specific rule needs.

Install

pip install -e .            # from this directory, for development
pip install aksci[ai]       # add cloud AI-assist (needs anthropic + an API key)
pip install aksci[dev]      # add pytest for running the test suite

Publishing (PyPI / TestPyPI)

The package already builds cleanly with the standard tooling:

pip install build twine
python -m build              # produces dist/aksci-X.Y.Z.tar.gz and .whl
twine check dist/*           # validates metadata before uploading

# Try it on TestPyPI first:
twine upload --repository testpypi dist/*
pip install --index-url https://test.pypi.org/simple/ aksci

# Once verified, upload for real:
twine upload dist/*

You'll need a PyPI (and/or TestPyPI) account and an API token — generate one under Account Settings → API tokens on each site, then use __token__ as the username and the token as the password when twine upload prompts you (or put them in ~/.pypirc).

GitHub

A CI workflow is already set up at .github/workflows/ci.yml — it runs the test suite on Python 3.10/3.11/3.12 on every push/PR to main, then builds and metadata-checks the distribution. To publish the repo:

git init
git add .
git commit -m "Initial commit"
git branch -M main
git remote add origin https://github.com/<your-username>/aksci.git
git push -u origin main

.gitignore and LICENSE (MIT, matching pyproject.toml) are already in place, so the repo is push-ready as-is.

Quick start

from aksci import ErrorResolver, UnifiedFrame, MicroPipeline, ml, stats
import pandas as pd

# 1. Error diagnosis
resolver = ErrorResolver()

@resolver.safe_run(reraise=False)
def load_price(row):
    return row["price"]

load_price({"prise": 19.99})
# stderr:
# [AKSCI] Caught KeyError: 'price'
#   Category   : missing_column (confidence 84%)
#   Likely fix : This column name does not exist in the DataFrame...

# 2. Unified frame API (works the same on pandas or Polars)
df = pd.DataFrame({"revenue": [100, 200, 300], "cost": [60, 90, 150]})
standardized = UnifiedFrame(df).standardize().to_pandas()

# 3. Simplified ML, including a from-scratch gradient descent for learning
model = ml.linear_regression(X, y)                       # scikit-learn under the hood
gd = ml.gradient_descent(X, y, learning_rate=0.1, epochs=300)  # hand-rolled, explicit

# 4. Memory-bounded streaming for large files
pipeline = MicroPipeline(chunk_size=50_000)
pipeline.add_stage("drop_nulls", lambda chunk: chunk.dropna())
for processed_chunk in pipeline.run_csv("big_file.csv"):
    ...  # only one chunk in memory at a time

Run python examples/demo.py for a full, working walkthrough of every module.

Run python examples/lead_scoring_pipeline.py for a realistic, messier example: a CRM-style CSV export with a renamed column, blank fields, and a zero-division case, scored end-to-end using install_global_hook() plus auto_fix=True on small, single-purpose helper functions.

The local diagnostic model

The offline classifier is trained on 130+ hand-written examples spanning 16 categories: missing_column, key_error_generic, type_mismatch, shape_mismatch, division_by_zero, missing_module, index_out_of_range, attribute_error, null_values, value_error_generic, file_not_found, permission_denied, json_decode_error, connection_error, memory_error, and timeout_error — covering both data-science errors (pandas/numpy/ sklearn) and general automation/API-script errors (file I/O, network, JSON parsing).

It trains once in-process (a fraction of a second) and caches itself to disk; the cache filename is tied to the size of the bundled training set, so upgrading aksci automatically invalidates any stale cached model instead of silently reusing old predictions.

Auto-fix (opt-in)

ErrorResolver(auto_fix=True) turns on a small, whitelisted set of safe recovery rules for any safe_run-decorated function that supplies auto_fix_context. Every rule is conservative — if it can't be confident, it declines rather than guessing — and every applied fix is printed to stderr, never silent.

resolver = ErrorResolver(auto_fix=True)

@resolver.safe_run(reraise=False, auto_fix_context=lambda a, b: {"default": 0.0})
def safe_divide(a, b):
    return a / b

safe_divide(10, 0)
# stderr:
# [AKSCI] Caught ZeroDivisionError: division by zero
#   ...
#   [AKSCI auto-fix] division_by_zero_guard: Division by zero; used fallback value 0.0 instead.
# -> returns 0.0

The four rules, and what each needs in auto_fix_context:

Rule Fires on Needs in context Does
missing_column_fuzzy_match KeyError available_columns, frame Fuzzy-matches the missing key against real column names (same 0.6 cutoff as suggest_column) and returns that column's value
null_values_fillna ValueError mentioning NaN/inf frame, optional fill_value (default 0) Calls .fillna() (or np.nan_to_num) on frame
division_by_zero_guard ZeroDivisionError default Returns default instead of raising
type_coercion TypeError/ValueError raw_value, target_type (int/float) Coerces raw_value to the target type

Important: auto-fix replaces a function's entire return value for that call — it can't resume execution partway through a function that failed on its second or third risky line. Wrap small, single-purpose helpers (one column lookup, one division) rather than a multi-step function; see examples/lead_scoring_pipeline.py for the pattern in a realistic script.

You can also call auto-fix directly without safe_run:

from aksci.error_handler import autofix

try:
    df["revenu"]
except KeyError as exc:
    result = autofix.try_fix(exc, {"available_columns": list(df.columns), "frame": df})
    if result.applied:
        print(result.description)

Enabling cloud AI-assist (optional)

from aksci import ErrorResolver, AIClient

resolver = ErrorResolver(ai_client=AIClient(api_key="sk-ant-..."))
# or set the AKSCI_ANTHROPIC_API_KEY environment variable instead of passing api_key=

Without a key, ErrorResolver still works — it just uses the offline model only.

Project layout

aksci/
├── .github/workflows/ci.yml    # test matrix (3.10-3.12) + build check
├── .gitignore
├── LICENSE                      # MIT
├── pyproject.toml
├── README.md
├── src/aksci/
│   ├── __init__.py             # public API surface
│   ├── ai_core/                 # the "AI" layer
│   │   ├── prompts.py           # hardcoded system prompts for cloud AI-assist
│   │   ├── client.py            # optional cloud AI-assist client (opt-in)
│   │   ├── diagnostics.py       # embedded offline ML classifier
│   │   └── _training_data.py    # bundled training examples (130+, 16 categories)
│   ├── error_handler/
│   │   ├── resolver.py          # ErrorResolver, safe_run decorator, global hook
│   │   └── autofix.py           # whitelisted, opt-in auto-fix rules
│   ├── micro_pipeline/
│   │   └── buffer.py            # BoundedBuffer, Stage, MicroPipeline
│   └── unified_api/
│       └── facade.py            # UnifiedFrame, ml, stats
├── tests/
│   └── test_basic.py            # 29 tests, all passing — covers real behavior,
│                                 # not just imports (e.g. gradient descent
│                                 # converging to known weights, auto-fix rules
│                                 # firing/declining correctly)
└── examples/
    ├── demo.py                  # runnable end-to-end walkthrough of every module
    └── lead_scoring_pipeline.py # realistic messy-CSV automation example

Running the tests

pip install -e .[dev]
pytest tests/ -v

Extending into a larger tool ecosystem

Each subpackage (ai_core, error_handler, micro_pipeline, unified_api) is independent and importable on its own — e.g. a larger coding-assistant environment could import just aksci.error_handler.ErrorResolver and reuse its explain() / diagnose() methods against exceptions it catches itself, without pulling in the rest of the package. Similarly, aksci.error_handler.autofix.try_fix() can be called directly against any caught exception, independent of safe_run.

Metadata

Release files for aksci 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aksci 0.1.1
File Size Uploaded Provenance Actions
aksci-0.1.1.tar.gz 32.9 kB Without provenance Details for aksci-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aksci 0.1.1
File Interpreter ABI Platform Actions
aksci-0.1.1-py3-none-any.whl Python 3 none any Details for aksci-0.1.1-py3-none-any.whl

Total release size: 62.4 kB

Release files / aksci-0.1.1.tar.gz

Download URL aksci-0.1.1.tar.gz
Size 32.9 kB
Tags Source
SHA-256 checksum
How to use checksums
39491bd2d7cc14e25a22e2d462aa8237fc2fe2409d5fa35f6414b1777346d2df
BLAKE2b-256 checksum
How to use checksums
4cd5d000664f37a6db577a52331e72d28210a795a6a03a3ade3ec6cd694e0034
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Release files / aksci-0.1.1-py3-none-any.whl

Download URL aksci-0.1.1-py3-none-any.whl
Size 29.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
67f4a58a3e59fb60c6bd8f0834b84b74a5903d2f0dfd48e8cb22ae9bec9987e6
BLAKE2b-256 checksum
How to use checksums
b763c818c0dbae8f604843056cee6ba32c565323cd311069db68f06151ac1f55
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.5

Security for aksci 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page