DataForge

DataForge is a local-first Python toolkit for inspecting datasets, learning core machine learning concepts, and generating editable Jupyter notebooks. Its Python library is the product; Flask is an optional local dashboard that calls the same library services.

V1 features

• Load and inspect CSV and .xlsx datasets without changing the source file.
• Review schema, missing values, duplicates, descriptive statistics, constant columns, category frequencies, and numeric correlations.
• In the dashboard, preview and apply missing-value imputation or removal, duplicate removal, type conversion, categorical encoding, and IQR outlier detection. Transformations affect the session's in-memory dataset; the uploaded source file is not overwritten.
• Create selectable histograms, boxplots, scatterplots, correlation heatmaps, and category bar/frequency charts.
• Run Linear Regression, Logistic Regression, K-means, Decision Tree, and KNN using shared train/test and preprocessing pipelines.
• Generate structured, editable .ipynb notebooks from Python or the dashboard, including selected cleaning steps, EDA, features, targets, model parameters, and optionally the current Playground experiment.
• Explore actual K-means, regression, and classification experiments in the Playground, using reproducible built-in samples or a session's uploaded dataset. K-means includes cluster assignments, centroids, sizes, an elbow curve, and a silhouette score when valid. Linear Regression includes a fitted line, held-out actual-versus-predicted points, and evaluation metrics.
• Launch a local dashboard on an explicitly selected port.

The dashboard is local and has a 25 MB upload limit. Upload CSV or .xlsx files; use the Remove uploaded data button to delete the current session's uploaded file. A new upload replaces and removes that session's previous file. If the browser session is lost before removal, stop DataForge and delete files from .dataforge/uploads manually. Downloaded notebook copies are temporary and are removed after transfer.

Cleaning can be staged, previewed, applied, and reset to the uploaded original. Outlier detection only flags potential anomalies; it never removes them. Ordinal encoding requires the user to explicitly provide the real category order. Supervised-model preprocessing (including one-hot encoding and scaling) is fitted inside the training pipeline after the train/test split. The Notebook Builder tests the selected workflow before enabling download, and the generated notebook remains editable and must be run by the user in Jupyter.

The K-means Playground offers an elbow curve so learners can compare K values, plus silhouette where mathematically valid. Step-by-step centroid iteration animation is not included in V1. Playground chart features are limited to one numeric predictor for Linear Regression and two numeric features for the other algorithms; the visualized features are the features used for the Playground model.

Install

Python 3.10 or newer is required. Install the core Python library and command-line tools with:

python -m pip install dataforge-sba

To use the optional local dashboard with Excel workbook support and notebook charts, install:

python -m pip install "dataforge-sba[dashboard,visualization]"

The distribution is named dataforge-sba; import it in Python as dataforge_sba. Both dataforge-sba and dataforge are installed as command names. Python 3.10–3.13 are declared supported. The core package requires pandas, NumPy, scikit-learn, and nbformat. Flask/openpyxl are in the dashboard extra; Matplotlib is in visualization.

For library and dashboard development:

python -m pip install -e ".[dev,dashboard,visualization]"
python -m pytest

The library itself does not start Flask or open a browser during import. Jupyter can be installed separately with python -m pip install jupyter to execute downloaded notebooks.

Launch the dashboard

dataforge-sba dashboard
dataforge-sba dashboard --port 8080
dataforge dashboard --port 8080
dataforge-sba --help
dataforge-sba --version
dataforge-sba --creator

--creator prints Selim Ben Ali | https://github.com/SelimBenAli/dataforge. The default bind address is 127.0.0.1; debug mode is off. If a port is already in use, DataForge reports it and exits; choose another with --port. Ports must be from 1 through 65535. Stop the dashboard with Ctrl+C. The dashboard requires the dashboard extra; if Flask is missing, the command prints the install instruction. The upload limit is 25 MB. Uploaded data stays in .dataforge/uploads under the directory where you launch the command, and each browser session refers to its own upload.

Using the dashboard

1. Upload a CSV or .xlsx file and review the column types, missing values, duplicates, and summaries.
2. Use Clean and prepare to stage changes, preview them, then apply or reset them. The uploaded file itself is not changed.
3. Use Explore with a chart to inspect selected columns.
4. In Notebook Builder, choose chart types, algorithms, targets, optional features, and model settings. Click Test selections to check the choices before downloading.
5. In the ML Playground, choose sample data or the uploaded dataset, run an experiment, and optionally include its configuration and code in the notebook.
6. In Jupyter, open the downloaded .ipynb, update DATA_PATH if necessary, and run cells from top to bottom. The notebook is an editable starting point; it runs on your computer.

Python API

from dataforge_sba import Dataset
from dataforge_sba.cleaning import impute_missing, missing_summary

dataset = Dataset("data.csv")
print(dataset.shape)
print(dataset.schema())
report = dataset.explore()

cleaned = impute_missing(dataset.frame, {"age": "median", "city": "mode"})
print(missing_summary(cleaned))

To fit a model, pass a DataFrame with the target column named explicitly:

from dataforge_sba.algorithms import run_algorithm

result = run_algorithm("linear_regression", dataset.frame, target="sales")
print(result["metrics"])

The returned model is a fitted scikit-learn pipeline. Preprocessing is fitted on training data only. Classification metrics are weighted averages; inspect the task and dataset before interpreting scores.

Generate a notebook

from dataforge_sba import Dataset
from dataforge_sba.notebooks import NotebookGenerator

dataset = Dataset("data.csv")
generator = NotebookGenerator(dataset)
generator.add_cleaning(drop_duplicates=True)
generator.add_visualizations()
generator.add_algorithm("kmeans", n_clusters=4)
generator.save("analysis.ipynb")

The generated notebook uses a portable dataset filename and a clear DATA_PATH variable. Move the original dataset beside the notebook or update DATA_PATH in its first code cell. Supervised experiments include a target and optional selected features; K-means does not ask for a target. The dashboard also offers algorithm-specific parameters, split fraction, seed, scaler choice, selected EDA charts, cleaning steps, and optional Playground export. Dashboard downloads are preflighted against the uploaded data and structurally validated; the notebook is not fully executed until you run its cells in Jupyter.

K-means Playground from Python

from dataforge_sba.playground import run_kmeans

experiment = run_kmeans(shape="moons", n_clusters=2, random_state=7)
print(experiment["silhouette"], experiment["cluster_sizes"])

Cluster labels are arbitrary identifiers, not rankings. Scaling affects distances, and silhouette scores are unavailable when the mathematical requirements are not met.

Algorithm catalog

Algorithm ID	Task	Current implementation
linear_regression	Numeric prediction	Train/test split; MAE, MSE, RMSE, R²
logistic_regression	Classification	Binary/multiclass; accuracy, weighted precision/recall/F1
kmeans	Clustering	Numeric features, optional scaling, cluster sizes, silhouette score
decision_tree	Classification	Configurable depth and reproducible split
knn	Classification	Configurable neighbor count and scaled features

Use dataforge_sba.algorithms.list_algorithms() to retrieve registry metadata. Extend the catalog through the central algorithm registry; UI labels are populated from its metadata endpoint.

Privacy

DataForge runs locally and sends no dataset data to external services. It does not execute uploaded Python code or overwrite the source dataset. The browser dashboard uses first-party local requests only. Keep .dataforge/ private if your data is sensitive, and remove temporary uploads when finished.

Roadmap

Known V1 limitation: K-means provides an elbow curve and silhouette score but does not animate centroid updates step by step. Playground visualizations use one numeric feature for Linear Regression and two for the other algorithms. The dashboard does not persist experiment history. Further algorithms, PCA, cross-validation, and experiment persistence are possible future work.

Troubleshooting

• dataforge-sba is not recognized: close and reopen the terminal after installation, check that Python's Scripts directory is on PATH, or run python -m pip show dataforge-sba to confirm which Python environment received the package.
• Dashboard reports that Flask is missing: install python -m pip install "dataforge-sba[dashboard]" in the same environment used to launch the command.
• Excel workbook cannot be read: install the dashboard extra (which includes openpyxl) or python -m pip install openpyxl.
• Windows reports a NumPy DLL or application-control error: use a fresh virtual environment and reinstall the package dependencies. If Windows security blocks a dependency DLL, resolve that Windows policy block before retrying; DataForge cannot bypass it.
• Notebook cannot find the dataset: put the original CSV/XLSX beside the notebook or edit DATA_PATH in the first code cell to point to the file.
• Port is already in use: select another port, such as dataforge-sba dashboard --port 8080.

Audit and release checks

Build locally with python -m build. Run tests with python -m pytest. Release verification must also install the wheel in a fresh environment and check both CLI names, dashboard assets, and a generated notebook. DataForge is not published to PyPI by this repository workflow; package publication requires a separate release decision.

Contributing and license

See CONTRIBUTING.md. DataForge is released under the MIT License; see LICENSE. "# dataforge"