simpleml-auronss 0.1.0

A lightweight, educational scikit-learn-like machine learning library

simpleml

A lightweight, educational implementation of a scikit-learn-like machine learning library in pure Python and NumPy.

Features

• Classification Models: Logistic Regression, Decision Trees, Random Forests, Naive Bayes, Support Vector Classifiers
• Regression Models: Linear Regression, Decision Tree Regression, Random Forest Regression, Support Vector Regression
• Clustering Algorithms: K-Means, DBSCAN
• Preprocessing: StandardScaler, MinMaxScaler, OneHotEncoder
• Model Selection: K-Fold Cross-Validation, Grid Search, Cross-Validation
• Metrics: Accuracy, Precision, Recall, F1-Score, Confusion Matrix, MSE, MAE, R² Score
• Consistent API: Inspired by scikit-learn's fit-predict interface

Installation

git clone https://github.com/sergeauronss01/simpleml.git
cd simpleml
pip install -r requirements.txt

Quick Start

Basic Classification

from simpleml.linear_model import LogisticRegression
from simpleml.preprocessing import StandardScaler
from simpleml.model_selection import train_test_split
from simpleml.metrics import accuracy_score
import numpy as np

# Generate sample data
X = np.random.randn(100, 5)
y = np.random.randint(0, 2, 100)

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Preprocess
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# Train model
clf = LogisticRegression(n_iterations=1000)
clf.fit(X_train, y_train)

# Evaluate
y_pred = clf.predict(X_test)
print(f"Accuracy: {accuracy_score(y_test, y_pred)}")

Decision Tree Classifier

from simpleml.tree import DecisionTreeClassifier

X = np.array([[0, 0], [1, 1], [0, 1], [1, 0]])
y = np.array([0, 1, 1, 0])

clf = DecisionTreeClassifier(max_depth=3)
clf.fit(X, y)
predictions = clf.predict(X)

Random Forest

from simpleml.ensemble import RandomForestClassifier

clf = RandomForestClassifier(n_estimators=10, random_state=42)
clf.fit(X_train, y_train)
predictions = clf.predict(X_test)

K-Means Clustering

from simpleml.cluster import KMeans

kmeans = KMeans(n_clusters=3, random_state=42)
kmeans.fit(X)
labels = kmeans.predict(X_new)

Cross-Validation

from simpleml.model_selection import cross_validate

scores = cross_validate(clf, X, y, cv=5)
print(f"Mean test score: {scores['mean_test_score']}")

Grid Search

from simpleml.model_selection import GridSearchCV

param_grid = {
    'learning_rate': [0.01, 0.1],
    'n_iterations': [100, 500],
}

gs = GridSearchCV(clf, param_grid, cv=5)
gs.fit(X_train, y_train)
print(f"Best parameters: {gs.best_params_}")

Module Structure

simpleml/
├── __init__.py              # Package initialization
├── base.py                  # Base classes (BaseEstimator, ClassifierMixin, etc.)
├── linear_model.py          # Linear regression and classification
├── tree.py                  # Decision tree models
├── ensemble.py              # Ensemble methods
├── cluster.py               # Clustering algorithms
├── naive_bayes.py           # Naive Bayes classifiers
├── svm.py                   # Support Vector Machines
├── preprocessing.py         # Data preprocessing
├── model_selection.py       # Model selection and evaluation
├── metrics.py               # Evaluation metrics
└── utils.py                 # Utility functions

Available Models

Supervised Learning

Classification

• LogisticRegression: Binary logistic regression with gradient descent
• DecisionTreeClassifier: Decision tree with Gini/Entropy splits
• RandomForestClassifier: Ensemble of decision trees
• GaussianNaiveBayes: Gaussian Naive Bayes classifier
• MultinomialNaiveBayes: Multinomial Naive Bayes for discrete features
• LinearSVC: Linear Support Vector Classifier

Regression

• LinearRegression: Ordinary least squares regression
• DecisionTreeRegressor: Decision tree regression
• RandomForestRegressor: Ensemble regression
• SVR: Support Vector Regression

Unsupervised Learning

Clustering

• KMeans: K-Means clustering algorithm
• DBSCAN: Density-based clustering

Preprocessing

• StandardScaler: Standardization (zero mean, unit variance)
• MinMaxScaler: Min-Max scaling to a fixed range
• OneHotEncoder: One-hot encoding for categorical features

Model Selection

• KFold: K-Fold cross-validator
• cross_validate(): Cross-validation scoring
• GridSearchCV: Exhaustive grid search

Metrics

All metrics are available in simpleml.metrics:

• accuracy_score: Classification accuracy
• precision_score: Precision for binary classification
• recall_score: Recall for binary classification
• f1_score: F1-Score for binary classification
• confusion_matrix: Confusion matrix
• mean_squared_error: Mean squared error
• mean_absolute_error: Mean absolute error
• r2_score: R² coefficient of determination

Testing

Run the test suite:

pytest tests/ -v

Run tests with coverage:

pytest tests/ --cov=simpleml

Documentation

Each module is thoroughly documented with docstrings following NumPy style. Use help() in Python:

from simpleml.linear_model import LogisticRegression
help(LogisticRegression)

Design Philosophy

This library is designed to:

1. Educate: Clear, understandable implementations without external ML dependencies
2. Mirror scikit-learn: Familiar API for users of scikit-learn
3. Be extensible: Easy to add new models and algorithms
4. Be tested: Comprehensive test coverage

Limitations

• No support for neural networks or deep learning
• No GPU acceleration
• Limited to basic algorithms (no advanced techniques like stacking, boosting beyond basic implementation)
• Not optimized for large-scale datasets

Performance

This library is for educational purposes. For production use and performance-critical applications, use scikit-learn or similar production libraries.

Contributing

To contribute:

1. Fork the repository
2. Create a feature branch
3. Write tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

License

MIT License

Author

Serge Auronss Gbaguidi

References

• Scikit-learn documentation: https://scikit-learn.org
• NumPy documentation: https://numpy.org
• Machine Learning fundamentals

---

Note: This is an educational project created to understand machine learning algorithms and best practices for library design. For production machine learning work, please use scikit-learn.