minisklearn 0.1.3

Mini Scikit Learn

Mini Sklearn Implementation Project

Overview

This project involves the implementation of a mini version of the sklearn library, designed to provide essential machine learning tools and algorithms from scratch. It aims to replicate key functionalities of sklearn, offering a customizable and scalable toolkit for data analysis and modeling.

Motivation

• Learning Objective: To gain a deeper understanding of the inner workings of machine learning algorithms by implementing them from scratch.
• Skill Enhancement: To improve programming and problem-solving skills.
• Custom Solution: To provide a lightweight and customizable alternative to the comprehensive sklearn library.

Goals

1. Core Implementations:
   • To implement core machine learning algorithms and utilities from scratch.
2. Efficiency and Reliability:
   • To ensure the implementations are efficient and reliable.
3. Modular and Extensible Codebase:
   • To create a modular and extensible codebase for future enhancements.

Modules Implemented

Cluster

• KMeans: Clustering algorithm that partitions data into k clusters.

Covariance

• EmpiricalCovariance: Computes the covariance matrix of a dataset.

Decomposition

• PCA (Principal Component Analysis): Dimensionality reduction technique that identifies principal components.

Feature Extraction

• SimpleCountVectorizer: Converts text documents into a matrix of token counts.

Feature Selection

• FeatureSelector: Utility for selecting a subset of features based on certain criteria.

Impute

• MissingIndicator: Detects missing values and encodes their presence as binary indicators.
• SimpleImputer: Handles missing data by imputing values using specified statistics.

Linear Model

• LinearRegression: Predicts target variables as a linear combination of input features.

Metrics

• Accuracy Metrics: accuracy_score, average_precision_score, confusion_matrix, f1_score, precision_score, recall_score, roc_auc_score, roc_curve
• Error Metrics: mean_absolute_error, mean_squared_error, median_absolute_error, r2_score
• Similarity Metrics: cosine_similarity, cosine_distances, euclidean_distances
• Kernel Functions: linear_kernel, rbf_kernel

Neighbors

• KNeighborsClassifier: Classification based on the majority class of nearest neighbors.
• KNeighborsRegressor: Regression based on the average value of nearest neighbors.
• NearestCentroid: Classifies data points based on the centroid of their nearest class neighbors.

Model Selection

• KFold: Cross-validation technique that splits data into k folds.
• GridSearchCV: Hyperparameter tuning method that searches through a specified parameter grid.

Naive Bayes

• GaussianNB: Naive Bayes classifier assuming features follow a Gaussian distribution.

Preprocessing

• Scalers: MaxAbsScaler, MinMaxScaler, Normalizer, RobustScaler, StandardScaler
• Encoders: Binarizer, KBinsDiscretizer, LabelBinarizer, LabelEncoder

SVM

• LinearSVC: Linear support vector classifier for binary classification tasks.

Tree

• DecisionTreeClassifier: Classification algorithm that builds a decision tree.
• DecisionTreeRegressor: Regression algorithm that builds a decision tree to predict continuous target variables.

Technical Stack

• Programming Language: Python
• Libraries and Tools: NumPy, matplotlib, scikit-learn (as reference)
• Development Environment: Jupyter Notebooks

Architecture

• Modular Design: Follows a structure similar to sklearn, with modules for different machine learning tasks.
• Components: Each module contains classes and functions for specific tasks.
• Interconnection: Modules are interconnected to enable seamless integration and workflow.

Implementation Details

• Key Algorithms: KMeans, PCA, Linear Regression, and various metrics.
• Code Structure: Organized into modules with hierarchical class structures, promoting code reusability and maintainability.
• Documentation: Comprehensive documentation provided for classes and functions.

Challenges and Solutions

• Challenge: Ensuring efficiency and scalability.
• Solution: Employing vectorization and algorithmic optimizations.

Testing

• Test Cases: Designed comprehensive test cases covering various scenarios.
• Testing Framework: Used pytest for automated testing to ensure correctness and consistency across implementations.
• Results: High correlation and accuracy with sklearn algorithms.

Packaging

• Distribution: The project is packaged and distributed using pip. To install, use the following command: pip install minisklearn

Future Work

• Enhancements: Addition of new algorithms and optimization techniques.
• Scalability: Further scaling of the project to handle larger datasets and more complex tasks.

Conclusion

• Summary: Successfully implemented core machine learning algorithms and utilities, creating a modular and extensible codebase.
• Learnings: Gained valuable insights into algorithm design and modular programming.
• Acknowledgements: Special thanks to collaborators, mentors, and resources that supported this project.