mlrose-ky 1.0.2

MLROSe-ky: Machine Learning, Randomized Optimization and Search

mlrose-ky: Machine Learning, Randomized Optimization, and SEarch

mlrose-ky is a Python package for applying some of the most common randomized optimization and search algorithms to a range of different optimization problems, over both discrete- and continuous-valued parameter spaces.

Project Background

mlrose-ky is a fork of the mlrose-hiive repository, which itself was a fork of the original mlrose repository. The original mlrose package was developed to support students of Georgia Tech's OMSCS/OMSA offering of CS 7641: Machine Learning.

This repository includes implementations of all randomized optimization algorithms taught in the course, as well as functionality to apply these algorithms to integer-string optimization problems, such as N-Queens and the Knapsack problem; continuous-valued optimization problems, such as the neural network weight problem; and tour optimization problems, such as the Travelling Salesperson problem. It also has the flexibility to solve user-defined optimization problems.

Main Features

Randomized Optimization Algorithms

• Implementations of: hill climbing, randomized hill climbing, simulated annealing, genetic algorithm, and (discrete) MIMIC;
• Solve both maximization and minimization problems;
• Define the algorithm's initial state or start from a random state;
• Define your own simulated annealing decay schedule or use one of three pre-defined, customizable decay schedules: geometric decay, arithmetic decay, or exponential decay.

Problem Types

• Solve discrete-value (bit-string and integer-string), continuous-value, and tour optimization (travelling salesperson) problems;
• Define your own fitness function for optimization or use a pre-defined function.
• Pre-defined fitness functions exist for solving the: One Max, Flip Flop, Four Peaks, Six Peaks, Continuous Peaks, Knapsack, Travelling Salesperson, N-Queens, and Max-K Color optimization problems.

Machine Learning Weight Optimization

• Optimize the weights of neural networks, linear regression models, and logistic regression models using randomized hill climbing, simulated annealing, the genetic algorithm, or gradient descent;
• Supports classification and regression neural networks.

Project Improvements and Updates

The mlrose-ky project is undergoing significant improvements to enhance code quality, documentation, and testing. Below is a list of tasks that have been completed or are in progress:

1. Fix Python Warnings and Errors: All Python warnings and errors have been addressed, except for a few unavoidable ones like "duplicate code." ✅

2. Add Python 3.10 Type Hints: Type hints are being added to all function and method definitions, as well as method properties (e.g., self.foo: str = 'bar'), to improve code clarity and maintainability.

3. Enhance Documentation: NumPy-style docstrings are being added to all functions and methods, with at least a one-line docstring at the top of every file summarizing its contents. This will make the codebase more understandable and easier to use for others.

4. Increase Test Coverage: Tests are being added using Pytest, with a goal of achieving 100% code coverage to ensure the robustness of the codebase.

5. Resolve TODO/FIXME Comments: A thorough search is being conducted for any TODO, FIXME, or similar comments, and their respective issues are being resolved.

6. Optimize Code: Vanilla Python loops are being optimized where possible by vectorizing them with NumPy to enhance performance.

7. Improve Code Quality: Any other sub-optimal code, bugs, or code quality issues are being addressed to ensure a high standard of coding practices.

8. Clean Up Codebase: All commented-out code is being removed to keep the codebase clean and maintainable.

Installation

mlrose-ky was written in Python 3 and requires NumPy, SciPy, and Scikit-Learn (sklearn).

The latest version can be installed using pip:

pip install mlrose-ky

Once it is installed, simply import it like so:

import src.mlrose_ky

Documentation

The official mlrose-ky documentation can be found here.

A Jupyter notebook containing the examples used in the documentation is also available here.

Licensing, Authors, Acknowledgements

mlrose-ky was forked from the mlrose-hiive repository, which was a fork of the original mlrose repository.

The original mlrose was written by Genevieve Hayes and is distributed under the 3-Clause BSD license.

You can cite mlrose-ky in research publications and reports as follows:

• Nakamura, K. (2024). mlrose-ky: Machine Learning, Randomized Optimization, and SEarch package for Python. https://github.com/your-repo-url. Accessed: day month year.

Please also keep the original authors' citations:

• Rollings, A. (2020). mlrose: Machine Learning, Randomized Optimization and SEarch package for Python, hiive extended remix. https://github.com/hiive/mlrose. Accessed: day month year.
• Hayes, G. (2019). mlrose: Machine Learning, Randomized Optimization and SEarch package for Python. https://github.com/gkhayes/mlrose. Accessed: day month year.

Thanks to David S. Park for the MIMIC enhancements (from https://github.com/parkds/mlrose).

BibTeX entry:

@misc{Nakamura24,
 author = {Nakamura, K.},
 title  = {{mlrose-ky: Machine Learning, Randomized Optimization and SEarch package for Python}},
 year   = 2024,
 howpublished = {\url{https://github.com/your-repo-url}},
 note   = {Accessed: day month year}
}

@misc{Rollings20,
 author = {Rollings, A.},
 title 	= {{mlrose: Machine Learning, Randomized Optimization and SEarch package for Python, hiive extended remix}},
 year 	= 2020,
 howpublished = {\url{https://github.com/hiive/mlrose}},
 note 	= {Accessed: day month year}
}

@misc{Hayes19,
 author = {Hayes, G.},
 title 	= {{mlrose: Machine Learning, Randomized Optimization and SEarch package for Python}},
 year 	= 2019,
 howpublished = {\url{https://github.com/gkhayes/mlrose}},
 note 	= {Accessed: day month year}
}

Collaborators

Nikhil Kapila

Kyle Nakamura

Contributors

hiive	Dr Genevieve Hayes	Kyle Nakamura	Chris Bilger	Nikhil Kapila	Agrover112
Dominic Frecentese	harrisonfloam	Alex Wendland	CAPN	Kevin Boyer	Jason Seeley
Muhammad Sareini	nibelungvalesti	W. Tad Morgan	Michael Schock	John Mansfield	David Strube
Austin Bowen	Ben Spivey	David	Jacob Williams	Keith Beattie	cbhyphen
Daniel Scott	Kira Yang