FLAML 2.0.0rc3

A fast library for automated machine learning and tuning

A Fast Library for Automated Machine Learning & Tuning

:fire: FLAML is highlighted in OpenAI's cookbook.

:fire: autogen is released with support for ChatGPT and GPT-4, based on Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference.

:fire: FLAML supports AutoML and Hyperparameter Tuning features in Microsoft Fabric private preview. Sign up for these features at: https://aka.ms/fabric/data-science/sign-up.

What is FLAML

FLAML is a lightweight Python library for efficient automation of machine learning and AI operations, including selection of models, hyperparameters, and other tunable choices of an application (e.g., inference hyperparameters for foundation models, configurations in MLOps/LMOps workflows, pipelines, mathematical/statistical models, algorithms, computing experiments, software configurations).

• For foundation models like the GPT models, it automates the experimentation and optimization of their performance to maximize the effectiveness for applications and minimize the inference cost. FLAML enables users to build and use adaptive AI agents with minimal effort.
• For common machine learning tasks like classification and regression, it quickly finds quality models for user-provided data with low computational resources. It is easy to customize or extend. Users can find their desired customizability from a smooth range: minimal customization (computational resource budget), medium customization (e.g., search space and metric), or full customization (arbitrary training/inference/evaluation code).
• It supports fast and economical automatic tuning, capable of handling complex constraints/guidance/early stopping. FLAML is powered by a cost-effective hyperparameter optimization and model selection method invented by Microsoft Research, and many followup research studies.

FLAML has a .NET implementation in ML.NET, an open-source, cross-platform machine learning framework for .NET. In ML.NET, you can use FLAML via low-code solutions like Model Builder Visual Studio extension and the cross-platform ML.NET CLI. Alternatively, you can use the ML.NET AutoML API for a code-first experience.

Installation

Python

FLAML requires Python version >= 3.7. It can be installed from pip:

pip install flaml

Minimal dependencies are installed without extra options. You can install extra options based on the feature you need. For example, use the following to install the dependencies needed by the autogen package.

pip install "flaml[autogen]"

Find more options in Installation. Each of the notebook examples may require a specific option to be installed.

.NET

Use the following guides to get started with FLAML in .NET:

• Install Model Builder
• Install ML.NET CLI
• Microsoft.AutoML

Quickstart

• (New) The autogen package can help you maximize the utility out of the expensive LLMs such as ChatGPT and GPT-4, including:
  • A drop-in replacement of openai.Completion or openai.ChatCompletion with powerful functionalites like tuning, caching, templating, filtering. For example, you can optimize generations by LLM with your own tuning data, success metrics and budgets.

  from flaml import oai
  
  # perform tuning
  config, analysis = oai.Completion.tune(
      data=tune_data,
      metric="success",
      mode="max",
      eval_func=eval_func,
      inference_budget=0.05,
      optimization_budget=3,
      num_samples=-1,
  )
  
  # perform inference for a test instance
  response = oai.Completion.create(context=test_instance, **config)
  

  • LLM-driven intelligent agents which can perform tasks autonomously or with human feedback, including tasks that require using tools via code.

  assistant = AssistantAgent("assistant")
  user = UserProxyAgent("user", human_input_mode="TERMINATE")
  assistant.receive("Draw a rocket and save to a file named 'rocket.svg'")
  

• With three lines of code, you can start using this economical and fast AutoML engine as a scikit-learn style estimator.

from flaml import AutoML
automl = AutoML()
automl.fit(X_train, y_train, task="classification")

• You can restrict the learners and use FLAML as a fast hyperparameter tuning tool for XGBoost, LightGBM, Random Forest etc. or a customized learner.

automl.fit(X_train, y_train, task="classification", estimator_list=["lgbm"])

• You can also run generic hyperparameter tuning for a custom function.

from flaml import tune
tune.run(evaluation_function, config={…}, low_cost_partial_config={…}, time_budget_s=3600)

• Zero-shot AutoML allows using the existing training API from lightgbm, xgboost etc. while getting the benefit of AutoML in choosing high-performance hyperparameter configurations per task.

from flaml.default import LGBMRegressor

# Use LGBMRegressor in the same way as you use lightgbm.LGBMRegressor.
estimator = LGBMRegressor()
# The hyperparameters are automatically set according to the training data.
estimator.fit(X_train, y_train)

Documentation

You can find a detailed documentation about FLAML here where you can find the API documentation, use cases and examples.

In addition, you can find:

• Research and blogposts around FLAML.

• Discord.

• Contributing guide.

• ML.NET documentation and tutorials for Model Builder, ML.NET CLI, and AutoML API.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

If you are new to GitHub here is a detailed help source on getting involved with development on GitHub.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [e-mail] with any additional questions or comments.