Here we will give an overview of commonly used and well-known Hyperparameter Optimization (HPO) tools (where only a few of them were developed by us). It is by no means complete. If you miss a tool on the list, please let us know.
Bayesian Optimization
Bayesian Optimization (BO) is considered to be a state-of-the-art approach for expensive black-box functions and thus has been widely implemented in different HPO tools.
Spearmint was one of the first successful open source Bayesian Optimization tools for HPO.
SMAC3 is well known for optimizing complex and structured spaces (e.g., for optimizing entire ML pipelines) based on RF-surrogate models.
Hyperopt is a distributed HPO tool implemented with a Tree of Parzen Estimators (TPE).
Scikit-optimize is a BO tool that is built on the top of scikit-learn (sklearn).
Bayesian Optimization is a (constrained) global optimization tool.
HyperMapper is a BO tool that supports working with unknown constraints and multi-objective optimization.
OpenBox is a general framework for black-box optimization, incl. HPO, and supports multi-objective optimization, multi-fidelity, early-stopping, transfer learning and parallel BO.
Dragonfly is an open source python library for scalable BO with multi-fidelity and multi-objective optimization.
GPflowOpt is a python tool for BO using GPflow and Tensorflow.
BoTorch is a tool doing BO with GPyTorch and PyTorch.
TurBO is a scalable BO tool for large scale paralleled HPO problems with many function evaluations.
Evolutionary Algorithms
Evolutionary algorithms (EA) are another popular black-box optimization framework, as EA can easily be parallelized and the time in each iteration does not grow as number of evaluations grow.
DEHB combines multi-fidelity optimization, such as in Hyperband, with differential evaluations.
DEAP is a novel evolutionary computation framework with lots of popular EA implementations
Nevergrad is a gradient-free optimization platform. Nevergrad implements different EA that meets the requirements of different problems.
Expert-prior supported algorithms
PriorBand replaces random sampling in Hyperband with a collection of sampling strategies, including sampling from user priors.
General HPO Tools
Here we will introduce several tools that implement both BO and EA approaches
Optuna is an automatic HPO framework that allows to dynamically construct the search space.
Oríon is an asynchronous framework for black-box function optimization.
Ray Tune is a scalable framework HPO framework that allows to employ several aforementioned framework as optimizers.