Hyperparameter-tuning Cookbook
A guide for scikit-learn, PyTorch, river, and spotPython
Preface
The goal of hyperparameter tuning (or hyperparameter optimization) is to optimize the hyperparameters to improve the performance of the machine or deep learning model.
spotPython (“Sequential Parameter Optimization Toolbox in Python”) is the Python version of the well-known hyperparameter tuner SPOT, which has been developed in the R programming environment for statistical analysis for over a decade. The related open-access book is available here: Hyperparameter Tuning for Machine and Deep Learning with R—A Practical Guide.
scikit-learn is a Python module for machine learning built on top of SciPy and is distributed under the 3-Clause BSD license. The project was started in 2007 by David Cournapeau as a Google Summer of Code project, and since then many volunteers have contributed.
PyTorch is an optimized tensor library for deep learning using GPUs and CPUs.
River is a Python library for online machine learning. It is designed to be used in real-world environments, where not all data is available at once, but streaming in.
Citation
If this document has been useful to you and you wish to cite it in a scientific publication, please refer to the following paper:
@ARTICLE{bart23earxiv,
author = {{Bartz-Beielstein}, Thomas},
title = "{PyTorch Hyperparameter Tuning -- A Tutorial for spotPython}",
journal = {arXiv e-prints},
keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Mathematics - Numerical Analysis, 68T07, A.1, B.8.0, G.1.6, G.4, I.2.8},
year = 2023,
month = may,
eid = {arXiv:2305.11930},
pages = {arXiv:2305.11930},
doi = {10.48550/arXiv.2305.11930},
archivePrefix = {arXiv},
eprint = {2305.11930},
primaryClass = {cs.LG},
adsurl = {https://ui.adsabs.harvard.edu/abs/2023arXiv230511930B},
adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}