Parameter-free Machine Learning through Coin Betting
19 dicembre 2019 alle 14:30
Sala Riunioni 8 Piano, via Celoria 18
Speaker: Francesco Orabona, Boston University, USA
Persona di riferimento: Nicolò Cesa-Bianchi
Machine Learning (ML) has been described as the fuel of the next industrial revolution. Yet, despite their name, the majority of the ML algorithms still heavily rely on having humans in the loop in order to set their "parameters". For example, when using a regularizer, the regularization weight is critical to obtain theoretical and practical optimal performance. Moreover, the minimization itself, usually done through stochastic gradient descent procedures, requires to set "learning rates" in order to achieve good performance. Are these parameters strictly necessary? Is it possible to have parameter-free ML algorithms? In this talk, I will show that both the problems of ML and stochastic optimization with quasi-convex losses can be reduced to a game of betting on a non-stochastic coin. Betting on a non-stochastic coin is a well-known problem that can be solved using tools from information theory. Moreover, the optimal coin betting algorithm is parameter-free, giving rise to parameter-free ML and stochastic optimization algorithms. This approach is very general, i.e. it works for any norm, and it gives optimal results in a number of settings, i.e. regression in reproducing kernel Hilbert spaces, without any parameter to tune. Beside the theoretical results, I will show that this approach can also be used in modern deep learning architectures, presenting state-of-the-art performance using the first stochastic gradient backprop procedure without a learning rate..
Francesco Orabona is an Assistant Professor at Boston University. His background covers both theoretical and practical aspects of machine learning and optimization. His current research interests lie in online learning, and more specifically he works on designing and analyzing adaptive and parameter-free learning algorithms. He received the PhD degree in Electrical Engineering at the University of Genoa, in 2007. He is (co)author of more than 70 peer reviewed papers.