Graph Representations for Data-Efficient Reinforcement Learning
Il 30 ottobre 2019 alle 16:30
Sala Consiglio 8 Piano, via Celoria 18
Speaker: Laura Toni, University College London
We are surrounded by large-scale interconnected systems, from the Internet to the power grid and social networks. While essential, the management of such networked systems is exceedingly hard mainly because of their intrinsic and constantly growing complexity. To overcome this challenge, data-efficient online decision strategies under uncertainty for high-dimensional and dynamic networks need to be designed. To learn efficiently in complex domains data, one must ultimately be able to discover and exploit the structure of the high-dimensional ambient space.
In this talk, we first comment on the importance of graph representation learning techniques in decision-making. Then, we focus on reinforcement learning (RL) problems in the challenging settings of high dimensional state or action spaces. We provide an overview of state-of-the-art value function approximation techniques and we highlight the importance of features learning for an improved RL strategy. We empirically show that in low-dimensional feat space the graph-embedding nod2vec, which learns the graph representation, constantly outperforms the commonly used smooth proto-value functions, that constructs the graph representation. We conclude showing the analogy between nod2vec and the successor features representation under random policy exploration, and its gain in transfer learning.
Laura Toni is an assistant professor in the Department of Electronic and Electrical Engineering at University College London (UCL). She received her PhD degree in electrical engineering in 2009 from the University of Bologna, Italy. She was a Post-Doc at the University of California at San Diego (UCSD) from 2011-2012 and at the Swiss Federal Institute of Technology (EPFL), Switzerland from 2012-2016.
Her major contributions are in the area of immersive communications, decision-making strategies under uncertainty, and large-scale signal processing for machine learning. Her current research focuses on online adaptive strategies for dynamic networks, with a deep focus on recommendations and reinforcement learning for optimal managing of large-scale systems. Her past research endeavours with Intel, Cisco, and Verizon have led to several patents and patents applications on streaming for novel media applications. She is the author of more than 20 journal papers, 40 conference papers, and 2 patents. She recently received the UCL Future Leadership Award, the Adobe System academic donation, and one Cisco Academic Award. She is General Co-Chair of ACM MMSys 2020 Conference, and she recently organized the UCL workshop on the power on graph in machine learning and sequential decision-making. She is an Associated Editor of IEEE Transaction on Image Processing and EURASIP Journal on Advances in Signal Processing.