Talk Professeur Eduardo Morales

Abstract:

Reinforcement Learning (RL) has been successfully applied to domains like games and robotics, however, it suffers from long training times. Incorporating models into RL (model-based ML) offers an alternative to produce faster convergence times.
Causal models (CM) have recently gain popularity due to their reasoning capabilities, however, learning causal models requires interventional data. RL naturally generates interventional data in its action variables.
In this talk, I will describe a framework to concurrently learn policies from RL and causal models, and how both learning processes can benefit from each other. In particular, on one hand, we propose to change the exploration strategy used in RL to favor the learning of CMs. On the other hand, CMs are used to guide/constrain the action selection strategy of RL to reduce its convergence times. Once a CM is learned for a particular domain, it can be easily transferred to another similar domain. We will present experimental results of this combination of causal and reinforcement learning, show its benefits, and how CMs can be transferred to another domain.

Short Bio:

Eduardo Morales received his B.Sc. degree in Physics Engineering from Universidad Autonoma Metropolitana, in Mexico City, his M.Sc. degree in Information Technology: Knowledge-based Systems from the University of Edinburgh, and his PhD degree in Computer Science from the Turing Institute - University of Strathclyde, in Scotland.
He has been responsible of more than 30 research projects sponsored by different funding agencies and has more than 200 articles in journals, book's chapters, and conference proceedings. He is a senior research scientist at Instituto Nacional de Astrofísica, Óptica y Electrónica (INAOE) in Puebla, Mexico where he conducts research in Machine Learning and Robotics.