Explanation

What does it mean to explain the behavior of autonomous agents in time-varying and relational environments?

Background

I had been working with longtime collaborators John Foley, Kaleigh Clary, and David Jensen on developing a new testing framework for reinforcement learning back before the pandemic. I designed a prototype version that my collaborators and I used to investigate the behavior of generalized agents in our work, Measuring and Characterizing Generalization in Deep Reinforcement Learning. I presented Toybox at the IBM AI Systems Day and as a poster at the 2018 NeurIPS Systems for ML Workshop.

Current status

This work is currently on pause due to a lack of resources (personnel, funding, time, compute). Explanation is a notoriously challenging research area and will likely remain relevant in the years to come.

Possible projects

I periodically revisit this work and remain broadly interested in building systems that seek to explain agent behavior in time-varying relational environments and would welcome efforts from students who might be interested in causal reasoning, experimentation, or explanation for autonomtous agents. As this work is not currently core to my research agenda, I would only consider working with exceptionally motivated collaborators on this project. While a background in machine learning and reinforcement learning will undoubtedly be helpful, solid software engineering skills are far more important for this work. If you'd like to discuss project possibilities, reach out.