Nick Roy is the Director of Quest Systems Engineering, a Professor in MIT’s Department of Aeronautics and Astronautics, and a principal investigator in CSAIL. His research interests in artificial intelligence, machine learning, human-computer interaction, and robotics drew him to the Quest’s Embodied Intelligence Mission, where he thinks about how machines can learn to handle the uncertainty that comes from sensing and acting in a dynamic, unstructured, and populated environment.
Q: The Embodied Intelligence Mission is one of the Quest’s flagship Missions. Why was embodied intelligence chosen as a critical research area for the Quest? What encompasses the Embodied Intelligence Mission and its goals?
NR: I am really happy that Embodied Intelligence is one of the Quest's flagship Missions! My own personal view is that the ability to interact with the environment with purpose (and that purpose is important!) is one of the hallmarks of intelligence, and there are precious few examples of natural intelligence out in the world that don't involve moving around, changing the world, understanding the consequences of actions, finding sources of energy to allow continued existence, and so on. If the core thesis of the Quest is that natural intelligence and artificial intelligence are two sides of the same coin, then studying the questions of how to interact with the physical world has to be part of the enterprise.
Embodied intelligence also requires us to think about how to represent the world in ways that scale — the world is a big place! And it requires us to decide what to pay attention to. The physical world is constantly changing, and so our systems need to know how to adapt, and to act to learn more about the world. We also have to think about risks and hazards in different ways — intelligent agents in a digital world can certainly take actions that can cause harm, or at a minimum burn a lot of power (or money) heedlessly, but there is something very focusing about the risks that come with acting in the real world, full of hazards.
Our goal is to develop the theories, models and algorithms that allow us to match natural intelligence performance, and part of that is defining tasks and benchmarks that allow us to regularly assess the performance of those models against natural intelligence.
Q: What does your research (and your broader lab’s research) look like? How does your lab’s research contribute to this Mission’s goals?
NR: A lot of my lab's research sits at the intersection of perception and decision making. I like to develop models and algorithms that allow a robot not only to build a representation of the world, but one that lets the robot make complicated decisions in that world. And I'm not the only one in the Quest interested in these questions — I have gone back and forth over the years as to whether or not the interesting question is in building the model from perception, or the interesting question is how to act with a model that is almost certainly incomplete and possibly wrong because the sensors were noisy. Right now, we're working on the problem of how to discover complex mechanisms. For instance, there are lots of mechanisms in the world like tools, storage containers, doors and drawers, you name it, where it's not at all obvious how to interact with the tool or the container. We've all been flummoxed at some point in our lives by a doorhandle that did not open the door the way we expected. How should a robot discover that there is a hidden mechanism at work, and can we figure out how it works? We've been doing this research in collaboration with several folks, which is one of the reasons why it's great to be doing this in the Quest — in that it has brought several of us together.
Q: The Quest Engineering Team has been working with your lab; what are the goals for that project?
NR: The Quest Engineering Team has been great at helping us make the experiments more repeatable and the underlying software that we use more robust. One of the challenges with robotics is that getting a robot to do anything requires many pieces of software to come together and work in synchrony. If that software is not well-engineered, or even is hard to start up, it gets very hard to do repeatable experiments. This is important to Quest research because again, part of our Mission is to assess the performance of new models against natural intelligence performance. The Quest Engineering Team in particular has been helping us get various discrete pieces of software in perception modelling to the point where we can run that software seamlessly more or less out of the box, and accelerate the experiments we want to do.