Past Events

The Niv lab focuses on the neural and computational processes underlying reinforcement learning and decision-making, studying the ongoing day-to-day processes by which animals and humans learn from trial and error. Of particular interest is how attention and memory processes interact with reinforcement learning.

Humans spend a lifetime learning, storing and refining a repertoire of motor memories appropriate for the multitude of tasks we perform. However, it is unknown what principle underlies the way our continuous stream of sensorimotor experience is segmented into separate memories and how we adapt and use this growing repertoire. I will review our recent work on how humans learn to make skilled movements focussing on how statistical learning can lead to multimodal object representations, how we represent the dynamics of objects, the role of context in the expression, updating and creation of motor memories and how families of objects are learned. 

For AI systems to be safe and effective, they need to be aligned with the goals and values of users, designers, and society. In this talk, I will discuss the challenges of AI alignment and go over research directions to develop safe AI systems. I'll begin with theoretical results that motivate the alignment problem broadly. In particular, I will show how optimizing incomplete goal specifications reliably causes systems to select unhelpful or harmful actions. Next, I will discuss mitigation measures that counteract this failure mode. I will focus on approaches for incorporating human feedback into objectives, interpreting and understanding learned policies, and maintaining uncertainty about intended goals.

Much existing work in reinforcement learning involves environments that  are either intentionally neutral, lacking a role for cooperation and  competition, or intentionally simple, when agents need imagine nothing  more than that they are playing versions of themselves or are happily  cooperative. Richer game theoretic notions become important as these  constraints are relaxed. For humans, this encompasses issues that  concern utility, such as envy and guilt, and that concern inference,  such as recursive modeling of other players, I will discuss some our  work in this direction using the framework of interactive partially  observable Markov decision-processes, illustrating deception,  scepticism, threats and irritation. This is joint work with Nitay Alon,  Andreas Hula, Read Montague, Jeff Rosenschein and Lion Schulz.

Recordings of neurons in cortical structures in behaving rodents show responses to dimensions of space and time relevant to encoding and retrieval of spatiotemporal trajectories of behavior in episodic memory. This includes the coding of spatial location by grid cells in entorhinal cortex and place cells in hippocampus, some of which also fire as time cells when a rodent runs on a treadmill (Kraus et al., 2013; 2015; Mau et al., 2018). Trajectory encoding also includes coding of the direction and speed of movement. Speed is coded by both firing rate and frequency of neuronal rhythmicity (Hinman et al., 2016, Dannenberg et al., 2020), and inactivation of input from the medial septum impairs the spatial selectivity of grid cells suggesting rhythmic coding of running speed is important for spatial coding by grid cells (Brandon et al., 2011; Robinson et al., 2023).

The emerging field of NeuroAI has leveraged techniques from artificial intelligence to model brain data. In this talk, Yamins will show that the connection between neuroscience and AI can be fruitful in both directions. Towards "AI driving neuroscience", he will discuss a new candidate universal principal for functional organization in the brain, based on recent advances in self-supervised learning, that explains both fine details as well as large-scale organizational structure in the vision system, and perhaps beyond.  In the direction of "neuroscience guiding AI", Yamins will present a novel cognitively-grounded computational theory of perception that generates robust new learning algorithms for real-world scene understanding.  Taken together, these ideas illustrate how neural networks optimized to solve cognitively-informed tasks provide a unified framework for both understanding the brain and improving AI.

Generally, inference problems in machine or biological vision rely on knowledge of prior probabilities. Recently, machine learning has resulted in dramatic improvements by using artificial neural networks. These prior probabilities are implicit and intertwined with the tasks for which they are optimized.

Quality Diversity (QD) algorithms are those that seek to produce a diverse set of high-performing solutions to problems. Clune will describe them and a number of their positive attributes. He will summarize how they enable robots, after being damaged, to adapt in 1-2 minutes in order to continue performing their mission. Next, he will describe our QD-based Go-Explore algorithm, which dramatically improves the ability of deep reinforcement learning algorithms to solve previously unsolvable problems wherein reward signals are sparse, meaning that intelligent exploration is required.