Biological Brain Outpaces AI in Learning, Thanks to Structured Research – Neuroscience News

Summary: When it comes to fast learning, biological brains seem to have an advantage over current AI technology.

Scientists worked with different models of reinforcement learning to understand the algorithms the brain uses to learn. They found that directed exploration by animals makes learning more efficient and requires less experience, unlike artificial agents that explore randomly.

Key facts:

1. The study shows that animal-driven research makes learning more efficient, which could help create better AI agents that can learn faster and need less experience.
2. AI agents need a lot of experience to learn something and explore the environment thousands of times, whereas a real animal can learn the environment in less than ten minutes.
3. The results of the study highlight the need for more efficient learning algorithms that can explain the behavior of animals as they explore and learn their spatial environment.

Source: Sainsbury’s Wellcome Centre

Neuroscientists have discovered how exploratory activities enable animals to learn their spatial environment more effectively. Their findings could help create better AI agents that can learn faster and require less experience.

Researchers at the Sainsbury Wellcome Center and the Gatsby Computational Neuroscience Unit at UCL have found that instinctive exploratory runs by animals are not random. These targeted actions allow mice to efficiently learn the world map.

A study published today Neurondescribes how neuroscientists tested their hypothesis that specific exploratory actions animals perform, such as darting rapidly toward objects, are important in helping them learn to navigate their environments.

“There are many theories in psychology about how doing certain activities facilitates learning. In this study, we tested whether simply observing obstacles in the environment is sufficient to learn about them, or whether purposeful, sensory-guided actions help animals build a cognitive map of the world,” said Professor Tiago Branco, Group Leader at Sainsbury’s Wellcome. Center and corresponding author on paper.

In previous work, SWC scientists observed a correlation between how well animals learn to avoid an obstacle and the number of times they had run toward the object. In this study, Philip Shamash, a PhD student at SWC and the first author of the paper, conducted experiments to examine the effects of interfering with the animals’ exploratory journeys.

By expressing a light-activated protein called channelrhodopsin in one part of the motor cortex, Philip was able to use optogenetic tools to prevent animals from initiating exploratory runs toward obstacles.

The team found that even though the mice had spent a lot of time watching and sniffing the obstacles, they didn’t learn if they weren’t allowed to run towards them. This shows that the very act of instinctive exploration helps animals learn a map of their environment.

To investigate the algorithms that the brain might use to learn, the team worked with Sebastian Lee, a PhD student in Andrew Sachse’s lab at SWC, to run different reinforcement learning models that humans have developed on artificial agents and observe which ones perform best. closely reproduces mouse behavior.

There are two main classes of reinforcement learning models: model-free and model-based. The team found that under some conditions the mice function without a model, while under other conditions they appear to have a model of the world. Therefore, the researchers introduced an agent that can discriminate between model-free and model-based. This is not necessarily the way the mouse brain works, but it helped them understand what was needed in a learning algorithm to explain the behavior.

“One of the problems with artificial intelligence is that agents need a lot of experience to learn something. They have to explore the environment thousands of times, while a real animal can learn the environment in less than ten minutes.

“We think this is partly because, unlike artificial agents, animal exploration is not random and instead focuses on salient objects. Such directed exploration makes learning more efficient and therefore requires less experience for them to learn,” explains Professor Branco.

The next steps for researchers are to examine the relationship between the performance of exploratory activities and the representation of subgoals. The team is now recording in the brain to discover which areas are involved in representing subgoals and how exploratory actions lead to the formation of representations.

Funding: This research was funded by a Wellcome Senior Research Fellowship (214352/Z/18/Z) and a Sainsbury Wellcome Center Core Grant from the Gatsby Charitable Foundation and Wellcome (090843/F/09/Z), Sainsbury Wellcome Center PhD Programme. and a Sir Henry Dale Fellowship from the Wellcome Trust and the Royal Society (216386/Z/19/Z).

For these AI and neuroscience research news

Author: April Kashin-Garbut
Source: Sainsbury’s Wellcome Centre
Contact person: April Cašin-Garbuts – Sainsbury’s wellness center
Image: Image featured in Neuroscience News

Preliminary study: Open access.
Sebastian Lee et al. “Mice identify subgoal locations using an action-oriented mapping process”. Neuron

---

Abstract

Mice locate subtargets using an action-based mapping process

Highlight

• Interrupting obstacle runs while exploring prevents sub-objectives from being learned
• The choice of sub-goal during escape depends on the position of the mouse in the environment
• A two-system reinforcement learning agent replicates mouse behavior

Summary

Mammals create mental environmental maps as they explore their surroundings. Here we explore which research elements are important to this process. We studied mouse escape behavior, in which mice are known to memorize the locations of subgoals—the edges of obstacles—to make efficient escape routes to refuge.

To examine the role of exploratory actions, we developed closed-loop neuron stimulation protocols to interrupt various actions while mice explored. We found that blocking running movements directed at the edges of obstacles prevented the learning of subgoals; however, blocking multiple control movements had no effect.

Reinforcement training simulations and spatial data analysis suggest that artificial agents can match these results when they have a region-level spatial representation and are explored with object-directed movements. We conclude that mice use an action-oriented process to integrate subgoals into a hierarchical cognitive map.

These findings expand our understanding of the cognitive toolkit that mammals use to acquire spatial knowledge.