Understanding learning by inference
“The work suggests a new framework for learning in structured environments that goes beyond incremental, experiential learning of associations,” said Erie Boorman, assistant professor in the UC Davis Department of Psychology and Center for Mind and Brain and senior author on the paper.
In structured environments, individual elements are systematically related to each other as they often are in the real world. The study’s insights could be harnessed to improve educational strategies that promote the use of a cognitive map for accelerated learning through inferences, and potentially, approaches to hasten transfer of learning in machine learning in artificial intelligence, Boorman said.
Learning by inference versus association
Most studies of learning have focused on learning by association — how animals learn to associate one thing with another, through trial and error. The difference between what was expected and what actually happened drives learning in such cases.
When there is a hidden structure behind those associations, you can use direct observations to infer indirect, unseen outcomes, leaping ahead of the chain of direct association.
For example, knowing that the quality of seasonal foods is governed by changes in weather allows you to infer which are best to eat based on which foods are ripe during the same season, Boorman said. Observing ripe apples allows us to infer that pears should also be ripe, but not strawberries. This sort of structure is important to know when making decisions.
Another example is an investor inferring that the drop in Facebook shares can be attributed to a tech bubble, suggesting that Microsoft shares will likely drop soon, too.
“Knowing this hidden relationship means you can learn a lot faster,” Boorman said.
Testing learning in a structured system
To investigate how humans can use a cognitive map to learn information, graduate student Phillip Witkowski, project scientist Seongmin Park and Boorman created a task. In a series of trials, volunteers were asked to choose between two of four abstract shapes that would lead to either of two different gift cards (e.g., either Starbucks or iTunes). The volunteers made their choices based on two pieces of information: their estimate of the probability that each shape would lead to a particular gift card, and a randomly assigned payout for each gift card.
The shapes were divided into two pairs. In each pair, the probability that a shape would lead to a particular outcome was the inverse of the other shape. For example, if there was a 70% chance that shape A would lead to outcome 1, there was a 30% chance that shape B would lead to the same outcome, and vice versa for outcome 2. So the subjects could gain information about the likelihood of one outcome by inference from the other, like Microsoft shares from Facebook shares. The pairs of shapes were not connected, so the subjects could not learn anything about the results of choosing shapes C or D from the outcomes of choosing A or B.
The researchers followed how the subjects learned about the system by observing their progress over a series of trials. After analyzing the results, they found that the volunteers were using inferential learning to make decisions about which shapes to pick.
Some of the volunteers were invited back for the second part of the experiment, performing the same task while their brain activity was measured with functional magnetic resonance imaging. Learning is reflected in the brain by a burst of activity, a “belief update” when there is a difference between your prior and newly acquired knowledge. Activity linked to inferential learning was found in the prefrontal cortex and the area of midbrain where the neurotransmitter dopamine is released.
At the same time, the researchers found a representation of the hidden (or latent) probability controlling associations for A and B in the prefrontal cortex.
The fMRI results show the brain representing different outcomes in relation to each other, Boorman said. This representation allows for those “aha” moments.
Conventional thinking holds that incremental learning about rewards from direct experiences is reinforced by the release of dopamine in the brain. The new study also implicates dopamine but for inferential learning.
“Our work suggests a more general role for dopamine signals in updating beliefs through inference,” Boorman said.