Understanding and modeling curiosity-driven learning and intrinsic motivation in humans, monkeys and machines

Curiosity is a form of intrinsic motivation that is key in fostering active learning and spontaneous exploration of new tasks, new stimuli and new environments. It is a central aspect of intelligent behavior and fuels many aspects of our cognition and behavior, ranging from mechanisms that push infants to explore and discover their world to the drive for intellectual inquiry and scientific research.

The importance of curiosity was noted by psychologists in the 20th century, who distinguished between perceptual, semantic or diversive curiosity, and proposed measures of curiosity-related personality traits, and in more recent years, curiosity was shown to enhance reward and memory-related activity in the human brain. However, we are yet to construct a comprehensive theory that defines curiosity in the context of quantitative models of learning and decision making and probes its neural mechanisms. The aim of the Neurocuriosity associated team-project is to address these questions using a collaborative approach that integrates computational, behavioral and neurophysiological studies of curiosity in children and non-human primates (rhesus monkeys). This will significantly expand our understanding of the brain’s capacity for motivated cognition – i.e., its ability to go beyond passively registering information, and actively guide its own search for knowledge in complex spaces.

The Neurocuriosity project started in 2013 as an Inria associated team, than was renewed in 2016 both as an Inria associated team and through and HFSP grant.

Project partners

Computational sciences/artificial intelligence: Flowers Lab, Inria and Ensta Paris-Tech, France (Pierre-Yves Oudeyer, Manuel Lopes)
Neuroscience: Gottlieb Lab, Univ. Columbia, NY, US (Jacqueline Gottlieb)
Developmental psychology: Rochester Baby Lab, Univ. Rochester, US (Celeste Kidd)

Funding

Inria associated teams, Human Frontier Science Program, Fullbright grant

News

We have co-organized the “Designing for Curiosity” workshop, at CHI 2017, Denver, USA, https://www.crowdcurio.com/research/workshops/chi2017/

We are co-editing two Frontiers topic on intrinsic motivation and curiosity:

• “Modelling Play in Early Infant Development” in the journal Frontiers in Psychology, https://www.frontiersin.org/research-topics/5527/modeling-play-in-early-infant-development
• “Intrinsically Motivated Open-Ended Learning in Autonomous Robots”, https://www.frontiersin.org/research-topics/6558/intrinsically-motivated-open-ended-learning-in-autonomous-robots

We have co-organized (with T. Gliga, Birbeck college) the Second Interdisciplinary Symposium on information-seeking, curiosity and attention (Neurocuriosity 2016) in London, gathering senior researchers from neuroscience, psychology, education and machine learning/computational modelling. Speakers: J. Nelson, D. Markant, R. Ligneul, S. Kouider, M. Gruber, K. Murayama, J. O’Reilly, G. Baldassarre, P. Dayan, P-Y. Oudeyer, K. Doya, W. Shultz, A. Bell, L. Hunt, J. Gottlieb, D. Bell, K. Begus, L. Goupil,  L. Feigenson, D. Bavelier, T. Gliga.
Most videos and slides of the symposium are available on the Neurocuriosity 2016 symposium web site.

In 2014, we had organized the First Interdisciplinary Symposium on information-seeking, curiosity and attention (Neurocuriosity 2014) in Bordeaux, with invited speakers: M. Franck, S. Gershman, K. Gurney, J. Gottlieb, T. Gliga, P-Y. Oudeyer, C. Kidd, G. Westermann, L. Shulz, L. Lopes, J. Triesch, F. Sommer, A. Kacelnik, J. Gottlieb.
Videos and slides are on the Neurocuriosity 2014 symposium web site.

Selected publications

Forestier S, Oudeyer P-Y.  2017.  Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning. arXiv.

Forestier S, Oudeyer P-Y.  (2017)  A Unified Model of Speech and Tool Use Early Development. Proceedings of the 39th Annual Meeting of the Cognitive Science Society.

Foley, N. C., Kelly, S. P., Mhatre, H., Lopes, M., & Gottlieb, J. (2017) Parietal neurons encode expected gains in instrumental information. Proceedings of the National Academy of Sciences, 114(16), E3315-E3323.

Oudeyer P-Y. , Gottlieb J. and M. Lopes (2016) Intrinsic motivation, curiosity and learning: theory and applications in educational technologies, Progress in Brain Research, 229, pp. 257-284.

Daddaoua, N., Lopes, M. and Gottlieb, J. (2016): Intrinsically motivated exploration guided by uncertainty reduction and conditioned reinforcement in non-human primates. Scientific Reports, Feb 3;6:20202. doi: 10.1038/srep20202.

Oudeyer P-Y. , Lopes P-Y., Kidd C. and J. Gottlieb (2016) Curiosity and Intrinsic Motivation for Autonomous Machine Learning
ERCIM News, 107, pp. 34-35.

Gottlieb J,, Lopes, M. and Oudeyer, P.Y (2016): Motivated cognition: Neural and computational mechanisms of curiosity, attention and intrinsic motivation, in ‘Recent Developments in Neuroscience Research on Human Motivation’, Sung-il Kim, Johnmarshall Reeve, and Mimi Bong, editors.

Forestier S, Oudeyer P-Y. (2016) Curiosity-Driven Development of Tool Use Precursors: a Computational Model. Proceedings of the 38th Annual Conference of the Cognitive Science Society. :1859–1864.

Forestier S, Mollard, Y., Caselli, D., Oudeyer P-Y. (2016) Autonomous exploration, active learning and human guidance with open-source Poppy humanoid robot platform and Explauto library, Poster and demonstration at NIPS 2016 conference.

Baranes, A., Oudeyer, P.Y and Gottlieb, J. (2015): Eye movements reveal epistemic curiosity in human observers. Vision Research. 117:81-90. doi: 10.1016/j.visres.2015.10.009. Epub 2015 Nov 12 CLICK HERE FOR FULL DATASET

Baranes AF, Oudeyer PY, Gottlieb J. (2014) The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration. Frontiers in Neuroscience. 8:317. doi: 10.3389/fnins.2014.00317. eCollection 2014

Gottlieb J, Oudeyer PY, Lopes M, Baranes A. (2013): Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends in Cognitive Science, 2013 17(11):585-93. doi: 10.1016/j.tics.2013.09.001
http://media.wix.com/ugd/dfe481_9d074dfa57564852b382c197d2246ade.pdf

Related previous publications

Baranes, A., & Oudeyer, P. Y. (2013). Active learning of inverse models with intrinsically motivated goal exploration in robots. Robotics and Autonomous Systems, 61(1), 49-73.

Gottlieb J (2012): Attention, learning and the value of information. Neuron 76(2):281-295

Lopes, M., Lang, T., Toussaint M. and P-Y. Oudeyer (2012) Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress. Neural Information Processing Systems (NIPS), Tahoe, USA, 2012. (pdf)

Kidd, C., Piantadosi, S.T., & Aslin, R.N. (2012) The Goldilocks Effect: Human infants allocate attention to visual sequences that are neither too simple nor too complex. PLOS ONE. 7(5): e36399. [PDF]

Oudeyer, P. Y., Kaplan, F., Hafner, V. V. (2007) Intrinsic motivation systems for autonomous mental development. IEEE transactions on evolutionary computation, 11(2), 265-286.

Selected videos/talks

Pierre-Yves Oudeyer, Inria, France
Diversity of forms and developmental functions of curiosity-driven learning in humans and machines
(from the Neurocuriosity 2016 Symposium)

Slides: https://www.dropbox.com/s/hmq95swxj8mibyk/neurocuriositylondonOudeyer16.pdf?dl=0

Manuel Lopes, Inria Flowers, France
An overview of active learning approaches in machine learning, and applications to educational technologies
(from the Neurocuriosity 2014 Symposium)

Jacqueline Gottlieb,
Sampling useless information: promises and challenges in current research
(from the Neurocuriosity 2014 Symposium)

Celeste Kidd
Rational approaches to learning and development
(from the Neurocuriosity 2014 Symposium)