The nEuro-economics seminar is a monthly seminar series organised by the HRL team. We invite Europe-based early career researchers to speak about their research at the crossroads of neuroscience, psychology and economics.
September 28, 2023 at 2pm, room Ribot
Pierre-Yves Oudeyer (Flowers lab, Inria Bordeaux): "Open-ended learning in humans and AI: the roles of curiosity and language"
Current approaches to Al and machine learning are still fundamentally limited in comparison with the amazing learning capabilities of children. What is remarkable is not that some children become world champions in certain games or specialties: it is rather their autonomy, open-endedness, flexibility and efficiency at learning many everyday skills under strongly limited resources of time, computation and energy. And they do not need the intervention of an engineer for each new task (e.g. they do not need someone to provide a new task specific reward function or representation).
I will present a research program, which I call Developmental Al, that studies models of open-ended development and learning. These models are used as tools to help us understand better how children learn, as well as to build machines that learn like children with applications in educational technologies and assisted scientific discovery.
I will ground this research program into several fundamental ideas proposed by developmental psychologists:
1. the chiId is autotelic, setting its own goals, spontaneously exploring the world like a curious little scientist while self-organizing it learning curriculum (e.g. Piaget, Berlyne);
2. intelligence develops in a social context, where language and culture are internalized to become cognitive tools (e.g. Vygostky and Bruner);
3. intelligence is embodied and develops through self-organization of the dynamical system formed by the brain-body-environment interactions (e.g. Thelen and Smith).
I will show how, together with many colleagues and students, we have worked on operationalizing these ideas in computer and robotic models. I will expiain how this has enabled to advance chiId development understanding and is now opening new possibilities for Al systems. ln particular, I will review our recent work on:
- Autotelic deep reinforcement learning, where agents learn to represent and sample their own goals towards open-ended learning
- The learning progress hypothesis, enabling efficient automatic curriculum learning in machines and humans
- Self-organization of developmental trajectories replicating fundamental dynamics of human sensorimotor learning
- Vygotskian Al, where deep RL agents leverage large language models (LLMs) as a cognitive tool for creative exploration.
June 15, 2023 at 2pm, room Ribot
Ruth Van Holst (Amsterdam UMC): "Risky Choices, Lost Bets: Reward Processing and Decision Making in Gambling Disorder"
Despite knowing that the house always wins, gambling remains popular, defying rationality. This enduring paradox exposes the interplay between economic, cognitive, and emotional factors. Some individuals experience a loss of control over gambling, leading to harms like debt, illegal activities, and conflicts. Gambling disorder, recognized as a psychiatric condition, exemplifies the extreme manifestation of this behaviour. Dr Ruth van Holst's talk offers insights from her lab's research on gambling disorder, focusing on reward and decision-making processes in healthy and gambling brains to discern how behaviour can spiral out of control.
June 1, 2023 at 2pm, room Ribot
Bjorn Lindstrom (Karolinska Institutet): "Human social learning from simple mechanisms"
Learning from and about others – social learning – is key for our success as a species. Ideas and behaviors are transmitted between individuals and across generations via social learning, which gives rise to the evolution of human cultures. Despite intense multidisciplinary interest, the mechanisms underlying social learning are not well understood. I will present two projects that analyze different types of human social learning through the lens of reward learning theory. First, using both observational data from different social media platforms and experimental data, I will demonstrate that human engagement with social media can be explained by basic reward learning mechanisms fueled by social rewards – “likes”. Second, I will present a series of agent-based simulations and four experimental studies that tests the hypothesis that reward learning provides a common mechanism for a range of human social learning heuristics, such as conformity, increased copying under uncertainty, and copying based on others' payoffs. Together, these projects show that a wide range of social learning phenomena might emerge from the fundamental principles of reward learning.
May 25, 2023 at 2pm, room Marbo
Lei Zhang (University of Birmingham): "Cracking the neurocomputational mechanisms of flexible (social) behavior in health and disease"
Flexible (learning) behavior refers to the adaptive change of behavior in response to changing environmental contingencies. Although flexible learning behavior has been studied extensively in the past, very little is known about its neurocomputational mechanism. In this talk, I will first illustrate how to assess flexible learning behavior through the lens of computational modeling, then I will showcase two studies, one (fMRI study) focusing on flexible learning in a social context, and the other (behavioral study with large sample) focusing on flexible learning in individuals with autism. These studies are expected to motivate new research avenues in social neuroscience and computational psychiatry.
April 20 2023 at 2pm, room Langevin
Jacquie Scholl (Lyon Neuroscience Research Centre): "Decision making and foraging under threat – neural activity and transdiagnostic clinical relevance "
Mood instability is a feature of bipolar disorder (BD). While, fluctuating mood episodes have traditionally be seen as lasting weeks or months, more recent work has shown that, in fact, bipolar patients show large day-to-day fluctuations in mood even when symptoms are in the non-clinical range. This degree of mood instability is affected by lithium treatment. Understanding the processes underpinning these fluctuations may help us develop and assess more effective treatment approaches Here we examined the cognitive and neural mechanisms – reduced stabilizing adaptations of choices to past outcomes - underlying these fluctuations and how they are changed with the mood stabilizer lithium.
In a separate line of work, we examined foraging under threat in a gamified task in which participants needed to balance their activities between those that were rewarding and others that ensure the safety of their avatar. In this naturalistic scenario, we measured how emotions (stress and excitement) were evoked by the environment and shaped behaviour. We also examined the impact of transdiagnostic dimensions (anxiety, apathy, compulsivity) on task behaviour in a large online sample. With 7T FMRI, we examined the neural subcortical underpinnings.
March 16, 2023 at 2pm, room Langevin
Wataru Toyokawa (University of Konstanz): "Self-organised collective intelligence with consistently risk averse individuals"
Conventional models of collective intelligence rely on individuals making unbiased, at least partially informed decisions. However, animal decision making through repeated experience may often be biassed due to the constraints in information sampling (so-called the hot stove effect). Considering the ubiquity of conformist social learning, a process widely considered to be bias-amplification, it seems paradoxical that improvements in decision-making performance under social influences still prevail. How can animals overcome the potentially suboptimal bias collectively? Here we show, through model analyses and large-scale interactive behavioural experiments with 585 human subjects, that conformist influence can indeed promote favourable risk taking in repeated experience-based decision making, even though many individuals are systematically biassed towards adverse risk aversion. Although strong positive feedback conferred by copying the majority's behaviour could result in unfavourable informational cascades, our differential equation model of collective behavioural dynamics identified a key role for increasing exploration by negative feedback arising when a weak minority influence undermines the inherent behavioural bias. This “collective behavioural rescue” highlights a benefit of collective learning in a broader range of environmental conditions than previously assumed and resolves the ostensible paradox of adaptive collective behavioural flexibility under conformist influences.
February 13, 2023 at 2:00pm, room Langevin
Azzurra Ruggeri (Max Planck Institute for Human Development): "The emergence and developmental trajectory of ecological active learning"
This talk will introduce the Ecological Active Learning framework, which focuses on children’s ability to adapt and tailor their active learning strategies to the particular structure and characteristics of a learning environment. In particular, I will present the results of several seminal studies indicating that efficient, adaptive search strategies emerge around 3 years of age, much earlier than previously assumed. This work highlights the importance of developing age-appropriate paradigms that capture children's early competence to gain a more comprehensive and fair picture of their active learning abilities. Also, it offers a process-oriented theoretical framework that can accommodate and reconcile a sparse but growing body of work documenting children’s active and adaptive learning.
January 19, 2023 at 2:30pm, room Langevin
Eva Pool (University of Geneva): "Neural mechanisms of affective value learning underlying problematic reward seeking behavior"
A common symptom across many clinical conditions, such as drug addiction, is the willingness to go to extraordinary lengths to obtain an object of desire, even though once obtained the object is not experienced as pleasurable. What are the mechanisms that make the human brain vulnerable to situations where choice behavior is hijacked in the service of outcomes that are not valued by the individual? To address this question, we conducted a series of studies combining classical experimental paradigms of affective value learning (i.e., Pavlovian conditioning), with eye-tracking and functional imagining techniques. During Pavlovian conditioning, participants generated a set of conditioned responses to a conditioned stimulus that predicted the subsequent delivery of an affectively significant outcome, namely food. Our results suggest that Pavlovian conditioning involves at least two anatomically and computationally distinct learning signals: one that learns the value of the outcome, and one that learns the sensory properties of the outcome. These neural learning signals generated multiple and parallel conditioned responses. Strikingly, these conditioned responses had different sensitivities to outcome devaluation: Pavlovian responses based on the representation of the outcome’s value flexibly adapted to outcome devaluation, whereas Pavlovian responses based on the sensory properties’ representation were resistant to outcome devaluation. These findings shed some light on the mechanisms underlying Pavlovian learning and provide new insights into the understanding of persistent reward-seeking behaviors when the reward is no longer valued by the individual.
October 14, 2022 at 2:45, room Ribot
Carlos Alos Ferrer (Zurich Center for Neuroeconomic ): "The Neural Foundations of Behavioral Anomalies in Decisions under Risk"
A number of behavioral anomalies have been systematically documented in decisions under risk in economic contexts. The explanations for those frequently invoke conflict between different decision processes or different dimensions of underlying decision values, which should elicit midfrontal theta activity in the human brain. We analyze the role of conflict in behavioral anomalies (the certainty effect and the classical preference reversal phenomenon) by targeting midfrontal theta activity in the human brain by means of the EEG (event-related potentials and frequency analysis).
June 3, 2022 at 2:30pm, room Ribot
Charley Wu (University of Tubingen): "The developmental tracjetory of learning and exploration"
Learning from past experiences helps orient the exploration of unknown environments. Yet, in real-world environments with a vast number of possibilities, it is simply not feasible to try out and explore all options. Instead of exploring randomly, a growing body of research shows that exploration is guided by predictive generalizations about expected rewards and uncertainty-directed exploration. These same mechanisms appear across a variety of spatial, conceptual, graph-structured, and social environments. The paradigm is both intuitive and richly complex, allowing us to characterize developmental changes in learning and exploration across a wide range of ages, from 5 to 55. Through both behavioral and model-based analyses, we provide the first empirical test of the "simulated annealing" analogy of ageing. We show that “cooling off” does not only apply to the single dimension of randomness (i.e., decision-noise), but rather, development resembles a stochastic optimization process in the space of learning strategies. What begins as large tweaks during childhood, plateaus and converges in adulthood. Remarkably, none of the optimization algorithms discovered reliably better regions of the strategy space than adult participants, suggesting an incredible efficiency of human development. Some notable differences between solutions discovered by human development vs. stochastic optimization provide valuable insights into how we make the most of limited cognitive resources.
May 20, 2022 at 2:30pm, room Ferdinand BERTHIER (U207)
Veronika Zilker (Max Planck Institute): "Nonlinear Probability Weighting Can Reflect Attentional Biases in Sequential Sampling"
Nonlinear probability weighting allows cumulative prospect theory (CPT) to account for key violations of utility maximization in decision making under risk (e.g., certainty effect, fourfold pattern of risk attitudes). It describes the impact of risky outcomes on preferences in terms of a rank-dependent nonlinear transformation of their objective probabilities. However, it is unclear how specific shapes of the probability weighting function come about on the level of cognitive processing. The attentional Drift Diffusion Model (aDDM) formalizes the finding that attentional biases toward an option can shape preferences within a sequential sampling process. Here I link these two influential frameworks. The aDDM is used to simulate choices between two options while systematically varying the strength of attentional biases to either option. The resulting choices were modeled with CPT. Changes in preference due to attentional biases in the aDDM were reflected in highly systematic signatures in the parameters of CPT’s weighting function (curvature, elevation). These results advance the integration of two prominent computational frameworks for decision making. Moreover, I demonstrate that attentional biases are also empirically linked to patterns in probability weighting as suggested by the simulations, and test whether these effects of attention allocation on probability weighting are causal. Overall, the findings highlight that distortions in probability weighting can arise from simple option-specific attentional biases in information search, and suggest an alternative to common interpretations of weighting-function parameters in terms of probability sensitivity and optimism. They also point to novel, attention-based explanations for empirical phenomena associated with characteristic shapes of CPT’s probability-weighting function (e.g., certainty effect, description–experience gap), and to possible interventions to alleviate common biases in decision making under risk.
April 21, 2022 at 4:30pm, room Camille Marbo
Christopher Summerfield (University of Oxford): "Learning and generalisation of task knowledge in humans and neural networks"
There has been a renaissance of interest in connectionist networks as models of biological computation. During sensory perception, deep neural networks learn representations that resemble those in primate neocortex. However, neural networks learn to perform and generalise cognitive tasks in very different ways to people. In my talk, I will explore these differences, and suggest computational adaptations that allow neural networks to learn multiple tasks in series, reconfigure task knowledge from limited data, and generalise knowledge between tasks.
March 29, 2022 at 11am, ENS, amphitheater Jaurès
Claire Gillan (Trinity College Dublin): "Getting personal with network theory of mental health and illness"
Network theory of psychopathology posits that mental health disorders like depression might be better understood as complex systems defined by interacting elements, or ‘symptoms’, like low mood, excessive guilt and insomnia. This challenges the traditional view in psychiatry that disorders themselves are the latent cause of symptoms and offers an explanation as to why psychiatry has failed to find clear neurobiological, genetic, or environmental causes of specific DSM disorders. Though there is much excitement about the potential for network approaches to explain individual differences in clinical presentation, help us understand vulnerability, and potentially tailor treatments, there is snag; almost all of the empirical research supporting network theory rests on between-subject analyses in cross-sectional data. In this talk, I will stress the need for constructing and interrogating personalised within-subject networks to move this field forward. This allows us to ask not whether things like insomnia and guilt correlate across individuals, but how reliably guilt precedes insomnia within a person. Focusing on a core prediction of network theory, that more tightly connected networks of symptoms are associated with vulnerability, severity, and persistence of illness, I will describe some recent efforts in this area using a variety of data sources. These include clinical panel data from >65,000 patients followed through cognitive behavioural therapy, personalised networks constructed from depression-related language in Tweets (N=946), and twice-daily self-reported affect from an experience sampling study (N=208) via the neureka app (www.neureka.ie).
February 28, 2020, 2:30pm, ENS, Jaurès building, Langevin room, 24 rue Lhomond, 75005 Paris
Dirk Wulff (CDS, University of Basel, Switzerland): "Strategic exploration and memory representations in decisions from experience"
Computational accounts of decisions from experience often treat the information contained in new experiences as a disposable product. They are used once to update running evaluations of the available options, but once they have served their purpose they are simply discarded. When taken literal, these accounts imply that people should possess no declarative memory of the experiences that they have made. In this talk, I present evidence from a reanalysis of the 1.2 million sampling decisions in our meta-analysis (Wulff et al., 2018) and new experimental data that demonstrates that people indeed possess robust memories of experienced outcomes, as well as their relative frequencies, and that these representations guide strategic exploration. These findings are difficult to reconcile with popular accounts of decisions from experiences, in particular those building on reinforcement learning models, and highlight the need for incorporating an episodic- or instance-based memory system in accounts of decisions from experience.
January 31st, 2020, 2:30pm, ENS, Jaurès building, Ribot room, 24 rue Lhomond, 75005 Paris
Valérie Dufour (IPHC, Strasbourg): "Decision making under risk and ambiguity in human and non-human primates"
Decision making under risk is fundamental in humans and other animals. Biologists generally aim at highlighting particular attitudes towards risk (i.e. risk proneness or risk aversion for example) that would reflect naturally selected adaptations to past environmental conditions for a given species. By contrasts with human studies of choice, those studies often fail to consider the interplay between the mechanisms involved in the decision such as the respective role of loss aversion, risk attitudes and probability distortion. In a first part of this talk, I will present data of four ape and two monkey species tested in a food gambling game to uncover the determinants of their gambling decisions. Data were first analysed using CPT and EUT models in an attempt to quantify parameters known to influence decision making in humans. In parallel, we investigated whether some individuals used decisional heuristics to reduce the cost of evaluating odds at each new trial. Data show that several subjects used the Maximax heuristic, focussing on potential gains, while ignoring potential losses. In a second part of this talk, we investigated the response of these species under ambiguous conditions. Subjects were tested in a test very similar to the gambling game but with some information missing. They had no exact knowledge of the odds associated to the outcome of their gambling. Dealing with unpredictability may be more challenging and require more cognitive flexibility. We expected some species to reject ambiguity and/or use even simpler decision rules than under risk. However, under this ambiguous context, individuals gambled as if they had built expectations about the missing information and more so in orangutans, gorillas and chimpanzees. Using decision trees, we could identify each step of the decisional process. Results show that non-human primates can combine several decision rules to make their decision in an unpredictable environment.
October 15, 2019, 11am, ENS
Yael Niv (Princeton University): "The interaction between mood and reward"
A good mood can be a blessing, but persistent bad moods or mood swings are so detrimental to our well-being that they are at the root of many psychiatric disorders. In this talk, I will present a computational model of mood and valuation that suggests that mood and unexpected outcomes (rewards or punishments) influence each other bi-directionally. After showing empirical evidence for both sides of the interaction, I will discuss the theoretical implications of such a positive feedback loop, especially for pathologies of mood instability. Finally, I will suggest that mood is not simply epiphenomonological, but might play a role in tracking global changes in the environment. This is especially useful in environments with dependencies between different states, and across time (momentum). Properly calibrated, mood may therefore allow rapid adaptation to some kinds of change and generalization of learning from one situation to another.
September 20, 2019, 2:30pm, ENS
Brian Hill (HEC): "Updating Confidence in Beliefs"
This paper develops a belief update rule under ambiguity, motivated by the maxim: in the face of new information, retain those conditional beliefs in which you are more confident, and relinquish only those in which you have less confidence. We provide a preference-based axiomatisation, drawing on the account of confidence in beliefs developed in Hill (2013). The proposed rule constitutes a general framework of which several existing rules for multiple priors (Full Bayesian, Maximum Likelihood) are special cases, but avoids the problems that these rules have with updating on complete ignorance. Moreover, it can handle surprising and null events, such as crises or reasoning in games, recovering traditional approaches, such as conditional probability systems, as special cases.
May 17, 2019, 2pm, ENS, room Langevin
Aurélien Baillon (Erasmus School of Economics): "Signal perception and belief updating"
This paper introduces a theory of signal perception to study how people update their beliefs. By allowing perceived signals to deviate from actual signals, we identify the probability that people miss or misread signals, giving indexes of conservatism and confirmatory bias. In an experiment, we elicited perceived signals from choices and obtained a structural estimation of the indexes. The subjects were conservative and acted as if they missed 43% of the signals they received. Also they exhibited confirmatory bias by misreading 19% of the signals contradicting their prior beliefs.
April 19, 2019, 3pm, ENS, room Langevin
Shauna Parkes (Institut de Neurosciences Cognitives et Intégratives d'Aquitaine): "Neural and psychological bases of goal-directed behaviour in the rat"
Appropriate decision making is critical for adapting to a changing environment. Every day we must make decisions based on internal goals and the expectation that a given action will lead to goal achievement. Such decisions are experimentally defined as “goal-directed.” Over the years, we have been particularly interested in the neural circuits of incentive learning and memory; that is, the brain regions and circuits that encode and retrieve goal values to guide adaptive choice. Current evidence indicates that interactions between the insular cortex, the striatum and the amygdala are crucial for such incentive learning. Here, I will review evidence from free operant tasks employing causal interventions in rats to outline the distinct involvement of each of these regions, and the neural pathways between these regions, in the mental representation of goal values. I will also share some purely behavioural data examining how context may influence the goal representation as well as the effect of environmental factors such as diet on the balance between actions and habits.
September 12, 2018, 11am, room Langevin
Elliot Ludwig (Warvic Business School - The University of Warwick): "Memory biases in risky decisions from experience"
May 2d, 2018, 2pm, room Langevin
Eric Schultz (Computational Cognitive Neuroscience Lab & the Data Science Initiative Harvard University): "Exploration and generalization in structured bandits"
February 3d, 2017, 11:30am-12:30, room Assia Djebar
Uri Hertz (Univ. College London): "How to influence others and get approval from your granny: The neural computations underlying strategic management of social influence"