In this study, reinforcement learning models that are not classically used in experimental economics has been applied to a multistep exchange task of the emergence of money derived from a classic search-theoretic paradigm for the emergence of money. This method allowed the researchers to highlight the importance of counterfactual feedback processing of opportunity costs in the learning process of speculative use of money and the predictive power of reinforcement learning models for multistep economic tasks. Those results constitute a step toward understanding the learning processes at work in multistep economic decision-making and the cognitive microfoundations of the use of money.

Abstract : 

Money is a fundamental and ubiquitous institution in modern economies. However, the question of its emergence remains a central one for economists. The monetary search-theoretic approach studies the conditions under which commodity money emerges as a solution to override frictions inherent to interindividual exchanges in a decentralized economy. Although among these conditions, agents’ rationality is classically essential and a prerequisite to any theoretical monetary equilibrium, human subjects often fail to adopt optimal strategies in tasks implementing a search-theoretic paradigm when these strategies are speculative, i.e., involve the use of a costly medium of exchange to increase the probability of subsequent and successful trades. In the present work, we hypothesize that implementing such speculative behaviors relies on reinforcement learning instead of lifetime utility calculations, as supposed by classical economic theory. To test this hypothesis, we operationalized the Kiyotaki and Wright paradigm of money emergence in a multistep exchange task and fitted behavioral data regarding human subjects performing this task with two reinforcement learning models. Each of them implements a distinct cognitive hypothesis regarding the weight of future or counterfactual rewards in current decisions. We found that both models outperformed theoretical predictions about subjects’ behaviors regarding the implementation of speculative strategies and that the latter relies on the degree of the opportunity costs consideration in the learning process. Speculating about the marketability advantage of money thus seems to depend on mental simulations of counterfactual events that agents are performing in exchange situations.

Reference: Lefebvre, Germain, Nioche, Aurélien and Bourgeois-Gironde, Sacha and Palminteri, Stefano (Nov. 2018). Contrasting temporal difference and opportunity cost reinforcement learning in an empirical money-emergence paradigm. PNAS. DOI: 10.1073/pnas.1813197115
 

Contacts:

Sacha Bourgeois-Gironde – sbgironde@gmail.com
Institut Jean-Nicod 
UMR CNRS 8129
Pavillon Jardin
Ecole Normale Supérieure, 29 rue d'Ulm
F-75005 Paris, France 

Stefano Plaminteri - stefano.palminteri@ens.fr
Laboratoire de Neurosciences Cognitives Computationnelles
UMR INSERM
Ecole Normale Supérieure, 29 rue d'Ulm
F-75005 Paris, France