Games in humans and non-human primates - the prospects for game theoretical approach to neuroscience of decision making
•
2 likes•5,450 views
1) Game theory combined with neuroscience methods can provide insights into decision making by examining behavior and neural activity during strategic games. 2) Studies in monkeys have found neurons in areas like the lateral intraparietal area and superior colliculus that encode decision values and selection during mixed-strategy games. 3) Neuroimaging studies of games in humans find areas like the anterior insula and dorsolateral prefrontal cortex associated with emotional and cognitive processing during social decisions.
Report
Share
Games in humans and non-human primates - the prospects for game theoretical approach to neuroscience of decision making
- 1. Games in Humans and Non-human Primates: Scanners to Single Units The prospects for game theoretical approach to neuroscience of decision making Prepared for Neuroeconomics book study Chapter 6. by Sanfey and Dorris 2009.04.07, 07p.m. Kyongsik Yun, Ph.D. Candidate KAIST yunks@kaist.edu
- 2. From matrix to real world The gap between the experiments and real world in decision making studies • Traditional decision making studies were done in the confined experimental settings, isolating a specific aspect of complex decision making (e.g. risky, uncertain, ambiguous). • However, real life decisions are more complex and made in the context of a social interaction.
- 3. The combination of game theoretical tasks and modern neuroscientific methods is the solution. • This chapter examines – Game theory – Invasive electrophysiological techniques in monkeys during game-theoretic tasks – Non-invasive imaging techniques in humans during game-theoretic tasks
- 4. Game theory attempts to mathematically capture behavior in strategic situations, in which an individual's success in making choices depends on the choices of others. – von Neumann and Morgenstern (1947) Utility Theory Utility theory is unable to explain why people are often simultaneously attracted to both insurance and gambling. • Prospect theory – (Kahneman & Tversky 1979) Real world decision making is less selfish and more willing to consider reciprocity and equity (Camerer 2003).
- 5. Ultimatum Game Player 1 can offer a fair (F) or unfair (U) proposal; player 2 can accept (A) or reject (R). Guth et al. 1982 Nash Equilibrium is at odds: Low offers of less than 20% of the total amount are rejected about half of the time. Neuroscience has begun to offer clues as to the mechanisms underlying these decisions.
- 6. Trust Game 48 = 12 MU own endowment + 36 MU tripled transfer A rational and selfish trustee will never honor the trust given by the investor. In real situation, a majority of investors send some amount of their money to the trustee. Baumgartner et al. Neuron 2008
- 7. Prisoner’s dilemma game- simultaneous trust game • The Nash Equilibrium is mutual defection. • Players exhibit 50% mutual cooperation. Cooperate defect Cooperate defect
- 8. Public goods game: a generalized form of PDG • The self-interested solution is to free-ride and hope that everyone else contributes. • Players on average contribute about half of their endowment to the public good.
- 9. Mixed strategy game (e.g. matching pennies and the inspection game) • A pure strategy provides a complete definition of how a player will play a game. • A mixed strategy is an assignment of a probability to each pure strategy. • The Nash Equilibrium is to select randomly with equal probabilities. • Players attempt to infer the strategy of our opponent, using theory-of-mind processes.
- 10. It is unclear whether the decisions emerge from strategic or altruistic motivations. • Ultimatum Game proposer behavior – fairness? Fear to be rejected? • Examining these games in a neural context can begin to offer clues as to the motivations behind the decisions.
- 11. The benefits of combined approach between experimental economics and neuroscience methods • More precise characterizations of behavior (how decisions are actually made) • Neuroscience can provide important biological constraints on the processes involved.
- 12. The animal model (Rhesus Macaque) • To investigate “black box” during social interactions • For studying higher-order decision processes • General organization of their nervous system is similar to that of humans • Monkeys and humans display comparable strategies in mixed-strategy games
- 13. Advantages and disadvantages of a systems neurophysiology (invasive animal model) approach • Advantage: – direct access to the neural substrate – Exquisite temporal (<1ms) and spatial (ind. neurons) resolution – Artificial manipulation of neuronal activity can provide causal evidence, complementing the correlational evidence provided by neuronal recordings. • Disadvantages – Limitations exist in using non-human primates to infer the neural processes underlying human social interactions – Not suitable for more sophisticated games (UG, PDG) – No verbal instruction and using only operant conditioning techniques
- 14. Saccadic ( jumping-fixating) eye movements • A saccade is a fast movement of an eye, head or other part of an animal's body or device. It can also be a fast shift in frequency of an emitted signal or other quick change.
- 15. Benefits of visuosaccadic system investigation for decision making • Efficiently extract visual information in perceptual decision making • The neural circuitry underlying visual processing and saccadic control is well understood.
- 16. Adapting games for non-human primates LIP: estimating the desirability of sensory stimuli SC: selecting/preparing actions, (control eye movement) PFC: evaluating the consequences of actions LIP – gateway for the emergence of the brain Newsome and Shadlen 2001 Nature
- 17. Encoding the desirability of choice stimuli in LIP • At the end of visual processing stream • Encode the saliency of visual targets Platt and Glimcher 1999 Nature Sugrue, Corrado, Newsome 2005 NRN
- 18. Visuosaccadic version of mixed- strategy inspection game Nash Eq. 0.9 0.3 0.5 Inspection cost Dorris and Glimcher 2004 Neuron
- 19. LIP firing rates varies with desirability, not probability of outcome Dorris and Glimcher 2004 Neuron
- 20. Correlation between LIP Firing Rates and an Estimate of the Relative Desirability of Choices Dorris and Glimcher 2004 Neuron
- 21. Evolving response selection in midbrain superior colliculus • Intimately involved in saccade generation • See Figure 6.4 (Dorris unpublished) – B) Preferred (black)/ unpreferred (red) – C) Stimulus induced saccades deviate slightly towards the target. – D) angular deviation of saccades increases as the time of target presentation approaches. (일찍 자극, 적게 변화)
- 22. Evaluating the consequences of actions in frontal cortex • dlPFC is involved in integrating choice (i.e. left vs right) and reward (i.e. rewarded or unrewarded) information. • dACC encoded critical information about the temporal delay of previous rewards within a sequence of responses (Seo and Lee 2007)
- 23. General significance of decision making studies in non-human primates • Recording single neurons allow for moment-to-moment correlations between neuronal activity and behavioral responses • Providing unprecedented insight into the neuronal mechanisms underlying stochastic choice.
- 24. Games in humans • Imaging studies of social interaction – fMRI, PET – Stimuli of other human faces (static, 2D) (Winston et al. 2002) – Mentalizing (Gallagher et al. 2000) – Moral reasoning (Greene et al. 2001) Pioneering works of its kind
- 25. Automatic and intentional brain responses during evaluation of trustworthiness of faces Explicit judgments: whether an individual was trustworthy Implicit judgments: an unrelated age assessment Insula STS l. Amyg r. Amyg. The findings extend a proposed model of social cognition by highlighting a functional dissociation between automatic engagement of amygdala versus intentional engagement of STS in social judgment. Winston, Strange, O’Doherty, & Dolan NN 2002
- 26. Reading the mind in cartoons and stories medial prefrontal cortex (paracingulate cortex) Gallagher et al. 2000
- 27. Problems in the previous studies • Does the pattern of brain activation in response to the picture of a static, 2D face accurately reflect the brain’s response to the dynamic, embodied faces that we encounter in everyday life? • Is the pattern of brain activation in response to reasoning regarding hypothetical, fictitious scenarios the same as when grappling with significant real life social problems?
- 28. Solutions to improve ecological validity of experiments in neuroeconomics • To interact with other people in real social exchanges from outside the scanner (McCabe et al. 2001) • Hyperscanning technology (Montague et al. 2002) – Data collection efficiency – Open new vistas in social cognitive neuroscience
- 29. Another solution is to manipulate specific neurotransmitter systems • Tryptophan depletion can be used to decrease brain serotonin levels, reducing cooperative behavior (Wood et al. 2006) • Oxytocin elevation increases trust (Kosfeld et al. 2005) • TMS to temporarily activate or deactivate a brain region and then examine its effects on decision making (van’t Wout et al. 2005) • Lesion study (Koenigs and Tranel 2007)
- 30. Trust Game behavior Oxytocin increased trust. Oxytocin: filled bars Placebo: open bars Baumgartner et al. Neuron 2008
- 31. Fool me once, shame on you. Fool me twice, shame on oxytocin. Baumgartner et al. Neuron 2008
- 32. What is the role of PFC in emotion vs. cognition conflict decision making? Economic rationality: Social rationality: • economic value maximization Social value maximization • Fairness monitoring • • Absolute valuation Relative valuation • • vmPFC lesion -> R↑ dlPFC disruption -> A↑ • Knoch et al. Science 2006 Koenigs & Tranel, J Neurosci. 2007
- 33. Current research direction: social motivation reward • Striatum: scale reward magnitude (Cromwell and Schultz 2003), specifically the magnitude of monetary reward or punishment (O’Doherty 2004, Knutson and Cooper 2005) • Striatum may register social prediction errors to guide decisions about reciprocity (Rilling et al. 2002) • Caudate: related to how much reciprocity the investor had shown on previous trials, “intention to trust” (King-Casas et al. 2005) • caudate: altruistically punish (de Quervain et al. 2004) • Social altruism: the striatum was engaged both by receiving money and by donations to charity (Moll et al. 2006), and the activation was enhanced when the donation was voluntary as opposed to forced (Harbauch et al. 2007). -> important implications for informing public policy
- 34. Caution should be used when attempting to “reverse engineer” from patterns of brain activity to cognitive and social processes • Correlation analysis, TMS modulation, or patient work is crucial to buttress the causal relationship between behavior and brain activity. • These results appear to demonstrate that complex social processes recruit more basic mechanisms within the human brain, providing support for the notion that the brain uses a common reward metric. This also furthers the connection between the disparate branches of neuroeconomics from primary and secondary rewards (food, money) to more abstract social rewards (reciprocity, fairness).
- 35. Competition, Cooperation and Coordination • Classical models of decision making, both utility theory and game theory, have largely ignored the influence of emotions on how decisions are made, but recent research has begun to demonstrate their powerful effect.
- 36. The emotional brain The Papez circuit theory (1937) of the functional neuroanatomy of emotion Emotional experiences or feelings occur when the cingulate cortex integrates these signals from the hypothalamus with information from the sensory cortex. stream of thinking stream of feeling Output from the cingulate cortex to the hippocampus (3) and then to the hypothalamus (4) allows top–down cortical control of emotional responses. Dalgleish 2004 NRN
- 37. MacLean’s limbic system theory of the functional neuroanatomy of emotion the hippocampus received sensory inputs from the outside world as well as information from the internal bodily environment (viscera and body wall). Emotional experience was a function of integrating these internal and external information streams. Dalgleish 2004 NRN
- 38. Map of brain areas commonly activated in social decision-making studies. Theory of Mind: STS, MPFC, OFC Sanfey 2007 Science
- 39. Neuroscientific studies of emotions • Negative emotional states have been observed behaviorally as a result of both inequity and non-reciprocity, such as unfair offers in a UG. • These emotional reactions have evolved precisely to foster mutual reciprocity, to make reputation important, and to punish those seeking to take advantage of others (Nawak et al. 2000) • Neuroscienctific studies of this nature offer the potential to go beyond speculation and to examine the causal relationship between an emotional reaction and subsequent social decision, as well as investigating whether areas specialized for the processing of basic emotions may be coopted for more complex affective reactions.
- 40. Behavior of social interaction in the Ultimatum Game Participants had a stronger emotional reaction to unfair offers from humans than to those from computers. 40 Sanfey et al. Science, 2003
- 41. Neuroimaging results of the Ultimatum Game Sanfey et al. Science, 2003 Anterior insula: pain, distress, hunger, thirst (Denton et al. 1999), autonomic arousal (Critchley et al. 2000) Right anterior insula: aversive conditioning (Seymour et al. 2005) Defection rate in PDG is correlated with right anterior insula activation (Rilling et al. 2008) Anterior insula may play a role in marking a social interaction as aversive, and thus discouraging trust.
- 42. dlPFC is linked to cognitive processes • Frontal, “top-down” processes in reward studies suppress striatal activation. • TMS increased acceptance rate of unfair offers, providing strong evidence for a causal relationship between activation in this area and social decision making (Knoch et al. 2006) – Why social? Disruption of dlPFC using TMS might reduce cognitive processes to accept the offer, decreasing acceptance rates.
- 43. Modulating affective system in the Ultimatum Game • People in a sad mood (watching a 5-minute video rated as “sad”) reject more 'unfair' offers (Harle and Sanfey 2007). • Examining decision making performance in participants with disregulated emotional processing, such as patients with depression or schizophrenia, may be a useful future avenue of research. • Contradictory study: vmPFC lesion patients also reject unfair offers more frequently than do controls (Koenigs and Tranel 2007)
- 44. Significance of neuroscience approach to decision making • Decision making appears to involve the interaction among multiple subsystems governed by different parameters and possibly even different principles. • Neuroscience approach to decision making has the potential to inform economic theories of interactive decision making (inequity aversion, social utility functions) • Economic models based on the underlying neural patterns may provide useful constraint.