Psychological Science http://pss.sagepub.com/
Does Confidence Use a Common Currency Across Two Visual Tasks? Vincent de Gardelle and Pascal Mamassian Psychological Science 2014 25: 1286 originally published online 3 April 2014 DOI: 10.1177/0956797614528956 The online version of this article can be found at: http://pss.sagepub.com/content/25/6/1286
Published by: http://www.sagepublications.com
On behalf of:
Association for Psychological Science
Additional services and information for Psychological Science can be found at: Email Alerts: http://pss.sagepub.com/cgi/alerts Subscriptions: http://pss.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
>> Version of Record - Jun 4, 2014 OnlineFirst Version of Record - Apr 3, 2014 What is This?
Downloaded from pss.sagepub.com at Universite de Paris 1 on July 8, 2014
528956
research-article2014
PSSXXX10.1177/0956797614528956de Gardelle, MamassianConfidence Comparison Across Tasks
Short Report
Does Confidence Use a Common Currency Across Two Visual Tasks?
Psychological Science 2014, Vol. 25(6) 1286–1288 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0956797614528956 pss.sagepub.com
Vincent de Gardelle1,2 and Pascal Mamassian3,4
1 Centre d’Économie de la Sorbonne, CNRS UMR 8174; 2Paris School of Economics; 3Laboratoire des Systèmes Perceptifs, CNRS UMR 8248, Paris, France; and 4Département d’Études Cognitives, École Normale Supérieure, Paris
Received 9/23/13; Revision accepted 2/25/14
Observers typically know about their own uncertainty during simple perceptual decisions: Better performance is linked to greater confidence in being correct (e.g., Fleming, Dolan, & Frith, 2012; Peirce & Jastrow, 1884; Yeung & Summerfield, 2012). However, there are many outstanding issues concerning how observers evaluate their confidence, an ability we refer to here as metacognition. At the processing level, computation of confidence could take place early (Baranski & Petrusic, 1994; Zylberberg, Barttfeld, & Sigman, 2012) or late (Pleskac & Busemeyer, 2010) during the decision process. Here, we focus on the representation level of confidence. More specifically, we report an experiment in which we investigated whether confidence is represented in a taskspecific or a generic (task-independent) format. Although conceptually very different, these two possibilities (task-specific vs. generic format) are not easily distinguishable via confidence ratings. Consider an orientation discrimination task: Observers judge the tilt of a bar as clockwise or counterclockwise relative to vertical and then rate their confidence in that judgment. The confidence rating could reflect the absolute tilt estimated in degrees (task-specific format) or, alternatively, the probability of success given this estimate (task-independent confidence). Critically, however, according to traditional signal detection theory (Green & Swets, 1966), these two quantities are perfectly correlated and indistinguishable (see Mathematical Appendix in the Supplemental Material available online). We therefore took another approach to address this question. In a computerized experiment, observers (N = 35) performed two visual tasks (orientation and spatialfrequency discrimination) on Gabor stimuli (for details on the method, see Supplementary Methods in the Supplemental Material). On each trial, two Gabor stimuli were presented in succession, and observers judged the second orientation as clockwise or counterclockwise relative to the first orientation, or judged the spatial frequency of the second stimulus as higher or lower
than the spatial frequency of the first stimulus. The tasks were known in advance and manipulated across blocks. After every pair of trials, observers made a comparative judgment of their confidence (Barthelmé & Mamassian, 2009, 2010), indicating the trial on which they felt they were more likely to have been correct. In two blocks, the same task was performed on all trials, so that confidence comparisons were for the same task (within-task condition); in the other two blocks, the task alternated between trials, so that confidence comparisons were for pairs of trials involving different tasks (across-task condition). If confidence is task-specific, then comparing confidence across two different tasks should be harder (i.e., observers should be less able to separate high-performance from low-performance trials) than comparing confidence across two instances of the same task. Alternatively, if confidence is accessed as an abstract and task-independent quantity (e.g., success probability), then comparing confidence in two decisions should not depend on whether the same task was performed. We constructed psychometric curves describing the proportion of responses indicating counterclockwise rotations in the orientation task or increase in spatial frequency in the frequency task, as a function of the angular difference between the two orientations or the log-ratio of the two spatial frequencies. We quantified performance via sensitivity (i.e., the slope of the psychometric curve; see Psychometric Analyses in the Supplemental Material). We measured metacognitive ability as the change in sensitivity from low-confidence to high-confidence trials, by using the observers’ confidence-comparison response for each trial pair to classify one trial as high confidence and the other as low confidence. As expected, sensitivity increased with Corresponding Author: Vincent de Gardelle, Centre d’Économie de la Sorbonne, 112 bvd de l’Hôpital, 75013 Paris, France E-mail:
[email protected]
Downloaded from pss.sagepub.com at Universite de Paris 1 on July 8, 2014
Confidence Comparison Across Tasks 1287
a
High Confidence Low Confidence
Frequency 1.0
.8
.8
Proportion “F2 > F1”
Proportion “O2 > O1”
Orientation 1.0
.6 .4 .2 .0 –5
0
.6 .4 .2 .0 –0.1
5
Angular Difference (O2 – O1)
b
0
0.1
Log (F2/F1)
Low Confidence High Confidence
Low Confidence High Confidence
Orientation
Frequency
67%
53%
68%
60
0.8
58%
0.6
0.4
0.2 0.0
Within-Task Comparison
Sensitivity (1/log-unit)
Sensitivity (1/deg)
50 40 30 20 10 0
Across-Task Comparison
Fig. 1. Experimental results. The graphs in (a) show 1 participant’s psychometric curves for low- and high-confidence trials in the orientation and frequency tasks. For the orientation task, the proportion of trials on which the observer reported that the second orientation (O2) was rotated counterclockwise relative to the first (O1) is plotted as a function of the angular difference between the two orientations (in degrees). For the frequency task, the proportion of trials on which the observer reported that the second spatial frequency (F2) was higher than the first (F1) is plotted as a function of the logarithm of the ratio of the two spatial frequencies. The data have been fitted with cumulative Gaussian curves. The graphs in (b) show mean sensitivity (the inverse of the standard deviation of the fitted cumulative Gaussian curve) for the entire sample, calculated separately for each combination of confidence level and comparison type for each task. Error bars represent 1 SEM. The percentages correspond to the average confidence modulation indices (calculated as the sensitivity difference between low- and high-confidence trials divided by the average sensitivity across both confidence levels).
confidence (see Fig. 1a) in both tasks—orientation: 0.35 versus 0.71 deg−1, t(34) = 15.43, p < .001; frequency: 27.5 versus 49.7 log-units−1, t(34) = 14.58, p < .001. In other words, observers were able to track their own uncertainty above and beyond its variance due to the stimulus. Critically, we assessed whether this metacognitive ability (i.e., the increase in sensitivity with confidence) was stronger in the within-task condition than in the across-task condition, as predicted by the task-specificformat hypothesis, or showed no effect of comparison type, as predicted by the generic-format hypothesis. Our data (Fig. 1b) supported the latter hypothesis: For both tasks, the Confidence (higher vs. lower) × Comparison Type (within-task vs. across-task) analysis of variance revealed main effects of confidence, both Fs(1, 34) > 106, ps < .001, but no main effect of comparison type, both Fs(1, 34) < 1, ps > .33. Critically, there was no interaction for either the orientation task, F(1, 34) = 0.01, p = .91, or the frequency task, F(1, 34) = 0.08, p = .78. Bayesian analyses (Masson, 2011) indicated positive evidence for the absence of an interaction (null hypothesis, H0) in both tasks—orientation: p(H0) = .85; frequency: p(H0) = .85. Thus, if an interaction exists, it is likely to be extremely weak. To pool observations across tasks, we defined a confidence modulation index (CMI) quantifying metacognitive ability as the sensitivity increase for high- relative to lowconfidence trials (as a percentage of the average sensitivity; see Psychometric Analyses in the Supplemental Material). A Comparison Type × Task analysis of variance on these values indicated a main effect of task, F(1, 34) = 4.93, p = .033; metacognitive ability was greater for the orientation task than for the frequency task. However, there was no main effect of comparison type, F(1, 34) = 0.50, p = .48, and no interaction, F(1, 34) = 0.19, p = .66. Again, Bayesian analyses indicated positive evidence for the absence of a difference between within-task and across-tasks comparisons (H0)—orientation: p(H0) = .85; frequency: p(H0) = .80; pooled: p(H0) = .82. Overall, our results indicated that observers can compare confidence across tasks and access perceptual uncertainty in an abstract format, without losing information. We next investigated whether this conversion to an abstract format was costly in terms of processing time. The latency of confidence-comparison responses provided no evidence for such costs: These latencies did not differ between within-task and across-task comparisons (511 ms vs. 503 ms), t(34) = 0.50, p = .51. However, perceptual responses were slower in the across-task condition than in the within-task condition (654 ms vs. 532 ms), t(34) = 6.70, p < .001, presumably because of task-switching costs unrelated to confidence per se (e.g., changing perceptual filters and response mappings between trials), although a confidence-conversion cost
Downloaded from pss.sagepub.com at Universite de Paris 1 on July 8, 2014
de Gardelle, Mamassian
1288 taking place during the perceptual decision might have contributed to the effect as well. Finally, additional analyses confirmed that this latency difference was not due to the variation of perceptual stimuli across trials (for further information on the response time analyses, see Supplementary Methods and Supplementary Results in the Supplemental Material). However, despite the time costs associated with the across-task condition, the precision of the metacognitive evaluation and the confidencecomparison judgments (as assessed by our psychometric analyses) was unchanged in our observers. To summarize, we investigated the representational format of confidence during perceptual decisions and found that whether confidence comparisons were made within the same task (and could be based on estimates of the stimuli themselves) or across two different tasks (and necessarily involved an abstract format), the discrimination between high- and low-performance trials was equally good. Thus, observers could metacognitively access their own uncertainty on an abstract, taskindependent scale with no loss of information, although possibly with a small time cost. This common currency for metacognitive confidence judgments could be a success probability scale, or any monotonic transformation of such a scale (e.g., log-odds; see, e.g., Zhang & Maloney, 2012). An important question for future research is whether the metacognitive system can access and compare any measures of perceptual uncertainty, even across sensory modalities (Hillis, Ernst, Banks, & Landy, 2002). Author Contributions V. de Gardelle and P. Mamassian designed the study. V. de Gardelle collected and analyzed the data. V. de Gardelle and P. Mamassian wrote the manuscript.
Declaration of Conflicting Interests The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.
Funding This work was supported by Grant ANR-10-BLAN-1910 from the French Agence Nationale de la Recherche.
Supplemental Material Additional supporting information may be found at http://pss .sagepub.com/content/by/supplemental-data
References Baranski, J. V., & Petrusic, W. M. (1994). The calibration and resolution of confidence in perceptual judgments. Perception & Psychophysics, 55, 412–428. Barthelmé, S., & Mamassian, P. (2009). Evaluation of objective uncertainty in the visual system. PLoS Computational Biology, 5(9), Article e1000504. Retrieved from http:// www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2F journal.pcbi.1000504 Barthelmé, S., & Mamassian, P. (2010). Flexible mechanisms underlie the evaluation of visual confidence. Proceedings of the National Academy of Sciences, USA, 107, 20834–20839. Fleming, S. M., Dolan, R. J., & Frith, C. D. (2012). Metacognition: Computation, biology and function. Philosophical Transactions of the Royal Society B: Biological Sciences, 367, 1280–1286. Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York, NY: Wiley. Hillis, J. M., Ernst, M. O., Banks, M. S., & Landy, M. S. (2002). Combining sensory information: Mandatory fusion within, but not between, senses. Science, 298, 1627–1630. Masson, M. E. J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679–690. Peirce, C. S., & Jastrow, J. (1884). On small differences of sensation. Memoirs of the National Academy of Sciences, 3, 73–83. Pleskac, T. J., & Busemeyer, J. (2010). Two-stage dynamic signal detection: A theory of confidence, choice, and response time. Psychological Review, 117, 864–901. Yeung, N., & Summerfield, C. (2012). Metacognition in human decision making: Confidence and error monitoring. Philosophical Transactions of the Royal Society B: Biological Sciences, 367, 1310–1321. Zhang, H., & Maloney, L. T. (2012). Ubiquitous log odds: A common representation for probability and frequency distortion in perception, action, and cognition. Frontiers in Neuroscience, 6, Article 1. Retrieved from http://journal .frontiersin.org/Journal/10.3389/fnins.2012.00001/full Zylberberg, A., Barttfeld, P., & Sigman, M. (2012). The construction of confidence in a perceptual decision. Frontiers in Integrative Neuroscience, 6, Article 79. Retrieved from http:// journal.frontiersin.org/Journal/10.3389/fnint.2012.00079/full
Downloaded from pss.sagepub.com at Universite de Paris 1 on July 8, 2014