11
$\begingroup$

The probability that people percept may be different from the real one due to a number of factors, including the form in which their are presented, their context and biases (due to misinformation or wishful thinking).

My question if there is any research showing the subjective probability judgement of being correct ($p_\mathrm{perceived}$) as a function of actual probability of correctness ($p_\mathrm{real}$) for situation were people estimate the chance that they are right in a test/trial (e.g. in a two answers forced choice task)?

Also, I am interested mathematical models with foundation in experimental data.

Improve this question
$\endgroup$
3
  • $\begingroup$ Was my answer useful? If not, could you specify more clearly what you are looking for? $\endgroup$
    – Speldosa
    Commented Mar 14, 2012 at 10:36
  • $\begingroup$ @Speldosa Partly. I know what is signal detection theory; and actually the question was motivated by it. However, here I am not interested here not in the underlying theory but solely in belief that one is right vs real probability that one is right. But thanks for pointing to the paper (Plescak and Busemeyer) as it answers my question for a particular setting. $\endgroup$
    – Piotr Migdal
    Commented Mar 14, 2012 at 11:05
  • $\begingroup$ @PiotrMigdal note that your $p_\mathrm{real}$ is not clearly defined. Is it the proportion of questions the participant answered correctly? Or is it the probability that a participant drawn at random from some sort of self-similar ensemble answers the answer correctly? Note that most processes in psychology are not ergodic so the two view-points described before are not usually equivalent. $\endgroup$
    – Artem Kaznatcheev
    Commented Jun 10, 2012 at 1:26

2 Answers 2

Reset to default
7
$\begingroup$

You'd probably want to take a look at signal detection theory. This is a set of tools that you use to analyse how well a person (or an animal, or a machine) is able to discern trials where there is a signal present (e.g. a picture is shown that has been shown before, a tumour is present in an x-ray image, a dot on a radar is an enemy aircraft) from trials where there is no-signal present (e.g. a picture is shown that hasn't been shown before, a tumour is not present in an x-ray image, a dot on a radar is not an enemy aircraft).

Whether a person perceives a signal or not is not always easy to discern. There will always be a certain interplay between the hit rate and the false alarm rate (in that you for example, if you always answered "signal", would have a perfect hit rate, although your false alarm rate would be terrible), and based on a number of criterias (time pressure, instructions, reward structure et cetera) one can manipulate the response patterns of a certain individual.

There has been attempts to create models where accuracy, speed and certainty all are taken into concideration; see for example Plescak and Busemeyer (2010).

Improve this answer
$\endgroup$
3
$\begingroup$

Unless I'm mistaken, it sounds like you're actually interested in meta-cognition and type-2 signal detection theory (the form of SDT that Speldosa has pointed to is type-1 signal detection theory). It's used to study our ability to reflect on our own knowledge, or what we think about what we think.

This wikipedia article might get you started on meta-cognition in general.

A type-2 SDT task might involve asking participants to detect a stimulus, as would be the case in a type-1 SDT task, giving hits and false alarms. However, the difference is that, in a type-2 SDT task, after reporting whether they thought the stimulus was present or absent, participants are then asked a second question. This second question asks them how confident they were that they were right in their decision.

With this additional information, type-2 SDT tasks can then probe the extent to which participants can tell the difference between when they have the correct answer and when they have the incorrect answer, which I believe is what you are interested in.

Here's a sample paper that explores this in detail, albeit related to a different task, but should be enough to get you started:

Higham, P.A. (2007) No Special K! A signal detection framework for the strategic regulation of memory accuracy. Journal of Experimental Psychology: General, 136(1): 1-22.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.