Factors that Create and Mitigate Confirmation Bias in Judgments of Handwriting Evidence Jeff Kukucka (Towson University) & Saul M. Kassin (John Jay College)

Abstract Three studies explored moderators of forensic confirmation bias (Kassin, Dror, & Kukucka, 2013). In Study 1, knowledge of a confession increased mistaken inculpatory judgments. In Studies 2 and 3, evidence lineups mitigated bias and influenced judgments in the same way as eyewitness lineups.

Study 1

Study 2

Design (N = 115 mTurk users)

Study 3 Design (N = 230 mTurk users)

Design (N = 473 mTurk users)

2 (Context: Confession vs. Denial) x 2 (Handwriting Similarity: High vs Low) x 2 (Time: First vs. Second Judgment) mixed design

Procedure Procedure

Introduction

1) Ps read a summary of a bank robbery in which the suspect either denied guilt or confessed (and recanted).

1) Ps read a summary in which the suspect denied guilt, confessed (and recanted), or provided an alibi.

Forensic science errors are alarmingly common in known wrongful conviction cases (e.g., Garrett & Neufeld, 2009). A growing literature suggests that some such errors may result from forensic confirmation bias -- i.e., the tendency to evaluate evidence in ways that validate one’s pre-existing beliefs (Kassin, Dror, & Kukucka, 2013).

2) Ps compared the culprit’s robbery note against the suspect’s (non-matching) Miranda waiver -- which was either high or low in pilot-tested similarity...

2) Ps were shown either an evidence showup (as in Study 1) or an evidence lineup, which either did or did not include a matching (i.e., target) robbery note...

Critics believe that giving examiners only one pair of samples may imply guilt (Whitman & Koppl, 2010), and that they should instead compare one sample against an array of others to decide which, if any, is a match (i.e., an evidence lineup; Wells, Wilford, & Smalarz, 2013). In the only prior study of evidence lineups, Miller (1987) found that individuals who saw only two samples were more likely to make a false ID than those who saw a lineup. Studies 2 and 3 aimed to give a more complete assessment of how evidence lineups impact judgment.

High-similarity pair: Target-Absent Lineup

3) After providing judgments, Ps were asked to revisit the summary and provide a second round of judgments.

• Choosing (i.e., identifying any sample as a match) • Judgment accuracy (i.e., a correct ID or rejection) • Judgment confidence (1-10)

Results Denial

Confession

5 4 3 2 1 0

50 40 30 20

High Similarity

70 60

Simultaneous

10

Alibi

80

Denial

Confession

60 40 20 0

TP Showups

TA Lineups

50 40 30 20 10 0

High Similarity

• Context influenced similarity ratings, d = 0.36.  Effect of Context for low similarity pairs (d = 0.72), but not for high similarity pairs (d = 0.17). • Marginal effect of Context on % match at Time 2 (φ = .19) but not at Time 1 (φ = .10). No moderation by Similarity. • No significant moderation by Time, or NFC (1-β = .24).

Target-Absent

Target-Present

• No effects of Context on choosing (φ = .07) or accuracy (φ = .00). No significant interactions involving Context. • No main effect of Format on accuracy (φ = .11).  As predicted, Format influenced accuracy for TA lineups (φ = .32) but not for TP lineups (φ = .16).

The findings of Study 1 reiterate the value of isolating forensic examiners from potentially biasing information (i.e., sequential unmasking protocols; Krane et al., 2008). Notably, participants undervalued the influence of case facts on their judgments (Kunda, 1990; Pronin, 2007).

TP Lineups

• Lineups led to more choosing than showups (φ = .28). Low Similarity

Sequential

100

TA Showups

0

Low Similarity

Results

Discussion

Results % Accurate Judgments

• Similarity rating (1-10) • Match judgment (yes/no/CBD) + confidence (1-10) • Self-reported influence by HW and case facts (1-10)

Target-Present Lineup

Dependent Measures

Dependent Measures

6

2) Ps were shown either a simultaneous or sequential evidence lineup. (DVs were the same as in Study 2.)

Overall

% Match Judgments (Time 2)

Study 1 aimed to replicate and extend these findings by testing three moderators – i.e., the similarity of the two samples, revisiting the evidence, and dispositional Need for Cognition (NFC; Cacioppo & Petty, 1982).

Low-similarity pair:

Similarity Rating (Across Time)

For example, Kukucka and Kassin (2014) found that individuals who knew of a defendant’s confession saw more similarity between handwriting samples from the defendant and culprit, and more often misjudged them as a “match,” than those unaware of the confession.

1) Ps read a summary in which the suspect either denied guilt or confessed (and recanted).

% Accurate Judgments

Procedure

Empirical studies have shown that confirmation bias can affect judgments of fingerprint evidence (e.g., Dror & Charlton, 2006), polygraph charts (Elaad, Ginton, & BenShakhar, 1994), handwriting samples (Kukucka & Kassin, 2014), burn patterns in arson cases (Bieber, 2012), and DNA mixtures (Dror & Hampikian, 2011), among others.

2 (Context: Confession vs. Denial) x 2 (Lineup Format: Simultaneous vs. Sequential)

3 (Context: Confession, Alibi, or Denial) x 2 (Presentation Format: Showup vs. Lineup) x 2 (Target: Present vs. Absent) between-subjects

 Confession increased choosing for showups (V = .27) but not for lineups (V = .08). No Alibi effects. • Showups produced greater overall accuracy (φ = .30).  Confession influenced accuracy for target-present (V = .34) and target-absent (V = .31) showups.  No Context effect on accuracy for TP lineups (V = .05). No Confession effect for TA lineups (V = .29).

In Study 2, simultaneous evidence lineups mitigated bias, but also reduced accuracy relative to showups. The latter finding is consistent with the eyewitness literature (e.g., Steblay et al., 2003). Study 3 extended the sequential superiority effect (Lindsay & Wells, 1985; Steblay et al., 2011) to judgments of handwriting evidence. Taken together, Studies 2 and 3 support the value of importing eyewitness identification procedures into the forensic sciences as a way to combat error due to bias.