Treasure Hunt: Social Learning in the Field Markus M. Mobius Harvard University and NBER
Tuan Phan Harvard Business School
Adam Szeidl UC Berkeley and NBER November 2010
Treasure Hunt
1 / 32
Motivation Introduction
• Motivation • Theory • Double-counting • Echo Chamber
• How is information aggregated in social networks? • Information from friends and relatives may affect important political and economic decisions.
Treasure Hunt Results Conclusion
◦ Voting: Lazarsfeld et al (1944) show US voters more influenced by friends than by mass media in 1940 Presidential election.
◦ Technology adoption: farmers learn from others in Ghana (Conley-Udry, 2010) but not in western Kenya (Duflo et al, 2010).
• New media like Facebook and Twitter may increase role of networks in diffusing and aggregating information.
• This paper: use a field experiment to shed light on mechanism of information aggregation in networks.
Treasure Hunt
2 / 32
Theory Introduction
• Motivation • Theory • Double-counting • Echo Chamber Treasure Hunt
Two leading theories of social learning in networks:
• DeGroot (1974) model: agents update by averaging opinions of network neighbors.
Results
◦ “Double-count” info reaching them through multiple paths.
Conclusion
◦ DeMarzo, Vayanos and Zwiebel (2003) and Golub and Jackson (2010).
• Rational model: agents follow Bayesian updating. ◦ Tag source of information: “streams” model. ◦ Variants explored in Bala and Goyal (1998), Acemoglu, Bimpikis and Ozdaglarz (2010). Distinguishing models matters, because DeGroot model generates 1. Overconfidence due to treating repeat information as independent; 2. Persistent differences in opinion across clusters. Treasure Hunt
3 / 32
Tagged vs Untagged Updating Introduction
• Motivation • Theory • Double-counting • Echo Chamber
I1
Treasure Hunt Results
Receiver
Sender
Conclusion
I2
• Sender observes a signal and transmits it to receiver through two uninformed intermediaries.
• DeGroot: receiver perceives two signals as independent and hence double-counts them.
• Streams: source of signal is tagged and hence receiver correctly counts it once. Treasure Hunt
4 / 32
Learning with binary signals Introduction
• Motivation • Theory • Double-counting • Echo Chamber Treasure Hunt Results Conclusion
• An agent learns about binary outcome: can Markus sing? ◦ Prior belief that Markus can sing is μ. • Agent receives conditionally independent binary signals s1 , ..., sn . ◦ Each signal correct with probability q > .5. • If s denotes share of positive signals, posterior belief μ ˆ satisfies log(
Treasure Hunt
μ q μ ˆ ) = log( ) + n × (2s − 1) × log( ). 1−μ ˆ 1−μ 1−q
5 / 32
Example 1: Double-counting Introduction
• Motivation • Theory • Double-counting • Echo Chamber
I1
Treasure Hunt Results
Receiver
Sender
Conclusion
I2
Model A: De Groot model
μ q μ ˆ ) = log( ) + 2 × (2s − 1) × log( ) log( 1−μ ˆ 1−μ 1−q
Treasure Hunt
6 / 32
Example 1: Double-counting Introduction
• Motivation • Theory • Double-counting • Echo Chamber
I1
Treasure Hunt Results
Receiver
Sender
Conclusion
I2
Model B: Streams model
μ q μ ˆ ) = log( ) + (2s − 1) × log( ) log( 1−μ ˆ 1−μ 1−q
Treasure Hunt
7 / 32
Example 1: Double-counting Introduction
• Motivation • Theory • Double-counting • Echo Chamber
I1
Treasure Hunt Results
Receiver
Sender
Conclusion
I2
• DeGroot model leads to overconfidence due to treating repeat information as independent (DeMarzo et al, 2003).
• Journalistic rule: If you don’t observe an event yourself have it confirmed by at least three independent sources.
Treasure Hunt
8 / 32
Example 2: Echo Chamber Introduction
• Motivation • Theory • Double-counting • Echo Chamber
I3
Sender 2
Treasure Hunt Results Conclusion
I1 Receiver
Receiver’s island
Sender 1
I2
• Receiver is more likely to have friendship loops within his own social island (school, workplace, university). Treasure Hunt
9 / 32
Example 2: Echo Chamber Introduction
• Motivation • Theory • Double-counting • Echo Chamber
I3
Sender 2
Treasure Hunt Results Conclusion
I1 Receiver
Receiver’s island
Sender 1
I2
• De Groot model overweights own-island signals → generates differences in opinion across clusters (Golub and Jackson). Treasure Hunt
10 / 32
Introduction Treasure Hunt
• Experimental design • Screen shots Results Conclusion
Treasure Hunt
Treasure Hunt
11 / 32
Field experiment: Overview Introduction Treasure Hunt
• Experimental design • Screen shots Results Conclusion
1. Elicit social network of about 4,800 students at Harvard (sophomores, juniors and seniors).
• Online elicitation using Facebook with small financial incentives.
• See Leider, Mobius, Rosenblat and Do (2009) for details on “trivia game” technique. 2. Invite subjects to play “Treasure Hunt” online game.
• Game involves collecting information from friends about quiz questions. 3. Track subjects’ conversations and guesses over time.
Treasure Hunt
12 / 32
The Treasure Hunt Introduction Treasure Hunt
• Subjects received invitation email with link to experiment.
• Experimental design • Screen shots
• They received binary signals on some imaginary treasure.
Results Conclusion
They were told that the majority of subjects had received correct signals.
• They were encouraged to login as often as they liked during a 4-day period and update their best guess.
• Correct guessers received two movie ticket vouchers. • 843 out of 1392 eligible subjects participated (about 25 percent of all juniors and seniors).
Treasure Hunt
13 / 32
Screen Shots - P1 Introduction
Treasure Hunt
Treasure Hunt
• Experimental design • Screen shots Results Conclusion
Instructions Welcome to the Treasure Hunt! You will receive two Movie Vouchers to any AMC/Loews movie theater if you find all the correct answers to the three questions below. You have four days until noon of Saturday, May 27. After the game ends, we will send you an email with the correct answers and the winners will have the opportunity to specify a postal address to which the movie vouchers will be sent. These are the three questions:
A treasure was discovered ...
either "at the bottom of the ocean" or "on top of Mount Everest"
The treasure was found by ... either "Julius Caesar" or "Napoleon Bonaparte"
The treasure is buried ...
either "in Larry Summers' office" or "under the John Harvard statue"
On the next two pages we will suggest three answers to you and we will ask you to submit your best guess.
Our suggested answers do not have to be correct. However, for each question the majority of participants in this experiment will receive the correct suggestion. So a good idea would be to talk to other participants of this game (about half of all Juniors and Seniors are invited). While this game is running you can login as many times as you want and modify your guesses.
Next Page >>
Treasure Hunt
1
2
3
4
5
14 / 32
Screen Shots - P2 Treasure Hunt
Introduction Treasure Hunt
Suggested Answers
• Experimental design • Screen shots
We have highlighted our suggestions to you in green.
Results Conclusion
A treasure was discovered ...
at the bottom of the ocean. on top of Mount Everest.
The treasure was found by ... Julius Caesar.
Napoleon Bonaparte.
The treasure is buried ...
under the John Harvard statue.
in Larry Summers' office.
About half of all juniors and seniors received invitations. You can view the names of potential participants by simply starting to type their first name, last name or their FAS username in the field below - a list of matches will appear below that field. If no list appears you might be using an old browser and we encourage you to use a more modern browser (such as IE 6, Firefox, Camino or Safari). Search for participants: may
Maya Eden Maya Eden
<
Treasure Hunt
Next Page >>
Maya Frommer
1
May Habib
2
3
4
5
15 / 32
Screen Shots - P3 Treasure Hunt
Introduction Treasure Hunt
• Experimental design • Screen shots Results Conclusion
You can return to this page as often as you like during the next four days and update your choices if you receive new information or change your mind. If all your choices are correct, you will receive your movie tickets. You submitted your last guess on Tuesday 21st of November 04:50:37 AM which is shown below. Please modify your guess if you changed your mind on a guess. You can use the invitation email to login again as often as you like!
A treasure was discovered ...
at the bottom of the ocean.
on top of Mount Everest.
The treasure was found by ...
Julius Caesar.
Napoleon Bonaparte.
The treasure is buried ...
in Larry Summers' office.
under the John Harvard statue.
Please don't forget to move to the next page so that your new guess gets saved!
<
Treasure Hunt
Next Page >>
1
2
3
4
5
16 / 32
Screen Shots - P5 Introduction
Treasure Hunt
Treasure Hunt
• Experimental design • Screen shots Results
Thank You! We sent you an email to remind you how to login when you want to update your guess. <
1
2
3
4
5
Conclusion
Treasure Hunt
17 / 32
Introduction Treasure Hunt Results
• Basic Results • Conversations • DeGroot vs Streams Conclusion
Results
Treasure Hunt
18 / 32
Estimating equation Introduction
Consider estimating equation:
Treasure Hunt Results
• Basic Results • Conversations • DeGroot vs Streams
Guess
= α + β · own signal + γ · sum of signals of direct friends + = δ · sum of signals of indirect friends +
Conclusion
• Here signals are coded as +1/ − 1 depending on whether they are correct.
• We further decompose signals depending on number of extra paths.
◦ Helps distinguish between DeGroot and streams model.
Treasure Hunt
19 / 32
Estimation results Introduction Treasure Hunt
Variable Intercept
Results
• Basic Results • Conversations • DeGroot vs Streams
CORRECTSIGNAL INFO D1 0
Conclusion
INFO D1 1 INFO D1 2 INFO D1 3 INFO D2 0 INFO D2 1 INFO D2 2 INFO D2 3
N Treasure Hunt
0.383∗∗ (0.053) 0.972∗∗ (0.071) 0.084∗∗ (0.032) 0.104∗ (0.042) 0.091∗∗ (0.037) 0.095∗ (0.045) 0.023† (0.012) 0.050∗ (0.025) 0.184∗ (0.071) 0.155 (0.120) 2,367 20 / 32
Summary Introduction Treasure Hunt Results
• Basic Results • Conversations • DeGroot vs Streams Conclusion
• Subjects are influenced by the signals of direct and indirect friends.
• Strong evidence for information decay. ◦ We should model decay, and which signal is being transmitted.
• Higher weight on indirect friends if more common paths. Multiple stories:
◦ Number of connecting paths might proxy probability of talking in rational model.
◦ Signals of these agents might be transmitted in parallel along several paths.
Treasure Hunt
21 / 32
Tracking conversations Introduction
Treasure Hunt
Treasure Hunt
Whom did you talk to?
Results
• Basic Results • Conversations • DeGroot vs Streams Conclusion
We are curious about the number of people with whom you have discussed the game since the last time you submitted a guess: 4 Also, we would like to know who these people are (if you can remember). To make it worth your while we will pay you 25 cents for every participant you name who also names you. Participants you talked to: Maya Eden Muriel Niederle Markus Mobius Brittany Castaneda
tany
Tanya Rosenblat
<
Treasure Hunt
Next Page >>
1
2
3
4
5
22 / 32
Reporting Quality Introduction
Did agents actually report whom they talked to?
Treasure Hunt Results
• Basic Results • Conversations • DeGroot vs Streams
• On average, subjects reported at each update to have talked to 2.32 people. They named 1.74 people by name (75 percent).
Conclusion
• 83 percent of reported links were informative (person had listened to her signal already)
Treasure Hunt
23 / 32
Sample Conversation A Introduction Treasure Hunt Results
• Basic Results • Conversations • DeGroot vs Streams Conclusion
5 4
1 Agent 3
6 2
Treasure Hunt
7
24 / 32
Sample Conversation B Introduction Treasure Hunt
3
Results
• Basic Results • Conversations • DeGroot vs Streams
5
Conclusion
1
Agent
4
2 6
Treasure Hunt
25 / 32
Estimation with conversations data Introduction Treasure Hunt
Variable Intercept
Results
• Basic Results • Conversations • DeGroot vs Streams
CORRECTSIGNAL INFO D1 0
Conclusion
INFO D1 1 INFO D1 2 INFO D1 3 INFO D2 0 INFO D2 1 INFO D2 2 INFO D2 3
N Treasure Hunt
0.134∗∗ (0.042) 1.173∗∗ (0.042) 0.590∗∗ (0.052) 0.490∗∗ (0.063) 0.630∗∗ (0.078) 0.490∗∗ (0.064) 0.115∗∗ (0.023) 0.204∗∗ (0.053) 0.338∗∗ (0.083) 0.318∗∗ (0.080) 2,367 26 / 32
Summary Introduction Treasure Hunt Results
• Higher coefficients: conversations data do measure whom subjects talked to.
• Basic Results • Conversations • DeGroot vs Streams
• But decay is still strong.
Conclusion
• Each conversation path between indirect informer and agent increases the weight on this informer’s signal.
• No evidence that additional paths between agent and a direct informer increases informer’s weight.
• How to distinguish two models with decay?
Treasure Hunt
27 / 32
Information Decay Introduction Treasure Hunt
I1
Results
• Basic Results • Conversations • DeGroot vs Streams Conclusion
Receiver
Sender
I2
• Assume indirect information is transmitted with probability p < 1. ◦ DeGroot: receive 2p signals in expectation. ◦ Streams: receive 2p − p2 signals in expectation. • Hard to distinguish them when p small. Treasure Hunt
28 / 32
Pivotal sources Introduction Treasure Hunt Results
Idea: Distinguish models by identifying informers where p is likely high.
• We look at agents who do not follow their signal on Q1.
• Basic Results • Conversations • DeGroot vs Streams
• For each such agent, define pivotal informers: indirect informers she
Conclusion
• Pivotal informers are likely to have been listened to with high
listened to who have a different Q1 signal.
probability.
• Do we observe increasing coefficients for Q2 and Q3 when source is pivotal?
◦ DeGroot: we should. ◦ Streams: we shouldn’t.
Treasure Hunt
29 / 32
Estimation with pivotal informers Introduction Treasure Hunt
Variable Intercept
Results
• Basic Results • Conversations • DeGroot vs Streams
NP INFO D2 0 NP INFO D2 1
Conclusion
NP INFO D2 2 PIV INFO D2 3 PIV INFO D2 0 PIV INFO D2 1 PIV INFO D2 2 PIV INFO D2 3
N
Treasure Hunt
0.134∗∗ (0.042) 0.095∗∗ (0.022) 0.195∗∗ (0.051) 0.350∗∗ (0.083) 0.310∗∗ (0.080) 0.280∗ (0.133) 0.294∗ (0.133) 0.318† (0.163) 0.338 (0.220) 1,560
30 / 32
Introduction Treasure Hunt Results Conclusion
• Conclusion
Conclusion
Treasure Hunt
31 / 32
Conclusion Introduction Treasure Hunt Results Conclusion
• Conclusion
• Agents incorporate direct and indirect information into their decisions.
• Significant information decay. ◦ Should model how payoffs affect which information is being transmitted.
◦ May explain differences in opinion; why social learning sometimes fails; and why some cascades are big.
• Agents attach greater weight to indirect signals that were received through multiple paths.
• Some support for the “streams” model.
Treasure Hunt
32 / 32
