stanford hci group
crowdsourcing a meeting of minds
designing the future of work Michael Bernstein Stanford Computer Science
20% of US jobs
[Blinder 2006]
45,000,000 workers [Horton 2013]
Combine non-expert contributions e.g., image labeling
[von Ahn and Dabbish 2005]
e.g., data collection [Deng et al. 2009]
e.g., text shortening [Bernstein et al. 2010]
Microtask crowdsourcing: distributed independent low-skilled
[Little et al. 2010]
[Bigham, Bernstein, Adar 2016]
Computationallyempowered crowd work collectives: complex interdependent expert [Kittur et al. 2013]
[Retelny et al. 2014]
Organizational Behavior and Human Decision Processes 118 (2012) 132–142
Out
of
Sight,
Out
of
Sync:
Understandin
Conflict in Distributed Teams COORDINATION NEGLECT: HOW LAY Contents lists available at SciVerse ScienceDirect Pamela J. Hinds * Diane E. Bailey THEORIES OF ORGANIZING Center for Work,Technology and Organization, Department of Management Science and Engineering, Stanford University, Stanford, California 94305-4026 Organizational Behavior and Human Decision Processes and Its The MutualKnowledgeProblem COMPLICATE COORDINATION IN
[email protected] *
[email protected] Collaboration Dispersed for Consequences ORGANIZATIONS 08-019 journal homepage: www.elsevier.com/locate/obhdp scholars and have noted and
Computationallyempowered crowd The team scaling fallacy: Underestimating the declining efficiency of larger teams work collectives: complex Who’s in Charge Here? How Team Authority Structure Shapes Team Leadership Team Familiarity, Role interdependent Experience, and Ruth Wageman Performance: expert Evidence from Abstract
practitioners expressed conThe bulkof our understanding of teamsis basedon traditional cern about one such challenge facing these teams: the teams in which all membersare collocatedand communicate prevalence and severity of conflict. Justifying their conface to face. However,geographicallydistributedteams,whose cern, reports from the field indicate that conflict is disCramton Durnell Catherine membersare not collocatedand must often communicatevia ruptive to performance in distributed teams. 22030-4444 Virginia SF5, Fairfax, Stop Mail University, are growingin Studies from the field School of Management,George Mason technology, prevalence. Field studies further indicate that geographically disare beginning to that distributed teams suggest geographically @som.gmu.edu ccramton tributed teams may experience conflict as a result of and operatedifferently experiencedifferentoutcomesthantra- two factors: The distance that separates team members ditional teams. For example, empiricalstudies suggest that and their reliance on technology to communicate and distributedteams experience high levels of conflict. These work with one another. Distance and technology mediaempiricalstudies offer rich and valuabledescriptionsof this conflict, but they do not systematicallyidentify the mecha- tion have gone unexplored in existing models of conflict a,⇑ organizations often fail to organize b c We argue that effectively because and their authors, for the al.performance 1998, DeSanctis 1996, Boudreau in teams because nisms by which conflictRousseau is engendered in distributed teams. etand individuals have lay theories about organizing that lead to coordination In this paper,we develop a theory-based most how team members were collocated part, assumed O'HaraKemske 1998,that Poole 1997,explanation Handyof 1995, "mutual is a maintaining knowledge" This paper proposes that a conflict.We do and communicating face to face. As a result, whether geographicaldistribution provokesteam-level University of neglect, North Carolina at Chapel the Hill, notion McColl Building, CB #3490, Chapel Hill,and NC 27599-3490, States We unpack of coordination neglect describe United Townsendet al. 1998). 1994, Devereaux and Johansen of geographically collaboration and dispersed central problem so by consideringthe two characteristicsthat distinguishdisthese two factors spur new antecedents of conflict is not b The Whartonspecific School, The University of Pennsylvania, 3730 Walnut Street, 566 Jon M. Huntsman Hall, Philadelphia, PA 19104, United States a model offailure todo so. Itpresents tracestheconsequences ofhow tributedteams from traditional people ones: Namely, we examine teams Geographically dispersed cognitive phenomena that underlie it. To solve the coordination norgroups is it clear conflict in distributed teams known,are c how from distant one's team members and on in of which is thirteen study geooftheseprocesses grounded being relying whomight UCLA Anderson School and Department of Psychology, 110 Westwood Plaza #D511, Los Angeles, CA 90095-1481, United States In this paper, we consider the possibe out reduced. witha commonpurpose carry interdependent problem, organizations must divide a task and then integrate the to mediate communication and collaborative work technology constitut- tasksacrosslocationsand graphically teams.Fivetypesofproblems that distance mediation bility technology Journal of Applied Psychology dispersed Copyright 2000 by the American and Psychological Inc.give rise to com-Association, time, using technology team members.Ouranalysisidentifiesantecedents to impacts 2000, VoL 85, No. 2, 273-283 0021-9010/00/$5.00 DOI: 10.I037t/0021-9010.85.2.273 components. Individuals display shortcomings that may create problems areidentified: failure tocomfailures ofmutual knowledge to conflict in distributed ing teams. We also examine how conflict that are uniqueto distributed teams. We thattheyuse face-to-face meetings much more than municate predict conflict might manifest itself over time as members of distributed andretain contextual information, unevenly municate conflict of all types (task,affective,andprocess)will be detriat both stages. First, lay theories often focus more on division of labor and andlearn Maznevski from Stamps 1997, (adapted Lipnack distributed teams how to work and communicate and understanding the communicating information, difficulty mentalto the performance of distributedteams,a resultthatis than on integration. We discuss evidence that individuals display partition such teamshas across The use of distances and use and outpaced Chudoba 2000). on traditional research teams. We also technology more effectively. investomuch infordifferences inspeedofcontrary accessto salienceofinformation, distributed teams, whose members conflict as a dynamic to determinehow Geographically and inexplicable tigate theirdynamics, of teams ourprocess understanding focus (i.e. they focus on partitioning the task more than on integration) of silence. themeaning anddifficulty mation, interpreting Journal of The Organizational Behavior these negativeimpactsover time. reside in different cities, countries, or continents, share mitigate might Article history: The competitive survival of many organizations depends on delivering projects on time and on budget. In a fielddescriptionof disin have been noted. problems of the of occurrence and severity each problem frequency J. Organiz. Behav. 31, 361–369 (2010) and component focus (i.e. they tend to focus on single components of a a number of properties commonly associated with tra(DistributedWork;DistributedTeams; VirtualTeams;Conflict) Received 16 November 2010 ofcognitive the areanalyzed. concept and Cole (1995, p. 187) collaboration, Published online inpersed WileycomInterScience Armstrong These firms facebydecisions concerning how teams to scale the sizeAttribution of work theory, teams. Larger teams can usually conceived teams. ditionally Namely, they are groups tightly interrelated set of capabilities, particularly investing to create how disareharnessed toexplain load,andfeedback dynamics Accepted 7 March 2012 of individuals that work "A decision one made in country observethese puzzles: (www.interscience.wiley.com) DOI: 10.1002/job.669 together interdependently to plete tasks more quickly, but the advantages associated with adding workers are often accompanied by John E. Mathieu Toniaaccomplish S. Heffner arelikely tointerpret failures ofmutual knowl- elicitsan unexpectedreaction highly specialized components). Second, when individuals attempt to persedpartners a members distinct social entities, and task, constitute Available online 10 April 2012 in anfrom team various disadvantages (such as the increased burden of coordinating efforts). We note several In the to a varietyreasons of factors University that characterize their team boundaries (Cohen and Bailey Pennsylvania State University of Tennessee at Chattanooga ofthese for theconsequences interpretations integrity edgeand response jointly manage integrate components of a task, they often fail to use a key mechanism for . between . escalate . other Conflicts strangely country theunrecognized the global expansion 1997, Hackman 1987). economy-including is suggested itconsequences that theeffort. In particular, why managers may focus on process gains of when they envision the ofmodem making a team larger, membersthe at kinds of problems that resisting of the marketplace the businessesgroups, that serve RecentGroup studies demonstrate integration: ongoing communication. Individuals exhibit inadequate it, the reason. Keywords: of dis- anddistributed in thesituations, andconstraints differences and why they may underestimate or underweight process losses.contexts, We document a phenomenon that we in mergers and acquisitions, rise and heightened in the to casetalk of distributed teams and that competuniquelybegin in Commentary a fewarise kilometers sitesseparated by even incan constitute "hiddenprofiles" that communication because the 'curse of knowledge' makes it difficult to take persedcollaborators Coordination neglect Gerald F. Goodwin E d u a r d o S a l a s a n d J a n i s A . C a n n o n B o w e r s itive pressures tothe reduce the time toofdevelop productsrender questionable the comprehensiveness of past modterm the team scaling fallacy—as team sizecrease increases, peopleofincreasingly underestimate number them'." the languageof 'us and situational rather than the likelihood dispositional Estimation Pennsylvania State University Naval Air the perspective of another and communicate effectively. More importantly, elsWarfare of groupCenter conflict and performance. For example, organizations increasingly are assembling teams whose labor hours required to complete projects. Using data from two laboratory experiments, and archival data for Modliterature on with cohesion and learning. This utilizes the communications attribution, consequences paper members are drawn from sites far and near. GeographPlanning fallacy Armstrong and Cole (2002) reported that conflicts in because specialists find it especially difficult to communicate with andteams erators and accelerators of these dynamics identified, of communi"mutual explore challenges knowledge" distributed face a number of uniqueto chaldistributed teams went unidentified and from projects executed at a software company, we find persistent evidence of are the team scaling fallacy and ically geographically Team scaling fallacy forbothdispersed andcollocated arebeingcation collaboration implications specialists in other areas, the general problem of communication will a* coachedand fromcollaboration unaddressed than conflicts in collocated teams. distance, lenges, including and technologylonger under dispersed The influence of teammates' shared mental models on team processes andcoping performance was tested GARY JOHNS explore a reason for its occurrence. Team size with the cost and stress of frequent travel, and dealing such empirical evidence, however, there is no discussed. Beyond often be compounded by insufficient translation. conditions. Mutual is knowledge knowledge mediated using 56 undergraduate dyads who "flew" a series of missions on a personal-computer-based flightDepartment of Management, John Molson School of Business, Concordia University, Canada ! 2012 All rights reserved. withInc. and Cole Virtual Teams; Elsevier Distributed Work; (Dispersed Collaboration; Dispersed repeated delays (Armstrong 2002). Many comprehensive theory-driven prediction and explanation
Chip Heath and Nancy Staudenmayer ABSTRACT
Bradley R. Staats
a r t i c l e
, Katherine L. Milkman , Craig R. Fox
i n f o
Abstract
a b s t r a c t
The Influence of Shared Mental Models on Team Process and Performance
Indian Software Services Robert S. Huckman Bradley R. Staats David M. Upton
[Kittur et al. 2013]
[Retelny et al. 2014]
Research in Organizational Behaviour, Volume 22, pages 153--191. 2000 by Elsevier Science Inc. ISBN: 0-7623-0641-6
Harvard University Some unintended consequences of job design
in common and that the communicating simulation. The authorsInformation both conceptually empirically distinguished parties betweenshare teammates' Exchange; Sharing; and Teams; Mutualcombat Knowledge; Information In thework Fussell 1990). theyshare (Krauss and Conm-theirknow Shared Understanding; Proximity; task- andAttribution; team-based mentalConmputer-Mediated models and indexed convergence or "sharedness" using individually 1047-7039/03/1406/0615 ORGANIZATION SCIENCE? 2003 INFORMS 1526-5455 electronic ISSN Vol. munication; Systems Dyn1amics; Cognitive Load) matrices 14, No. 6, November-December 2003, pp. 615-632 Herbert Clark and his associof communication theorist completed paired-comparisons analyzed using a network-based algorithm. The results illustrated
Colin Fisher
is referred to more ates, mutual knowledge that both shared-team- and task-based mental models related positively to subsequent team process and broadlyas integral "common andbetween considered performance. Furthermore, team processes fully mediated the ground," relationship mental modelto thecoorof actions convergence and team effectiveness. Results are discussed in terms of the (Clark role of 1996, shared Clark cognitions in and Carlson 1982, dination
Can computing orchestrate crowd collectives that achieve complex, interdependent goals?
Organizational Behavior and Human Decision Processes 118 (2012) 132–142
Contents lists available at SciVerse ScienceDirect
Organizational Behavior and Human Decision Processes journal homepage: www.elsevier.com/locate/obhdp
The MutualKnowledgeProblemand Its ConsequencesforDispersedCollaboration
The team scaling fallacy: Underestimating the declining efficiency of larger teams Bradley R. Staats a b c
a,⇑
, Katherine L. Milkman b, Craig R. Fox c
University of North Carolina at Chapel Hill, McColl Building, CB #3490, Chapel Hill, NC 27599-3490, United States The Wharton School, The University of Pennsylvania, 3730 Walnut Street, 566 Jon M. Huntsman Hall, Philadelphia, PA 19104, United States UCLA Anderson School and Department of Psychology, 110 Westwood Plaza #D511, Los Angeles, CA 90095-1481, United States
a r t i c l e
i n f o
Article history: Received 16 November 2010 Accepted 7 March 2012 Available online 10 April 2012 Keywords: Coordination neglect Estimation Planning fallacy Team scaling fallacy Team size
The competitive survival of many organizations depends on delivering projects on time and on budget. These firms face decisions concerning how to scale the size of work teams. Larger teams can usually complete tasks more quickly, but the advantages associated with adding workers are often accompanied by various disadvantages (such as the increased burden of coordinating efforts). We note several reasons why managers may focus on process gains when they envision the consequences of making a team larger, and why they may underestimate or underweight process losses. We document a phenomenon that we term the team scaling fallacy—as team size increases, people increasingly underestimate the number of labor hours required to complete projects. Using data from two laboratory experiments, and archival data from projects executed at a software company, we find persistent evidence of the team scaling fallacy and explore a reason for its occurrence. ! 2012 Elsevier Inc. All rights reserved.
Introduction Across a wide range of industries and functions, from construction to consulting and from healthcare to new product development, work is delivered to customers in the form of projects completed by teams (Edmondson & Nembhard, 2009; Ilgen, Hollenbeck, Johnson, & Jundt, 2005). Organizations turn to teams for many reasons, one of which is the increased speed with which projects can be completed when work is divided among many people. Organizations also rely increasingly on teams because knowledge is evolving so rapidly that in many settings, no single person has the depth of knowledge required to adequately serve customer needs. Teams also allow for specialization of member roles through the division of labor and can increase the knowledge resources available both within a team and through members’ external connections (Haleblian & Finkelstein, 1993; Moreland, Levine, & Wingert, 1996; Reagans & Zuckerman, 2001). In many project-based organizations that rely on teams, an important key to competitive success is accurately estimating and adhering to project budgets and deadlines. For a business that delivers projects to customers, missing promised budget and deadline estimates can tarnish a previously good reputation with patrons, resulting in lost business. Such errors in forecasting may also turn projects that should have generated profits into money-losing ventures (Heskett, Sasser, & Schlesinger, ⇑ Corresponding author. Fax: +1 919 962 6949. E-mail address:
[email protected] (B.R. Staats).
1997; Wheelwright & Clark, 1992). Despite the importance of meeting deadlines and correctly estimating costs, industry statistics suggest that many project-based organizations struggle with these activities. For example, studies in the construction, healthcare, aerospace, and information technology industries have found that anywhere from 33% to 88% of projects are delivered late and over budget (Knight, 2011; Standish, 2009; Watson, 2008). One possible explanation for these budget and deadline overruns is that process challenges arise when people work together, yet estimators do not properly account for them. Research on teams has shown that although increasing a team’s size provides the potential for many benefits (e.g., through increased specialization and expanded knowledge networks), the team’s actual productivity may suffer due to process losses (Levine & Moreland, 1998; Steiner, 1972). Increasing a team’s size can hamper its coordination, diminish its members’ motivation, and increase conflict among team members (Hare, 1952; Ingham, Levinger, Graves, & Peckham, 1974; Moreland et al., 1996). An interesting question is whether estimators are sufficiently sensitive to these problems. In this paper, we investigate whether estimators exhibit a bias that we term the team scaling fallacy—a tendency to increasingly underestimate task completion time as team size grows. We confirm the hypothesis that the team scaling fallacy plagues estimators in both the laboratory and the field. We also identify and test an important driver of this phenomenon: the tendency to focus too much on the process gains associated with increasing team size, relative to the process losses.
Abstract
Commentary
Ruth Wageman
Harvard University
Colin Fisher
Journal of Applied Psychology 2000, VoL 85, No. 2, 273-283
University of Tennessee at Chattanooga
& Bell, 2003; Lawler, Mohrman, & Ledford, 1995), the meaning of team leadership remains
elusive. Two F. factors contribute to this ambiguity. First,Eteam Gerald Goodwin d u a r dleadership o S a l a s aencompasses n d J a n i s A . aCwide annon-Bowers Pennsylvania State University
Naval Air Warfare Center
variety of activities; it can mean everything from deciding to form a team in the first place to The influence of teammates' shared mental models on team processes and performance was tested
exhorting to exertdyads morewho effort (Burke et of al.,missions 2006; Fleishman et al., 1991; Hackman usingmembers 56 undergraduate "flew" a series on a personal-computer-based flight- & combat simulation. The authors both conceptually and empirically distinguished between teammates'
Walton, 1986; Morgesonmental et al.,models 2010;and Zaccaro, & Marks, 2001). Second, team taskand team-based indexed Rittman, their convergence or "sharedness" using individually Team Familiarity, Role completed paired-comparisons matrices analyzed using a network-based algorithm. The results illustrated that both models related positively process leadership can shared-teambe enactedand by task-based multiple mental people; indeed, it would betoa subsequent tall orderteam for any oneand performance. Furthermore, team processes fully mediated the relationship between mental model Experience, and convergence and team effectiveness. Results are discussed in terms of the role of shared cognitions in individual to provide all the leadership necessary for a well-functioning team. Because team team effectiveness and the applicability of different interventions designed to achieve such convergence. Performance: Evidence from leadership involves a wide variety of behaviors enacted by multiple people, many scholars and Indian Software Services practitioners have embraced a functional Increased technology has contributed to the complexity of view many of team leadership (McGrath, 1962, p. 5), in
Kraiger, 1996). The present research was designed to empirically examine the impact that teammates' mental models have on team
tasks performed in the workplace, making it difficult for employ-
ees to complete theirteam workleadership independently. In response to or theget done, which is defined as “to do, whatever is not being process and performance in adequately a dynamic and exciting laboratory Robert S. Huckman technological advances, many organizations have adopted a team flight simulation. Bradley R.approach Staats to work. Teams viewed as being more 1993; suitableWageman for A Hackman team can be2009). defined as "a distinguishable set of two or more handled forare group needs” (Ginnett, & complex tasks because they allow members to share the workload, people who interact, dynamically, interdependently, and adapDavid M. Upton monitor the work behaviors of other members, and to develop and tively toward functions a common that and promote valued goal/objective/mission, who Some scholars have attempted specify the key leadership team
contribute expertise on subtasks. An abundance of research has have each been assigned specific roles or functions to perform, and been conductedeffectiveness. on the factors that contribute Hackman to high team perforhave posited a limitedthat life-span membership"is(Salas Specifically, and Wagemanwho (2005) teamofeffectiveness a et al., 1992, p. mance (for reviews, see Gist, Locke, & Taylor, 1987; Salas, 4). Research has identified numerous factors that affect teams and Dickinson, Converse, & Tannenbaum, 1992). One variable that (a) the haslevel offered modelsmembers of team functioning (Guzzo & Dickson, joint function of three performance processes: of several effort group collectively has recently received much theoretical attention concerns the in1996). Although these different models vary in details, they all fluence of team members' mental models on team-related proshare an input-process-outcome (I-P-O) framework. Inputs to cesses and behaviors (Klimoski & Mohammed, 1994; Kraiger & such models are conditions that exist prior to a performance Wenzel, 1997; Rentsch, Heffner, & Duffy, 1994; Stout, Salas, & episode and may include member, team, and organizational char acteristics. Performance episodes are defined as distinguishable periods of time over which performance accrues and feedback is available. Processes describe how team inputs are transformed into John E. Mathieu and Gerald F. Goodwin, Department of Psychology, outputs, Outcomes are results and by-products of team activity that Pennsylvania State University, University Park; Tonia S. Heffner, Departare valued by one or more constituencies. Hackman (1990) idenment of Psychology, University of Tennessee at Chattanooga; Eduardo tified three primary types of outcomes: (a) performance-including Salas and Janis A. Cannon-Bowers, Training Systems Division, Naval Air quality and quantity, (b) team longevity, and (c) members' affecWarfare Center, Orlando, Florida. five reactions. Although we recognize the importance of all three Tonia S. Heffner is now at the Arm), Research Institute, Washington, types, for our purposes we will concentrate on performance DC. Gerald F. Goodwin is now at the American Institutes for Research, outcomes. Washington, DC. Eduardo Salas is now at the Department of Psychology, University of Cenlral Florida, Orlando, Florida. Empirical examinations of I-P-O models have demonstrated The views, opinions, and findings contained in this article are those of their utility (e.g., Campion, Medsker, & Higgs, 1993; Gladstein, the authors and should not be construed as an official position, policy, or 1984; Guzzo & Dickson, 1996). However, the large number of decision of their respective affiliations. We thank Dana Born, Gwen Fisher, factors that influence outcomes has prohibited a comprehensive Kirsten Hobson, Kara Ivory, Mike Trip, and Nikki Windefelder, who were examination of the model. Many variables that have been proposed instrumental in the running of the teams and data coding portions of this to influence team processes and thereby team performance have work. yet to receive much empirical examination. Included among these Correspondence concerning this article should be addressed to John E. are members' knowledge and its organizational structure. This Mathieu, who is now at the School of Business Administration, Departoversight has occurred despite acknowledgement of the imporment of Management, University of Connecticut, 368 Fairfield Road, Copyright © 2007, 2008 by Robert S. Huckman, Bradley R. Staats, and David M. Upton tance of knowledge organization for individual and team perfor6-41MG, Storrs, Connecticut 06269-2041. Electronic mail may be sent to jmathieu @for sba.uconn.edu. mance (Cannon-Bowers & Salas, 1990; Cannon-Bowers, Salas, & Working papers are in draft form. This working paper is distributed purposes of comment and 273
Department of Management, John Molson School of Business, Concordia University, Canada ORGANIZATION SCIENCE, C) 2001 INFORMS Vol. 12, No. 3, May-June2001, pp. 346-371
While teams accomplish much of the work in modern organizations (Hills, 2007; Kozlowski John E. Mathieu Tonia S. Heffner Pennsylvania State University
Some unintended consequences of job design GARY JOHNS*
Copyright 2000 by the American Psychological Association, Inc. 0021-9010/00/$5.00 DOI: 10.I037t/0021-9010.85.2.273
Boston University
The Influence of Shared Mental Models on Team Process and Performance
discussion only. It may not be reproduced without permission of the copyright holder. Copies of working papers are available from the author.
Rousseau 1996, Boudreau et al. 1998, DeSanctis and Poole 1997, Handy 1995, Kemske 1998, O'Hara"mutual is a knowledge" Thispaperproposesthatmaintaining of geographically collaboration and Devereauxand Johansen1994, Townsendet al. 1998). dispersed central problem a model offailure todo so. Itpresents tracestheconsequences Geographicallydispersedteamsare groupsof people in studyofthirteen geo- witha commonpurposewho carryout interdependent oftheseprocesseswhichis grounded constitut- tasksacrosslocationsand time,usingtechnology graphically dispersed teams.Fivetypesofproblems to comareidentified: failure tocomofmutual knowledge ingfailures meetings municatemuchmorethantheyuse face-to-face distributed andretain contextual information, unevenly municate and understanding the (adaptedfromLipnackand Stamps1997,and Maznevski communicating information, difficulty differences inspeedofaccesstoinfor- and Chudoba2000). The use of suchteamshas outpaced salienceofinformation, of theirdynamics,and inexplicable our understanding of silence.The themeaning anddifficulty mation, interpreting in the problemshave been noted.In a fielddescriptionof disof each problem of occurrence and severity frequency ofcognitive persedcollaboration, theconcept teamsareanalyzed. Attribution theory, and Cole (1995, p. 187) Armstrong areharnessed toexplainhowdis- observethesepuzzles: "A decisionmade in one country load,andfeedback dynamics arelikelytointerpret failures ofmutual knowl- elicitsan unexpectedreactionfromteammembersin anpersedpartners oftheseinterpretations fortheintegrity edgeandtheconsequences othercountry. . . Conflictsescalate strangelybetween it is suggested thatunrecognized of theeffort. In particular, groups,resistingreason. Group membersat of dis- distributed in thesituations, andconstraints differences contexts, beginto talkin constitute "hiddenprofiles" thatcan in- sitesseparatedby even a fewkilometers persedcollaborators ratherthansituational thelanguageof 'us and them'." creasethe likelihoodof dispositional forcohesionandlearning. Modliterature on withconsequences This paper utilizesthe communications attribution, areidentified, and eratorsand accelerators of thesedynamics "mutualknowledge"to explorechallengesof communiforbothdispersed andcollocated are cationand collaborationunderdispersedand technologycollaboration implications discussed. mediatedconditions.Mutual knowledgeis knowledge (Dispersed Collaboration; Dispersed Teams; Distributed Work; Virtual that the communicatingpartiesshare in common and Teams; Mutual Knowledge; InformationExchange; InformationSharing; knowtheyshare(Krauss and Fussell 1990). In thework ConmShared Understanding;Attribution;Proximity;Conmputer-Mediated munication; SystemsDyn1amics;Cognitive Load) of communication theoristHerbertClarkand his associates, mutualknowledgeis referredto more broadlyas "commonground,"and consideredintegralto thecoordinationof actions(Clark 1996,Clarkand Carlson1982, Clark and Marshall 1981). But membersof dispersed The organizationof groupworkand the means of comteamsdo notstandon commonground.Indeed,theusage municationto supportit are changing.Developmentsin "commonground"suggestshow deeplyengrainedphysicommunicationand collaborative technologies have made it feasible for groups to work togetherdespite cal copresenceand sharedphysicalsettingmaybe to esand affiliation. In 1990, tablishingsharedunderstanding havebeen physicaldispersionof members.Organizations withgeographicallydispersedwork Krauss and Fussell raisedthequestionof how theuse of quick to experiment technologiesto supportcooperative teamsto takeadvantageof interorganizational and inter- newcommunications workwouldinteract muwiththeproblemofestablishing nationalopportunities and maximizetheuse of scarceresources.This is likelyto be an increasingly prevalentand tual knowledge.This paper takes up thatquestionand formof workin the years ahead (Arthurand important adds to it two additionalones: "How does geographic
Journal of Organizational Behavior J. Organiz. Behav. 31, 361–369 (2010) Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/job.669
Who’s in Charge Here? How Team Authority Structure Shapes Team Leadership
0749-5978/$ - see front matter ! 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.obhdp.2012.03.002
08-019
CatherineDurnellCramton
Mail Stop SF5, Fairfax,Virginia22030-4444 School of Management,George Mason University, @som.gmu.edu ccramton
a b s t r a c t
1047-7039/0 1/1203/03461$05.00 1526-5455electronicISSN
COORDINATION NEGLECT: HOW LAY THEORIES OF ORGANIZING COMPLICATE COORDINATION IN ORGANIZATIONS By intent or default, all jobs have a design that constitutes a context for their incumbents, and that design is embedded in a larger work context. The purpose of this article is to examine the unintended Chip Heath and Nancy Staudenmayer and sometimes negative consequences of job designs and their related contexts. Several themes will emerge in what ABSTRACT follows. First, the larger context in which jobs are embedded can either shape or countervail intended job design effects. Second, many job characteristics have a paradoxical doubleWe argue that organizations often fail to organize effectively because individuals lay theories about organizing that leadsame to coordination edged have quality. For example, the autonomy that leads some academics to produce creative neglect, We unpack the notion of coordination neglect and describe scientific enables others to produce crackpot ideas in the name of academic freedom. specific cognitive breakthroughs phenomena that underlie it. To solve the coordination problem, organizations must divide design a task and integrate the Third, question forthen what purpose? is important to answer. Thus, job designs that support of the Out of Job Out Sight, Sync: Understanding components. Individuals display shortcomings that may create problems in-role might support creativity or learning or citizenship or ethical behavior or Conflict in Distributed Teams not at high both stages. First, performance lay theories often focus more on division of labor than on integration. We discuss evidence that individuals display partition employee health. Finally, the identity of job incumbents is an important but seldom examined factor in Pamela J. Hinds Diane Bailey focus (i.e. they focus on partitioning the E.task more than on integration) and component focus (i.e. theyof tendjob to focus on single components of a the consequences design. This content downloaded from 171.64.70.138 on Tue, 21 Oct 2014 14:48:35 PM All use subject to JSTOR Terms and Conditions
* Center for Work,Technology and Organization, Department of Management Science and Engineering, Stanford University, Stanford, California 94305-4026
*
[email protected] tightly interrelated
[email protected] of capabilities, particularly by investing to create highly specialized components). Second, when individuals attempt to integrate to use have a key for scholarsfail and practitioners notedmechanism and expressed conAbstract components of a task, they often The bulkof our understanding of teamsis basedon traditional cern about one such challenge facing these teams: the integration: ongoing communication. Individuals exhibit inadequate teams in which all members are collocated and communicate prevalence and severity of conflict. Justifying their conface to face. However,geographicallydistributedteams,whose field indicate that conflict to is disreports from themakes communication because the 'curse ofcern, knowledge' it difficult take membersare not collocatedand must often communicatevia ruptive to performance in distributed teams. are growingin of Studiesand from the field technology, prevalence. the perspective another communicate effectively. importantly, Field studies further indicateMore that geographically disare beginningto suggestthatgeographicallydistributedteams tributed teams may experience conflict as a result of andexperiencedifferent trabecause specialists findoutcomes it than especially difficult to thatcommunicate with operatedifferently two factors: The distance separates team members ditional teams. For example, empiricalstudies suggest that and their reliance on of to communicate and technology specialists in other areas, theThese general problem communication will distributedteams experience of conflict. high levels work with one another. Distance and technology mediaempiricalstudies offer rich and valuabledescriptionsof this often bethey compounded byidentify insufficient tion have gone unexplored in existing models of conflict do not systematically the mecha- translation. conflict, but
The Fragility of Meaningfulness In a comprehensive test of the Job Characteristics Model, Johns, Xie, and Fang (1992) found that experienced meaningfulness was a particularly robust mediator of the connection between all core job characteristics and work outcomes, a finding subsequently confirmed in a meta-analysis by Humphrey, Nahrgang, and Morgeson (2007). Given the potent affective and motivational properties of meaningfulness, it should play a key role in the design of jobs. However, research has shown that the contextual cues that stimulate meaningfulness can be rather subtle, and thus overlooked, that other aspects of job design or job context can damage inherent meaningfulness, and that people can extract meaningfulness from cues rather far removed from the intended design of the job. On the surface, soliciting scholarship money for deserving students or detecting cancer would seem to be inherently meaningful tasks. However, as Grant and Parker (2009) imply, such jobs, as designed, often inadvertently isolate incumbents from beneficiaries in a way that attenuates empathy and motivation to help. Thus, in a field experiment, Grant and colleagues (Grant, Campbell, Chen, Cottone,
nisms by which conflict is engenderedin distributedteams. and performance in teams because their authors, for the In this paper,we develop a theory-basedexplanationof how most part, assumed that team members were collocated and communicating face to face. As a result, whether geographicaldistributionprovokesteam-levelconflict.We do so by considering the two characteristics that distinguish disthese factors153--191. spur new antecedents of conflict is not Research in Organizational Behaviour, Volume 22,two pages tributedteams from traditionalones: Namely, we examine known, nor is it clear how conflict in distributed teams 2000 Elsevier Science Inc. howby distantfrom one's team membersand relying on being might be reduced. In this paper, we consider the possito mediatecommunicationand collaborativework technology ISBN: 0-7623-0641-6 bility that distance and technology mediation give rise impactsteam members.Ouranalysisidentifiesantecedentsto to conflict in distributed teams. We also examine how conflict that are uniqueto distributedteams. We predictthat conflict might manifest itself over time as members of conflictof all types (task,affective,andprocess)will be detri- 153 distributed teams learn how to work and communicate mentalto the performanceof distributedteams,a resultthatis across distances and use technology more effectively. contraryto muchresearchon traditionalteams.We also invesGeographically distributed teams, whose members tigate conflict as a dynamicprocess to determinehow teams reside in different cities, countries, or continents, share mightmitigatethese negativeimpactsover time. a number of properties commonly associated with tra(DistributedWork;DistributedTeams; VirtualTeams;Conflict) ditionally conceived teams. Namely, they are groups of individuals that work together interdependently to accomplish a task, constitute distinct social entities, and In response to a variety of factors that characterize jointly manage their team boundaries (Cohen and Bailey the modem economy-including the global expansion 1997, Hackman 1987). of the marketplace and the businesses that serve it, the Recent studies demonstrate the kinds of problems that rise in mergers and acquisitions, and heightened competarise uniquely in the case of distributed teams and that itive pressures to reduce the time to develop productsrender questionable the comprehensiveness of past models of group conflict and performance. For example, organizations increasingly are assembling teams whose members are drawn from sites far and near. GeographArmstrong and Cole (2002) reported that conflicts in ically distributed teams face a number of unique chalgeographically distributed teams went unidentified and unaddressed longer than conflicts in collocated teams. lenges, including being coached from a distance, coping with the cost and stress of frequent travel, and dealing Beyond such empirical evidence, however, there is no with repeated delays (Armstrong and Cole 2002). Many comprehensive theory-driven prediction and explanation 1047-7039/03/1406/0615 1526-5455electronicISSN
ORGANIZATIONSCIENCE? 2003 INFORMS Vol. 14, No. 6, November-December2003, pp. 615-632
* Correspondence to: Gary Johns, Department of Management, John Molson School of Business, Concordia University, 1455 de Maisonneuve Blvd. West, Montreal, Quebec, H3G 1M8, Canada. E-mail:
[email protected] This content downloaded from 171.64.70.138 on Tue, 21 Oct 2014 14:52:55 PM All use subject to JSTOR Terms and Conditions
Copyright # 2010 John Wiley & Sons, Ltd.
Received 13 September 2009 Accepted 18 September 2009
expert crowdsourcing
with flash teams Daniela Retelny, Sébastien Robaszkiewicz, Alexandra To, Walter Lasecki, Jay Patel, Negar Rahmati, Tulsee Doshi, Melissa Valentine,
Michael Bernstein. UIST 2014. Best paper award.
Computationallyempowered crowd work collectives: expert complex interdependent [Kittur et al. 2013]
[Retelny et al. 2014]
Computationallyempowered crowd work collectives: expert complex interdependent [Kittur et al. 2013]
[Retelny et al. 2014]
Could we crowdsource… The design process, starting from a napkin sketch, in one day?
crowds of experts Mechanical Turk
Upwork
microtask worker microtask worker microtask worker microtask worker microtask worker
programmer designer video editor musician statistician
organizational behavior Self-managed teams are inefficient, riddled with frustrated members, and poorly coordinated. [Bunderson and Boumgarden 2010]
organizational behavior Lightweight team scaffolds significantly outperform pipelined and self-managed efforts. [Valentine and Edmonson 2012]
flash teams Computationally-guided teams of crowd experts supported by lightweight, reproducible and
scalable team structures. Input
Flash Team
design
Output
predefined workflows Only limited kinds of work can be achieved through predefined workflows… At only a small team scale… Flash teams explore what’s possible with these constraints.
Low-fi Mockup UI Designer
Revised Mockup UI Designer
Input: napkin sketch Output: low-fi mockups Goal: 1hr
Input: low-fi mockups Output: revised low-fi mockups Goal: 2hrs
Heuristic Evaluation UX Researcher
High-fi Prototype Developer
Input: low-fi mockups Output: heuristic evaluation Goal: 1hr
Input: low-fi mockups, HE Output: high-fi prototype Goal: 4hrs
0h Low-fi
5h
10h
UI
(revised) Low-fi Heuristic evaluation
low-fi mockup
heuristic evaluation
UX
Developer
User testing
Hi-fi prototype development
napkin sketch
20h
15h
(revised) low-fi mockup
(revised) Hi-fi prototype
user study hi-fi prototype report
(revised) hi-fi prototype
running a flash team 1. Introspect on the team composition & convene experts from the crowd 2. Convey the team through the workflow, shepherding files between tasks and sharing schedule updates
Upwork
programmer designer video editor musician statistician
computational affordances
of flash teams Modularity Elasticity Pipelining Planner
Scale Grow + shrink Optimize Create on-demand
modularity
replicate team structures at scale
design
design
design
modularity
replicate team structures at scale
design
design
elasticity
growth on-demand 0h
5h
10h
Development v1
15h
Is it ok to leave DRI on this slide and next slide even though I don’t mention it when I speak?
20h
25h
Development v2
30h
elasticity
growth on-demand 0h
5h
hired by default elastic worker elastic worker
10h
15h
20h
Development v1
Elasticity enables growth by dynamically adding: Extra workers to complete job on time Workers with specialized skills
25h
Development v2
30h
pipelining
pass along incomplete results 0h
5h
Lo-fi v1
10h
15h
20h
25h
Low-fidelity prototype v2 Heuristic evaluation Development v1
Development v2 User testing
can accept in-progress input can stream in-progress output
30h
pipelining
pass along incomplete results 0h Lo-fi v1
5h
10h
15h
Low-fidelity prototype v2 Heuristic evaluation Development v1
Development v2
User testing
20h
25h
30h
creation by request “I have a napkin sketch of a design, and I’d like an animation describing the idea.”
creation by request Synthetic team created from compatible tasks found within previous teams.
creation by request Translate all known tasks into strips action planning language and utilize boolean satisfiability solvers.
foundry Web platform that allows:
Requesters to author flash teams
Team members to track the progress of tasks
authoring in foundry
36
authoring in foundry
37
authoring in foundry
authoring in foundry
39
authoring in foundry
40
foundry as manager
foundry as manager
42
foundry as manager
43
foundry as manager
44
foundry as manager
45
foundry as manager
46
foundry in sum Author structured, modular representations of flash teams Grow, shrink, pipeline and recombine the flash teams Recruit from Upwork Maintain situational awareness as the team works
flash team examples Recruited from paid crowd marketplace Upwork Three team types:
Napkin sketch (design & web programming) Animation (video making) MOOC (online education)
user-tested hi-fi prototypes in one day Completion time
Team size
Total cost
Emotion tracking
31:30
3
$744.48
Event bullet board
18:00
5
$1270.28
Social meetups
23:10
5
$1200.97
Design Goal
animation team overview
Objective: explore how flash teams can support creative outputs and non-engineering domains 0h
5h
Script
,
10h
Storyboard Character design Background design
,
15h
20h
Director Scriptwriter
Animator Illustrator
24h Sound engineer
Animation Music
Voiceover Mix Editing
script idea
storyboard script characters
backgrounds
video track audio track
animated video
animation team overview
Objective: explore how flash teams can support creative outputs and non-engineering domains 0h
5h
Script
,
10h
Storyboard Character design Background design
,
15h
20h
Director Scriptwriter
Animator Illustrator
24h Sound engineer
Animation Music
Voiceover Mix Editing
script idea
storyboard script characters
backgrounds
video track audio track
animated video
accenture’s flash teams Project #1: Marketing video 177 work hours
Project #2: Mobile application 54 work hours
Project #3: Business dev. video 132 work hours
accenture’s flash teams Result: 1/8th — 1/6th cost of agency estimate for equivalent quality
are flash teams effective? field experiment
Do flash teams complete tasks equally effectively but in less time? Controlled experiment: 22 experts across six napkin sketch teams (UI design, UX research, web dev) Flash teams vs. self-managed teams
conditions
flash teams vs. self-managed teams Flash teams: full Foundry with flash team workflow
Control teams (self-managed): full Foundry with just one 13hr block
field experiment
napkin sketch design team Task: party planning mobile web application Input: Napkin sketch Requested time limit: 13 hours Measured: total number of work hours across team
flash teams:
50% fewer work hours Flash teams (mean 13hr2min) are significantly faster than self-managed teams (mean 23hr47min), p=0.05 The slowest flash team finished in fewer hours than the fastest team in the control condition
Flash teams introduce computational infrastructure for crowdsourcing diverse, ondemand teams of experts.
but! Only limited kinds of work can be achieved through predefined workflows… At only a small team scale… What about goals that are more complex and evolving, that cannot be predefined?
we are dynamo:
collective action with crowd workers Niloufar Salehi, Lilly Irani, Michael Bernstein, Ali Alkhatib, Eva Ogbe, Kristy Milliland, Clickhappier. CHI ’15. Best paper honorable mention.
crowd work:
promise or peril? If flash teams are a new form of work collective, we also need new forms of worker counterbalance. Crowdsourcing is a populist form of information work, but the technical infrastructure actively disempowers workers.
[Irani and Silberman ’13]
“If by ‘union’ you mean a ‘labor union’, I would not feel comfortable taking part. It runs against my grain because I am an individualist. I do not want to feel forced to go along with the ‘majority thinking’ of the leaders within a labor union. I have never been a member of a union and hope to continue along my merry way. I consider myself self-employed...not working for anyone in particular.” http://turkernation.com/archive/index.php/t-18874.html
crowd collective action One year of ethnography with crowd workers, understanding and building relationships. This work led to the creation of:
1) Idea 2) Vote 3) Discuss 4) Mobilize
structured human behaviors These kinds of publics require special action to preserve their kinetic energy. For example: debates with deadlines
act and undo
This labor could not have been written into software:
it consists of human scripts undertaken by a trusted party.
ethical research guidelines First campaign: curb poor academic research practices Spawned when an IRB-approved economics researcher ran an experiment to inject false information into Turkopticon
Wiki-written guidelines covering fair pay, rejection, and
IRB arbitration for poor requesters
http://guidelines.wearedynamo.org — 250 signatures so far
humanization in the media
crowd research: unlocking the gates
to the ivory tower ongoing work with Rajan Vaish, Snehal (Neil) Gaikwad, Adam Ginzberg, Imanol Ibarra, Geza Kovacs, Ranjay Krishna, Durim Morina, Catherine Mullings, Camelia Simoiu, Andreas Veit, Michael Wilber, Sharon Zhou, Serge Belongie, Sharad Goel, James Davis, and hundreds of others
setting our sights higher Must we restrict ourselves to research problems that are solvable alone or in small groups? Must we deny access to motivated aspiring researchers?
Could people around the world work together to… Build a new crowdsourcing platform? Design and run hundreds of parallel experiments? Develop computer vision algorithms?
well-stated problems Polymath project [Gowers and Tao]
[Cranshaw and Kittur 2011]
FoldIt [Cooper et al. 2010]
open-ended, messy research Could we invite anyone from around the world to participate, and crowdsource large-scale, open-ended research problems? These are problems for which a felicitous approach to finding the solution cannot be known in advance.
challenge: coordination My Stanford group is eight students. And that keeps me pretty busy. Google managers are asked to have no more than seven reports. So how could we possibly run a research team of hundreds?
challenge: credit Eventually, we need to decide on an author order, and participants will need recommendation letters. How do we measure impact? Assumption: advisor goes last in the author order, but other authors should be ranked by contribution to the project
1097 signups, predominantly from India and United States 27% female, median age 21, average team 3 people 73% undergraduate, 22% masters, 4% PhD, 1% high school 71% engineering-oriented areas of study
three parallel projects HCI Michael Bernstein, Stanford Building a new crowd marketplace
three parallel projects Computer vision James Davis, UCSC
Serge Belongie, Cornell Hybrid crowd-computer vision algorithms
three parallel projects Data science Sharad Goel, Stanford Hundreds of experiments testing the wisdom of the crowd
coordination strategy Exploration during the week, reset to argmax each weekend Saturdays: team meeting + milestone opens
Thursdays: milestone closes
Fridays: peer feedback and ranking + staff collation
coordination strategy Divergence
Convergence
Every interested contributor submits a milestone, then peers upvote high-quality submissions
Empower active community members to create temporary ad-hoc teams
Tools: Telescope (Reddit clone), Wiki
Tools: Google Hangout, Slack, Google Presentations
intuition: transform credit into a network problem Each participant allocates 100 points across other contributors. This produces a credit network, but some groups of participants rally a few friends to vote for them and artificially increase their influence.
pagerank-based credit Approach: run a modified PageRank algorithm over the network Informally, PageRank identifies the universally-respected contributors, then weighs their votes more heavily. This process iterates until convergence.
milestone self-assignment
engineering
prototypes and storyboards
data analysis
brainstorming and writing
Andrew Ng, Stanford and Baidu Research
Peter Norvig, Google
Anant Agarwal, MIT and EdX
longevity 200 180 160 140 120 100 80 60 40 20 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
number of contributors
People reading and writing on Slack
week Reading
Writing
paper writing
a.k.a. “Michael loves editing intros.”
Crowd Advisor
pagerank
pagerank
pagerank
the future of work is… Complex and interdependent Advocating for pro-social outcomes Solving open-ended challenges
the future of work is… Complex and interdependent Advocating for pro-social outcomes Solving open-ended challenges Thanks to the NSF, Accenture Tech. Labs, Stanford Cyber, Brown Institute, Precourt, HPDTRP
