A Study of Users Cooperation in CoScripter Andréia L. Sampaio
Clarisse S. de Souza
Dep. de Informática, PUC-Rio
Dep. de Informática, PUC-Rio
Mq. de São Vicente 225, Gávea
[email protected] Rio de Janeiro-RJ, Brazil
[email protected]
Críston P. de Souza Dep. de Informática, PUC-Rio
Simone D.J. Barbosa
[email protected]
Dep. de Informática, PUC-Rio
[email protected]
Abstract This paper presents a study investigating cooperation in end-user programming through web scripting with CoScripter.
Keywords cooperation, web scripting, CoScripter
ACM Classification Keywords H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.
Introduction End-user programming has attracted significant attenCopyright is held by the author/owner(s). CHI 2009, April 4 – April 9, 2009, Boston, MA, USA ACM 978-1-60558-247-4/08/04.
tion in the last years, due to the motivation for users to have a unique experience and tailor software to their frequent or changing needs [3]. Social networks, in turn, have emerged as engaging environments where users share their knowledge and skills with others. Enduser programming on the Web aims to join the strengths of both EUP and social networks to increase and improve access to technology, especially by the socalled naïve users. IBM’s CoScripter [2,7] is a web scripting system freely available as a plug-in to the Firefox web browser. It promotes end-user programming via macro recording, representing scripts in a pseudo-natural language to make it easier for nonprogrammers to edit scripts. In addition, CoScripter’s designers clearly promote the cooperation among users through a public catalog of shared scripts and discussion forums. However, until now there has been little research on how cooperation has been taking place in CoScripter. This paper investigates the following questions: What signs of user cooperation can we find? How is the users’ community responding to the designers’ intent regarding social collaboration and cooperation? We present three independent studies about users’ cooperation via CoScripter, and we conclude with a triangulation of their main results.
2
Semiotic Inspection
2) We encourage users to pick scripts in the Wiki.
The Semiotic Inspection Method (SIM) [4] is a Semiotic Engineering [5] evaluation method that allows us to analytically evaluate the communicability of computer systems. SIM helps the analyst reconstruct the designer-to-user message, and detect potential breakdowns in communication.
Our goal was thus to analyze the metacommunication message of CoScripter’s designers with relation to user cooperation. For this purpose we analyzed different classes of signs from CoScripter’s home page, online documentation, toolbar and wiki. We summarize below the CoScripter’s designers’ message emerging from our inspection, focusing specifically on cooperation, where ‘we’ means ‘we, the designers’ and ‘you’ means ‘you, the users’: 1) We encourage users to share their recipes.
pick and Relevant Scripts (they are scripts that were made to web page that you are currently viewing).”
3) We encourage users to build on others’ scripts.
The scripts on the wiki "have been written by a large community of users who help each other".
People are automating "their most difficult web-based processes".
"We would like you contribute with the Wiki!"
"Together we can make the web easier to use!"
We encourage users to create public scripts: "Note that every script you create is automatically saved on the CoScripter website for other people to view and
Forum posts analysis The analysis of posts in CoScripter’s forum provides insight about cooperation. We summarize below some of the results, followed by some examples of user statements. 1) Some users are satisfied with CoScripter but have not shared their scripts.
“I (…) have used it very successfully to automate a boring & time-consuming process.” “This is a fantastic idea and I like what I have seen.”
2) Some users prefer to share scripts within specific communities, and not with the wider public.
“Is there a way to set up an internal wiki that will use the ‘load in sidebar’ feature? I'd prefer not to have the scripts public on this site, or to fight with local save, but we do have a small group that will benefit from sharing data.”
“Is it possible to publish scripts on my own website?”
3) Some users are worried about malicious scripts.
“How do I know that a CoScript [sic] script isn't just another form of phishing? How do I know what's being captured under the covers of CoScript?”
run! The exception to this rule is private scripts, which are only visible to you when you are logged in to the system."
If a user edits someone’s script, we will offer the “overwrite” option as a default, rather than the “make a copy” option.
"We hope this encourages people to share their recipes for doing things and to collaborate on improving their workflow."
"We recommended some scripts for your use: Editor’s
4) It is not clear to some users how to add comments.
“How do you add comments in your commands?”
3
5) Some users reported bugs and security problems with variables 1.
“There is still a privacy problem caused by ‘example values’.”
Public scripts analysis The goal of this study was to analyze user cooperation in the public repository of CoScripter scripts. For this purpose we downloaded all the 3117 public scripts and their edit histories (7569 versions) from the CoScripter’s Web site as of September 30, 2008. After that, we parsed these scripts and versions to collect the data used in this study. We investigated two kinds of user cooperation, namely the execution of scripts created by other users and the editing of scripts created by other users. The study of editing cooperation has more details, since the public repository provides the editor and the time of editing (which is not the case of script execution). Execution of scripts created by other users
Bogart et al. [1] collected all public CoScripter scripts on December 18, 2007, and partitioned them into three groups, namely ManyUsers (scripts executed by more than three users), ManyRuns (scripts executed by at most three users, and that averaged six or more runs per users), and FewUsers/FewRuns (the remaining scripts). We observe a reduction in the proportion of ManyUsers scripts (from 9% to 7%), and a significant increase in the proportion of ManyRuns scripts (from 7% to 16%). These results suggest a higher use of one-user scripts and/or an increasing interest for scripts shared by small groups of users (two or three users). Editing of scripts created by other users Most editions (92%) were done by the script’s creator. In addition, we observe in the remaining editions many cases of modification of the script’s purpose. We believe that in most cases this change of purpose is caused by user mistakes (they should have saved the script as a copy, instead of overwriting it).
If we could loosely consider that all users of a script execute it equally often (in average), we would have that 13% of the executions were done by a user other than the script’s creator2. Furthermore, 77% of scripts have only one user (the creator), which suggests that little attention is directed to scripts of other users. As a time evolution analysis, we observed the current state of scripts’ popularity categories proposed in [1].
1 Bogart et al. [1] reported that scripts that use variables are more likely to be executed by multiple people. 2
One outlier script was omitted because it was a recursive script with 81% of all executions.
figure 1. Graph of cooperation in edits. Each node represents a user. Two users are connected by an edge whenever both edited at least one script in common.
4
In Figure 1 we present clusters of users with relation to editing cooperation. We considered the script creation as an edition. We observed that most of the scripts involved in cooperation were recommended by the designers, giving rise to the biggest cluster. In addition, most users (80%) did not edit scripts that were created or edited by other users (these non-cooperating users are not shown in Figure 1). From the remaining users, 23% are in 2-user clusters and 10% are in 3-user clusters. This proportion of users per cluster size was very stable across time.
Current conclusions and further work
[1] Bogart, C., Burnett, M., Cypher, A., Scaffidi, C. End-user programming in the wild: a field study of CoScripter scripts. VL/HCC 2008, 39-46. [2] CoScripter. Simplifying Web-based processes. http://services.alphaworks.ibm.com/coscripter/browse/ about. [3] Cypher, A. In “Watch What I Do: Programming by Demonstration”. The MIT Press. Cambridge MA, 1993. [4] de Souza, C. S., Leitão, C. F., Prates, R. O., and da Silva, E. J. (2006). The semiotic inspection method. In Proc. of VII Brazilian Symposium on Human Factors in Computing Systems, 2006. IHC '06. ACM, 148-157. [5] de Souza, C. S. (2005) The semiotic engineering of human-computer interaction. Cambridge, MA. The MIT Press. [6] de Souza, C. S. and Cypher, A. 2008. Semiotic engineering in practice: redesigning the CoScripter interface. In Proc. of the Working Conference on Advanced Visual interfaces, 2008. AVI '08. ACM, 165172. [7] Leshed, G., Haber, E. M., Matthews, T., and Lau, T. 2008. CoScripter: automating & sharing how-to knowledge in the enterprise. In Proc. of CHI'08. ACM, 1719-1728.
We outline below the current conclusions resulting from the triangulation of the studies presented in this paper:
CoScripter’s designers seem to have succeeded in their overall cooperation intent but, as of now, the users’ interests are concentrated in a small group of scripts (the scripts recommended by the designers).
CoScripter’s designers (through the system interface) invite the users to overwrite scripts of others. However, script editing cooperation is rare, and in many cases edits of users other than the script’s creator erroneously alter the script’s purpose.
Some users who employ CoScripter to automate their process do not intend to share their scripts. In many cases they prefer to cooperate in small communities.
Some users reported problems with two CoScripter’s resources that we consider important for cooperation: comments and variables.
Our next steps on this research path are:
An in-depth empirical study about the perceived potential and obstacles to further cooperation based on CoScripter.
More detailed studies of script edits: What is the proportion of cooperative editing? What patterns of cooperation are in place? How can the current cooperation mechanisms be improved?
Acknowledgments Clarisse de Souza and Simone Barbosa thank CNPq for supporting their work. Andréia L. Sampaio thanks CNPq for her Ph.D. scholarship. Críston P. de Souza thanks FAPERJ for his Ph.D. scholarship.
References
