Tools and Principles for Collaborative Design Mark D Gross CoDe Lab Carnegie Mellon University [email protected]

Ellen Yi-Luen Do ACME Lab Georgia Institute of Technology [email protected]

ABSTRACT

Advocacy

We reflect on our experience and efforts engaged in tool building to support design from shared drawing to both physical and virtual space. We briefly describe the characters of these projects and outline their commonalities to discuss the design principles.

A tool implicitly advocates for a particular type of interactions or a familiar practice. This quality expresses the tool builder’s position about how the practice would be. Buoyancy

For a tool to possess buoyancy it must have an easy to use interface with an elastic spirit. This principle is concerned about the multiple modalities with which people would interact with the tool.

Author Keywords

Sketching, annotation, physical computing, multimodal interaction. ACM Classification Keywords

Efficacy

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

By efficacy we mean the ability to provide functional capabilities, to produce effects, and to affect the intended outcome. This principle reminds us of the requirement specifications of the tool.

INTRODUCTION

Design of various kinds often involves collaborations among many disciplines. These collaboration activities can happen synchronously, or asynchronously, at the same location or between remote places. For example, architects use drawing documents such as a floor plan or a detailed elevation to discuss a design with engineers and clients. City planners and neighborhood residents meet at community workshops to discuss and review future developments over aerial maps and physical models. Interactive exhibit space engages an audience of different backgrounds as participants to affect the multimedia performance of sound, light or video. These conventions of practice are well accepted among the communities because they have worked (to a certain extent), as the result of years of use and incremental modification. Recently we have also seen the advent of computing (software and hardware, physical or virtual) that provides added value to facilitate these types of work.

Fluency

Each tool that effectively supports design collaboration would exhibit certain degrees of fluency. As with fluency in speech, language, music, poetry, or thought, fluency in collaborative tools supports a smooth and easy flow of actions and transitions. Transparency

A good tool would exhibit the transparency of the design. The interface and interactions should be transparent to the users and match users’ perceptions of how these tools and technologies can be used seamlessly in our collaborative environment. PROJECTS

The aforementioned design principles were derived from our experience and reflections of several of our past projects built to support collaborative design. The “shared space” for collaborative practice in these projects can be characterized into the following three categories: (1) shared drawing surface and multimodal interaction, (2) annotations in the virtual environments, and (3) computationally enhanced physical Space.

DESIGN PRINCIPLES FOR COLLABORATIVE TOOLS

Over the past decade, we have developed different tools to support collaborative design activities. Most of the tools and projects we built were inspired by our own experience in practice and the need to communicate and to collaborate. Even though the “shared spaces” for collaborative practice are different for these projects, they all have certain characteristics and design principles. We come to reflect that the characters of Advocacy, Buoyancy, Efficacy, Fluency and Transparency are important principles for designing collaborative tools. Below we briefly discuss their qualities and speculate on the implications for tool making and tool use.

Shared Drawing Surface and Multimodal Interaction

Our first round of efforts focused on the idea of shared drawing boards that support sketching and drawing. This drawing-centered view of designing results from the observation that drawing is typically the most prevalent representation of the object under consideration throughout

1

the design process in the architectural domain, from early conceptual sketches to final fabrication drawings.

Annotations in the Virtual Environments

In the domain of architectural design, the eventual artifact of design is inherently three-dimensional. Therefore, to support collaboration between designers and their partners such as clients and contractors, we first built the Redliner [8], a web-based system that enables users to post text annotations on a 3D model to support asynchronous collaboration.

Figure 1. Synchronous collaboration with design history, locking and ephemeral arrow gesture in NetDraw

The Electronic Cocktail Napkin [4, 5] and the Right Tool at the Right Time projects [3] support recognition of freehand sketching symbols, analysis of spatial relations, and inference of design intentions to activate knowledge based design tools that support the tasks at hand. For example, a designer could sketch symbols of floor plan elements (e.g., window, door, furniture) to retrieve a similar spatial layout from a case library, or to activate a simulation program of visual access to inform design decisions. NetDraw [11] supports synchronous collaborative drawing across the network with server-client architecture and design history capture that allows backtracking to a previous stage in time for editing. Ephemeral gesture commands support deictic reference to draw attention to the point of interest with drawing marks that gradually fade away. NetDraw’s concurrency control provides a locking synchronization to assure that no object can be edited at the same time as well as flagging the “being edited” object status with changed color display.

Figure 3. Recording design rationale with annotation markers representing different stakeholders in Redliner.

Collaborators can navigate and walk through the virtual model just as in a game environment, and can click on any surface to leave “post-it note” comments. These comments could be design rationale, critiques, concerns or suggestions. The viewpoint when the annotation was made is saved and associated with the annotation mark. Any visitor can click on any of the annotation markers to be “tele-ported” into the location where the annotation was made. This is particularly useful when the concern is about visual access from a specific vista point, or to identify nearby or intersecting objects that might not be easily perceived on a 2D drawing (e.g., beam overhang obstructs access to duct work). This tool effectively creates a “placebased” discussion forum in which corresponding concerns and responses are co-located, with easily identified author color and ID markers. Alternative design options can also be embedded in the virtual model for other collaborators to compare and leave comments.

Figure 2. Recording, playback and search of the graphical and spoken design conversation in Design Amanuensis

While NetDraw supports a chat function for capturing design discussion, the Design Amanuensis [6] combines the capture of strokes with a continuous speech-to-text recognizer as multimodal design conversation that can be playback and analyzed later. The speech record and drawing events are linked through their time stamps. The design record can be searched by key phrases in the speech or by graphic components in the drawing. The playback of speech record highlights both the text in the transcripts and the drawing components that were drawn at that time. Selecting any part of the drawing plays back the corresponding associated text record.

Figure 4. Sketching annotation on temporary surface and on the wall to suggest design alternatives in Space Pen.

The lessons learned from user studies of Redliner led to the development of Space Pen [9]. The Space Pen project [10] supports both sketching and text annotations on any 3D surface. For example, a user could quickly sketch the shape and location for an opening on any existing surface (e.g., the wall), or create a temporary drawing surface anywhere

to sketch a mark where no surface existed, or to create a new foam board like shape surface in the space. All viewpoints of the annotations are saved and displayed on a floor plan view, and user can click on the viewpoints for a guided tour. A location based threaded discussion is supported by clicking on any surface to initiate new comments, or by clicking on any existing yellow post-it note marker to add or view past discussions.

The Plant Tiles project and the Alphabet Paint Space [1] also encourage people to use their body movement to affect their experience of the environment by changing the outcome of a multimedia presentation. The subject matter of Plant Tiles is the transformation and the cycles of growth in plants. Moving forward and backward on the home-made pressure sensitive floor tiles controls the playback speed and sequence of these video segments and time-lapse photography of these plants in different scale. By moving around the space the occupants control the showing of individual growth of the plants.

Computationally Enhanced Shared Physical Space

Embodiment in a real physical space with computation could enhance and empower our experience in collaboration. The design of the MouseHaus Table [7] is a computationally enhanced physical environment to support discussion in urban design. The system consists of a custom-made table with a rear projection screen, a video camera, projector and a simple pedestrian movement simulation program. MouseHaus Table provides a physical interface that enables participants who have no previous computer experience to interact with a pedestrian simulation program by cutting and putting colored paper on the table to represent placements of functional objects such as buildings and parks in an urban design context.

Figure 7. Movements on Plant Tiles triggered different playback speeds and displays of educational video of plants

The Alphabet Paint Space “paints” with people’s motion. The environment is a building atrium decorated with an abstract painting wall and a set of large randomly placed Sesame Street style letters with embedded photocell sensors.

Figure 5. MouseHaus Table facilitates collaborative urban design with color papers and pedestrian movement simulation

The People Pretzel [12] is a computationally enhanced play board designed to promote informal interaction in public gathering spaces. The game board uses audio, light and visual information to intrigue and entice passers-by to participate in a playful social interaction. It augments physical group interaction in a multimedia performance. Players and audience alike engage in such group activity both actively and in a passive manner. The board can also serve as a programmable physical interface to support a variety of games or to control environment parameters such as lights and background music. This artifact encourages people to shape and enrich their collective experience in a community gathering space. Using the People Pretzel game board tends to make people laugh and it initiates spontaneous conversation, even among strangers.

Figure 8. Interactive mural capturing people movements in Alphabet Paint Space

A video camera captures images of people moving through the space. The processed images are projected onto a large screen resulting light-painted mural traces movement that fades slowly over time. The traces disappear so that the mural constantly evolves, reflecting the current state of the space. Passing in front of a letter causes that character to appear on the bottom of the movie screen and this encourages communications and coordination about movement among people to collaboratively construct poetic text as if writing an abstract title together. THE BE-FAT DESIGN PRINCIPLES

We have described a sample of our tools that aim to support collaborative practice in design. We have come up with the BE-FAT –Buoyancy, Efficacy, Fluency, Advocacy and Transparency principles. We are interested in the study of methods for designing design tools, maybe with the concerns of “designerly ways of knowing” [2]. Below we briefly recapitulate these principles in light of the projects outlined above:

Figure 6. People Pretzel game board encourages social interaction and game playing with light and sound.

3

Buoyancy is the flexibility and ease of use. For example, the ephemeral gesture commands in NetDraw suggests a “Cheshire cat” lightheartedness that encourages casual and frequent engagements (one can mark up as much as he wants to call for attention and discussion without messing up the drawing). People Pretzel encourages people to participate either as players or audience by stepping on it or putting any body part on it. The “walk up and use” aspects of Space Pen, Plant Tiles and Alphabet Paint Space all attest to the attractiveness of Buoyancy. Efficacy can be seen in the process of using the tools. Effects are produced by operations. A tool that supports easy operations would achieve its efficacy. For example, the ideas of the Right Tool at the Right Time or Design Amanuensis all support the shared artifacts (drawing and conversation) that can be manipulated effectively. Redliner and Space Pen support annotations that can be retrieved and viewed at the appropriate locations. Moving and cutting color papers to represent design objects in the MouseHaus Table project is an effective way for citizen participations. Fluency - Many of the sketching systems we describe exhibit the quality or state of flowing or being fluent by simply allowing people to draw what they want. Recognizing a familiar interaction pattern can also be considered as fluency in action. Electronic Cocktail Napkin, MouseHaus Table and Plant Tiles all support operation of smooth transitions between states. Advocacy is the position about how a collaborative tool would facilitate people to perform intended actions. For example, Redliner’s default click function opens a text input window and leaves an annotation mark in space. This encourages leaving comments for other people to see. Design Amanuensis leverages the fact that people engage in multimodal communication (speech and gesture), and supports searching with both textual and graphic inputs. MouseHaus Table suggests the affordance of putting everyday objects on the table to facilitate communication. Likewise, the pressure sensitive Plant Tiles simply advocates movement for exploration of plant lives. Transparency- The tool is supposed to facilitate collaboration, and should not get in the way of creative practice of design. The functionalities of the tools therefore should be obvious for the intended users. For example, the “play back” control panel in Design Amanuensis makes it obvious and transparent that one can “forward” or “rewind” the recorded design conversation. Running forward on Plant Tiles would fast-forward the plant life video also demonstrates this transparency principle. ACKNOWLEDGMENTS

We thank our students who participated in the projects mentioned in this paper, their names appeared either as paper co-authors or were acknowledged in these papers. This research was supported in part by the National Science Foundation under Grants CCLI DUE-0127579 and ITR-

0326054. The views and findings contained in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation REFERENCES

1. Camarata, K., Gross, M.D. and Do, E.Y.-L. A Physical Computing Studio: Exploring Computational Artifacts and Environments. International Journal of Architectural Computing, 1 (2). 170-190. 2. Cross, N. Designerly ways of Knowing: Design Discipline Versus Design Science. Design Issues, 17 (3). 49-55. 3. Do, E.Y.-L. The Right Tool at the Right Time Investigation of Freehand Drawing as an Interface to Knowledge Based Design Tools. Georgia Institute of Technology, Atlanta, 1998. PhD Dissertation 4. Gross, M.D. The Electronic Cocktail Napkin - working with diagrams. Design Studies, 17 (1). 53-69. 5. Gross, M.D. and Do, E.Y.-L. Demonstrating the Electronic Cocktail Napkin: a paper-like interface for early design. in CHI 96, Conference on Human Factors in Computing Systems, ACM, Vancouver, British Columbia, Canada, 1996, 5-6. 6. Gross, M.D., Do, E.Y.-L. and Johnson, B.R. The Design Amanuensis: an instrument for multi-modal design capture. in Vries, B.d., Leeuwen, J.P.v. and Achten, H.H. eds. CAAD Futures 2001, Kluwer Academic Publishers, Eindhoven, 2001, 1-13. 7. Huang, C.-J., Do, E.Y.-L. and Gross, M.D., MouseHaus Table, a Physical Interface for Urban Design (poster). in User Interface Software Tools (UIST), (Vancouver, 2003), ACM Press. 8. Jung, T. and Do, E.Y.-L. Immersive Redliner: Collaborative Design in Cyber Space,. in ACADIA 2000, Associations of Computer Aided Design in Architecture National Conference, Washington D.C., 2000, 185-194. 9. Jung, T., Gross, M.D. and Do, E.Y.-L. Sketching Annotations in a 3D Web Environment. in CHI 2002, Conference on Human Factors in Computing Systems, ACM Press, Minneapolis, 2002, 618-619. 10.Jung, T., Gross, M.D. and Do, E.Y.-L. Space Pen: annotation and sketching on 3D models on the Internet. in Vries, B.d., Leeuwen, J.P.v. and Achten, H.H. eds. CAAD Futures 2001, Kluwer Academic Publishers, Eindhoven, 2001, 257-270. 11.Qian, D. and Gross, M.D. Collaborative Design with NetDraw. Computers in Building, Computer Aided Architectural Design Futures. 213-226. 12.Shaer, O., Ziraknejad, B., Camarata, K., Do, E.Y.-L. and Gross, M.D. People Pretzel: A Computationally Enhanced Play Board for Group Interaction. Pervasive Computing 2004, Hot Spot Paper. 357-361