Instrumentness in Clusters of Artefacts -- a First Take on Collaborative Creativity Olav W. Bertelsen Dept. of Computer Science University of Aarhus Denmark [email protected]

Morten Breinbjerg Institute of Aesthetic Studies University of Aarhus Denmark [email protected]

ABSTRACT

Based on the concept of instrumentness and the notion of interaction through clusters of artefacts, we discuss collaborative creativity. We suggest that the close group interaction is not a sufficient perspective in understanding artistic collaborative creativity. Author Keywords

Collaboration, instrumentness, computer mediated creativity, music composition, interface aesthetics, metonymy, mediation, materiality, activity theory, instrumental interactions, clusters of artefacts. ACM Classification Keyword

H.5.0 HCI: general; INTRODUCTION

Creativity is collective, but still it is realized through the agency and actions of individuals engulfed in culture. This paper is based on two earlier papers, that we bring into constellation to approach creative collaboration. We adopt the concept of “interaction through cluster of artefacts” (Bertelsen & Bødker 2002), which is an activity theoretical perspective on mediated collaboration, in revisiting the concept of instrumentness as introduced at the present conference (Bertelsen, Breinbjerg & Pold 2007). In our paper on instrumentness (op. cit.) we focus on two complementary aspects namely the materiality of the tool and the form of representation, in order to understand the specifics of computer-mediated creativity. Instrumentness points to the way musical instruments are controlled and conceptualized through values such as virtuosity and playability, which are important for computer-mediated

.

Søren Pold Institute of Aesthetic Studies University of Aarhus Denmark [email protected]

creative work supporting development in use beyond what is initially designed for. INSTRUMENTNESS IN ELECTRONIC MUSIC

In the electronic music composers we studied, the categorical difference between the interaction instrument and the domain object is blurred in the sense that the interaction instrument itself (the software) equally becomes their object of interest, i.e. the domain object. In short, the interaction instrument is comprehended as more than just a tool for manipulating the domain object. Software is not an instrument in the same sense as a violin, but several instrument characteristics are traceable in the composers’ use and understanding of software. First of all the software is playable. The sound generating processes run in real time and the sound can be manipulated instantaneously by moving a slider or turning a knob in the GUI interface or by typing in numbers or altering codes. Secondly, the software (filters, oscillators, reverbs etc.) has a unique sound profile due to the nature of the sound algorithms and as such they are comparable to different instruments. Also the understanding of the software as an instrument can be seen in the way they accept they have to discipline themselves in the use of the software in order to benefit from its complexity, but most important also to extend or perhaps even transcend its limitations. Like a violin, such software is not easily mastered, but with enough work it provides a possibility for achieving virtuosity. It is worth noting, however, that this should not lead to the idea that design for mastery can be obtained by making the artefact clumsy or inaccessible. Rusty violin strings and instable or awkward user interfaces do not per se further creativity. The software mediated composition and production of music can almost never be described in terms of a single user-tool-object triangle of mediation. In general we see long chains of mediation; from the composer to the music experienced by the audience. E.g. Max/MSP mediates the programming of a Max/MSP patch, but the patch itself mediates both the performance situation (as a filter and instrument), and the composition situation (as a material resistance and as inspiration). An example of the complex mediation going on in electronic music was a composition

made in collaboration between two composers. The performance involved the two composers operating their PowerBooks and various attachments, as well as two bass players and a viola player. The strings played music partly written in ad hoc notation, and their sounds were to some extent processed through the computers. Not only is this a long chain of mediation, but it is also folded together in a cluster of artefacts. We see here a typical example of how computer applications take several simultaneous roles when used in a creative context, in particular how the alteration of the components between being instruments and objects is so rapid that the distinction almost seems to break down. This dynamism can be understood in the concept of computer artefacts always being clusters of primary, secondary and tertiary artefactness as proposed in (Bertelsen 2006) with reference to Wartofsky’s (1973) analysis of how human perception develops historically through the use of artefacts in productive practice (mediated by primary and secondary artefacts) as well as in the detached domain of art (mediated by tertiary artefacts). The point is that the computer artefact is not only mediating the (re) shaping of the object, it is also (reference to) representations of modes of acting, and (reference to) derived cultural images of the artefact. As such, the composers play the software as an instrument (as described above) exploring its material dimensions and resistance. When composing and playing, they are not only occupied with product, the resulting music, but also with the process, its intricacies and challenges. The software resists the composers’ intentions and it yields surprising and unexpected results that are important for the creative process. When dealing with digital instruments a special aspect concerning the resistance of the material is the metaphorical design of the interface. To sum up, the key aspects of instrumentness are that the instrument is oscillating between positions of being transparent means and objects of reflection, and that this oscillation is rooted in the materiality of the instrument and the metonymical form of representation yielding familiarity as well as transcendability. Furthermore, the instrumentness was observed as being related to rather complex mediation structures. INSTRUMENTNESS IN CLUSTERS OF ARTEFACTS

Our paper (op. cit.), does not deal with collaboration in a direct sense. However, the mediation structures we identified can appear to be similar to the complex mediation that can be seen from a collaboration perspective. It is, however, quite obvious that music production has strong collaborative elements. Not only in the sense of a group of musicians collaborating synchronously and collocatedly in performing the music, but also across situation and time in a looser coupled sense.

mediation identified by Bertelsen & Bødker (2002). The lessons are quoted from the paper and set in boldface. Lesson 1: use is more than one use situation, and use situations and contexts are non-hierarchical, overlapping, transient and heterogeneous. In the case of the electronic music composers we saw how a sequencer could be a tool for laying down a beat in one situation and for structuring a whole track in another. The instrumentness ensures that the instrument is likely to function in such varieties of situations. By the materiality users are enabled to turn the object into an object by itself and thereby transfer it between fundamentally different situations. Likewise, because instrumentness relies on metonymy rather than on metaphor, the representations in the interface lend themselves more easily to transient uses and purposes. Lesson 2: many actors and artefacts mediate the same activity/focus on the same object. In the composer case, the object could be the music that the audience experience at a concert, the music recorded, or similar. The composer as well as the possible musicians, producers, etc are involved; they all contribute in various ways by means of different tools. The instrumentness of the involved artefacts does that the same artefact can become very different things (or rather mediators) in the hands of different actors with various actions in mind. Lesson 3: the outcome of one activity becomes the artefact of others. In the composer case, this was a recurring theme. A typical example is how a Max/MSP patch is an object in one part of the composers’ activities, but turn artefact in the creation of a specific piece of music. In general, the instrumentness of an artefact involves the ability to oscillate between being tool and object as a central feature. Lesson 4: The overall activity may be carried out through chains of objects turn artefacts, i.e. remediation. The production of a music experience for an audience, starting form the early idea, is such a chain, and so is the above mentioned example form the composer case about the production and use of Max/MSP patches. In general, instrumentness is related to this kind of remediation, based on the object-like materiality in the instrument.

10 LESSONS ON COMPLEX MEDIATION

Lesson 5: Substitution of artefacts happens and may or may not be hierarchical.

To explore the collaborative aspects of instrumentness, we revisit ten collaboration-oriented lessons on complex

Most of the artefacts seen in the composer case are redundant in the sense that they do not carry unique

functionality per se the way the interviewed composers reported they used the tools showed that. We believe that, in general, instrumentness furthers a kind of uniqueness of each artefact that makes this kind of substitution more specifically understandable and more depending on situation. Thereby, substitution is not a result of functional equivalence but rather a matter of fine, local adjustment. Lesson 6: There is more than one artefact at a time, working together to mediate the activity, i.e. juxtaposition. Also this lesson was supported by the electronic music composer case. Instrumentness means that the purpose and potentials of the instrument are continuously debated. As the instrument become artefact it almost necessarily functions in concert with other, complementary, artefacts. Lesson 7: Artefacts modify other artefacts. This happens over and over in the composer case. For instrumentness in general this means that the instrument never reaches a final stage of development, and that it in the oscillation be being instrument and being object can be modifier and modified at the same time. Lesson 8: Artefacts lend themselves to open and heterogeneous uses, where the artefacts are used •

In other ways than they were intended,

•

Differently depending on purpose,

•

And in manners that are difficult to anticipate.

Al these features were seen in the composer case. E.g. in the way samplers were systematically violated in highly unintended use having very interesting outcomes. In general, this un-anticipatedness of use has been one of the starting points that were confirmed and handled in the development of the concept of instrumentness. Lesson 9: One artefact mediates many purposes at once. We saw many instances of this in the composer case. The sequencer is one of the very clear examples of this. Max/MSP is an even stronger example in the sense that the production of patches for the community almost tends to overshadow the creation of music. The way the concept of instrumentness provides an understanding of mediation that breaks with the clear unidirectional purposefulness otherwise taken for granted opens for this “one artefact – many purposes” idea. Lesson 10: Artefacts are clusters of primary, secondary and tertiary artefacts (possibly distributed into chains of mutually modifying artefacts). The software we saw in the composer case had a clear primary artefactness. It also had an abstracted secondary

artefactness in the sense that the software clearly pointed to its own use. At the same time, however, the metonymic features turned the secondary artefactness into a quasitertiary artefactness, pointing to how the software could possibly be used in the future. The way that the sequencers were both tape decks and not at all so was a good example of such a poetic opening through metonymy. In general, instrumentness implies this kind of clustering of primary, secondary and tertiary artefactness. The materiality of the software instrument means that it is can be an object that reifies and mediates functionality and at the same time be an object for reflection at several levels and an object that points to the past, present and future use of the software. This simultaneous pointing back and forward in developmental time is also supported by the metonymic aspects of the software. A COLLABORATIVE PERSPECTIVE

In our analysis of the composers work with their software we focused on the interaction at the level of the interface (op. cit.). Thus, the resulting concept of instrumentness at first sight seems to be a ”one user - one computer” concept. However, from the analysis along the lines of the collaboration-oriented concepts of complex mediation it becomes clear that the mediated creativity we described was also collaborative. As pointed out music is always about several people, either in the direct sense or in a looser coupled way. In the direct sense, we saw how the creation of interactive sound structures with Max/MSP can be central in the collaborative performance of an electronic composition. The way that ongoing stylistic changes are continuously crystallized and re-crystallized into the music software, through versioning by manufacturers, through sharing configurations and patches over the internet, and through re-combination of software in use reflects a collaborative aspect that is not tied to the small group, but rather is going on at a cultural level. Thus, in the case of technology-mediated creativity in artistic work, we see collaboration that is fundamentally different from the collaboration discussed by Bertelsen & Bødker (op. cit.). Whereas Bertelsen & Bødker see collaboration being directed to a common activity with a shared motive, and thereby fundamentally places the group collaboration at centre stage, the systematic concept of human activity seems less obvious in our case. In artistic production and reception, collaboration is distributed across time, and possibly over various eras; it is distributed across persons who do not necessarily have a shared motive or object of activity; and most importantly collaboration is distributed across situations. In the case of software mediated artistic creativity, it is also clear that individual agency is central – creativity is realized through the actions of individuals. While perspectives like that of Bertelsen & Bødker acknowledge the existence of individual action, the work of the composers can indeed be seen as completely individual, and has traditionally been

seen as such. However, the concept of instrumentness enables us to get closer to an understanding of technologymediated creative collaboration. In short we argue that perspectives on creative collaboration should transcend the close group setting and look into the role of cultural and societal formations as locus for creativity and change. As part of this reorientation the role of CSCW and related disciplines should aim to understand and deal with the transformative aspects crystallized or manifested in products of art, literature and music. This change of focus is parallel to the current aesthetic turn in human-computer interaction as manifested in new approaches such as the interface criticism (Bertelsen & Pold 2004).

REFERENCES

1. Bertelsen O. W. (2006). Tertiary Artefactness at the interface, In Fishwick, P. (ed.) Aesthetic Computing. Cambridge MA: The MIT-Press. 2. Bertelsen, O.W., Breinbjerg, M. & Pold, S. (2007). Instrumentness for Creativity - Mediation, Materiality & Metonymy. In Proc Creativity and Cognition 2007. 3. Bertelsen, O. W. & Bødker, S. (2002). Interaction through clusters of artefacts. In Proc. of 11th European Conference on Cognitive Ergonomics (ECCE-11), Catania, Italy, September 2002. 4. Bertelsen, O. W. and S. Pold (2004). Criticism as an approach to interface aesthetics. Proceedings of the third Nordic conference on Human-computer interaction. Tampere, Finland, ACM Press 5. Wartofsky, M.W. (1973). Perception, representation, and the forms of action: toward an historical epistemology. In Models. Dordrecht: D. Reidel Publishing Company, 1979, 188-210.