EXPRESSIVE 2016
Computational Aesthetics in Graphics, Visualization, and Imaging Angus Forbes and Lyn Bartram (Editors)
Relational interactive art: A framework for interaction in a social context Nuno Cabrita1 and Gilberto Bernardes2 1 Universidade
2 Sound
do Minho and Music Computing, INESC TEC
Abstract Interactive art implies an active dialogue between the participant and the surrounding space, mediated by a computational system. Reciprocity and recursiveness are key principles to the bidirectional flux of information in this setting, guaranteeing a continuous interaction loop between the participant and the digital system. Viewing the human body as a natural interface, we focus on non-invasive tracking methods for embodiment sensing, such as infra-red depth cameras. Current limitations in participant engagement of interactive artworks in public spaces are introduced and analyzed from the perspective of group dynamics. In this paper we approach Bourriaud’s concept of relational aesthetics, relate it to the inherent social context of interactive artwork exhibition, and propose a framework for the development of relational interactive artworks. Interactive art implies an active dialogue between the participant and the surrounding space, mediated by a computational system. Reciprocity and recursiveness are key principles to the bidirectional flux of information in this setting, guaranteeing a continuous interaction loop between the participant and the digital system. Viewing the human body as a natural interface, we focus on non-invasive tracking methods for embodiment sensing, such as infra-red depth cameras. Current limitations in participant engagement of interactive artworks in public spaces are introduced and analyzed from the perspective of group dynamics. In this paper we approach Bourriaud’s concept of relational aesthetics and relate it to the inherent social context of interactive artwork exhibition, and propose a framework for the development of relational interactive artworks. Categories and Subject Descriptors (according to ACM CCS): [Applied Computing]: Arts and Humanities—Media Arts
1. Introduction Interactive art compels participant to establish active dialogue with the surrounding physical space through a computational system [Pen96]. A bidirectional flux of information between the participants actions and the digital system’s responses establishes a recursive loop, which shifts and empowers the role of the audience from a traditional passive participant to an active agent, without whom the artwork ceases to exist. The human body is a natural interface to the human consciousness and to the physical word and is a key element of seamless and transparent communication in interactive art, which sensor technology translates into the digital domain to be used by computational systems. Yet, the social context in which artworks are commonly exhibited, such as art galleries and museums, is a major drawback to the use of the body as an interface. These spaces commonly inhibit people from actively engaging with the artwork, due to not being accompanied, or to the discomfort of performing bold actions in a public place. Participant inhibition in interactive art has been analyzed through group dynamics [SHSHB13], which evaluates the impact on the critical reception of an artwork depending on social settings, such c 2016 The Author(s)
c 2016 The Eurographics Association. Eurographics Proceedings
as whether we are surrounded by strangers, friends or family. This element has itself been a topic of interest in contemporary art, and more particularly, in relational aesthetics, which concerns art as a space of social encounter, striving to build interpersonal relationships among its several agents. The term relational art was coined to describe the type of artworks that artists such as Rirkit Tiravanija developed in the 1990’s [MOM12]. It is related to a critique of the “mechanization of the social functions” [Bou02] promoted by the repressive dynamics of capitalist production. Relational art is characterized by its primary goal of gathering people and creating a social bond through an artwork’s aesthetic properties. In this paper, we approach the currently underdeveloped link between relational aesthetics and interactive art. By focusing on the body and its actions in space as the main tool for interaction with an artwork, we establish a connection between interactive art and relational aesthetics as an artwork practice framework that aims to rethink and enhance the role of the “relational” in the interaction. In so doing, we strive to contribute to the discussion on the problems with interaction engagement caused by inhibition in public spaces as well as proposing new strategies to deal with such constraints.
N. Cabrita & G. Bernardes / Relational interactive art: A framework for interaction in a social context
The remainder of this paper is structured as follows: Section 2 reviews interactive art, focusing on the body as an interface and embodiment sensing entity. Section 3 assesses the limitations that exist in the fruition of interactive art. Section 4 introduces the concept of relational aesthetics. In Section 5 we establish a framework for the execution of relational interactive artworks. Section 6 summarizes the existing discussion on this topic and provides conclusions.
To establish a direct relation between the analogue realm of the body and physical space and the digital system that characterizes interactive art, sensor technology emerges as an essential element. It allows for the capture and conversion of real-world analogue signals into digital ones that can be used by the computational system.
2.2. Embodiment Sensing 2. Interactive Art In art, interactivity results from an effective feedback loop between the audience and physical scape [AS03] mediated through a realtime computational system [Pen96]. The audience within this context are considered as participants [AS03], in the sense that their embodied actions transform the aesthetic properties of the physical space and are essential for the emergence of meaning [AS03]. Walsh highlights reciprocity and recursiveness as key aspects of interactivity [Wal11]. This implies a bi-directional dynamic in the definition of behaviors. The participant’s actions affect the system responses, which in turn, compels the former to take action. This feedback behavioral loop is, to a certain temporal extent, selfperpetuating, in the sense that a specific procedure can repeat itself on top of it’s own previous characteristics. Within the scope of our work we are interested in audiovisual expressions, so in this context we shall focus our approach on what Kluszczynski calls the “Strategy of Spectacle” in interactive art [Klu10]. This means that the intended aesthetic experience of the artwork exists as a spectacle to be contemplated. The contemplative aspect here implies the possibility of different degrees of interaction with the artwork: from semi-passive, in which the participant might simply trigger the beginning and end of a set event, to fully engaged, where the participant affects the system’s response more significantly through an extended spectrum of actions. The closer we move from the former to the latter, the richer the interaction. An example of this is Lozano-Hemmer’s work Body Movies [LH], where both contemplation and significant interactive involvement are present: people using their own shadows as an interactive device to affect the response of the large-scale mural in front of them. 2.1. The Body as an Interface The body is our instrument of navigation in the world, as its senses and motion capabilities allow us to interact with our physical surroundings. Merleau-Ponty [MPS96] reflected on this issue, viewing the body as an element that bridges our consciousness with the perceived world. Our sense of kinaesthetic awareness concerns one’s perception of their actions in relation to their own body (location, direction and orientation of movement). Proprioception is the individual’s own awareness in relation to his actions in the surrounding space, which are controlled by i.e., the adjustment of the eyes’ focus, the articulation of speech, or by guiding the overall movement of the body in the surrounding space [BV07]. In order to take advantage of the aforementioned properties of the human body, interactive art aims to impact our senses in the first place, leaving rational thought to appear only later in the process of interaction / fruition. This constitutes a visceral approach [SR09].
The sheer existence of the full human body in physical space as an interface for interaction with an artwork is enhanced through transparent and seamless tracking methods. Infra-red sensors have become the most common solution for non-invasive body tracking in the context of interactive art. They capture depth data, i.e. the distance between a person and an optical sensor, while retaining the body’s “skeleton” structure. For example, Leap Motion [Mot] works with a limb scale, making it suitable for the use of precise hand-guided interaction. Intel’s RealSense camera [Int] features facial recognition data, which allows for the exploration of facial expression as a means of interaction, among other possible applications. Due to cost accessibility and reliable full-body scale, the most popular camera used in this context is Microsoft’s Kinect [How]. The capability of capturing the whole body’s structure (head, torso, limbs) in three dimensions, while not totally precise in some settings, has made it the most widely used sensor in interactive installations. It can also track multiple people simultaneously, which makes it suitable for utilization in multi-user installations. Obermaier’s, D’Alessio’s and Menegon’s EGO installation [Exi] makes use of the possibilities of full body-scale 3D camera detection, approaching the participants’ bodies in the physical world as a means of artistic expression in the digital realm. As shown in Figure 1, Bongers [Bon00] defines the interaction between humans and a computational system as follows: the two agents of interaction (a human body and a computer) each perform input and output functions simultaneously. The human body captures the physical reality through its senses (e.g., hearing, vision, and touch) and acts on the physical space through its effectors (e.g., muscle action, kinetic energy, breath, and speech). On the other hand, the computer system uses sensors to capture the existent physical signals, and its output is achieved through the use of actuators (e.g., any device able to emit physical signals such as light, sound, and kinetic energy). The interaction sequence happens in a continuous loop in which the output element on one end is sending signals to the input capabilities on the other, then inverting the course of action and sending its own signals. Both the human and computer use memory and cognition skills to process the received signals and guide their own subsequent behavior.
3. Limitations in Interactive Art: Inhibition as Interaction Blockage Public exhibition spaces such as museums and art galleries are the most common places in which interactive artworks are exhibited [SHSHB13]. As opposed to purely traditional contemplative art, digitally-mediated interaction requires the participant to physically act to enhance the artwork’s aesthetic properties and obtain c 2016 The Author(s)
c 2016 The Eurographics Association. Eurographics Proceedings
N. Cabrita & G. Bernardes / Relational interactive art: A framework for interaction in a social context
4. Relational Aesthetics ˘ ˘ ˙I to Bourriaud [Bou02] coined the term âAIJrelational aestheticsâA denote art which incites interpersonal relations. He draws a theory in which the artist acts as a “curator” of the aesthetic experience, thus redefining the role of the author. This is best shown in “Untitled (Free)” by Rirkit Tiravanija [MOM12], in which a group of people around a dining table gather at a gallery to share a meal cooked served by the artist. This setting promotes dialogue among the participants, as well as interactions with the artist. Furthermore, it leaves room for the unexpected to shape the nature of the artwork over time.
Figure 1: Human-Machine Interaction Diagram [Bon00].
artistic meaning from them [SHSHB13]. The public nature of exhibition spaces creates awareness of the participant’s relations to the surrounding space, and particularly to other individuals. We now, we discuss some problems in interactive art preventing the participants from fully engaging with an artwork, which we then analyze through the lens of group dynamics in order to fully comprehend these limitations and therefore reflect on possible ways to overcome them.
Relationships established among people in art exhibition spaces are fundamental to relational aesthetics. Bourriaud reflects on how the modern communication channels (“communication superhighways” in his own terms) have significantly shaped the relationships between humans, and how it turns ‘users into consumers of miles and their by-products” [Bou02]. The possibility for this type of relationships is referred to as the “social interstice” [Bou02], meaning that there is a dimension in which one can escape the “mechanization of social functions” [Bou02]. This notion seems to draw its roots from Debord’s critique of repressive capitalist production [Deb95], defending the lack of participation as a form of dehumanizing society. Art exhibition spaces are then seen as a state of encounter, whose rhythm differs from that of day-to-day life, where the relationship with the other can take place and flourish.
Visitors in a public exhibition space can find themselves abstaining from interacting with an artwork for various reasons. They can feel uncomfortable because they don’t know exactly what their actions should be in relation to the artwork (possibly based on the lack of existing interaction instructions), thus they are hesitant to try it having watched others fail to successfully interact. They can be afraid to be perceived as incompetent social actors [SHSHB13]. The lack of direct engagement with the artwork can also be explained by a conscious choice to purely observe others interacting with the artwork. By abstaining from interacting with an interactive artwork, the fruition of the aesthetic experience is done indirectly, through a third party who might have already been present when they arrived at the exhibition space. The physical intensity of the interactions is another factor that can cause inhibition: upon understanding that vigorous body movements or vocal utterances are the appropriate ways to interact with an artwork, visitors might decide they are not willing to participate because it makes them uncomfortable to perform such bold actions in unfamiliar surroundings. These hypotheses are not mutually exclusive; they can be evoked simultaneously to justify a lack of participation.
5. Towards a Framework for Relational Interactive Art
Group dynamics are at play in an interactive artwork exhibition. The exhibition space becomes a more comfortable setting for visitors when in the presence of acquaintances. This might trigger collective effort to discover the interaction mechanisms of the artwork, thus forming a common goal, which in itself makes the inhibition to disappear in favor of a more carefree attitude [SHSHB13]. On the other hand, individual visitors might be more likely to become self-conscious in an unfamiliar setting and thus restrain from interacting.
Based on the human-computer interaction dynamics suggested in Figure 1, we now discuss the extent by which human-human interaction shapes the flux of information of an interactive artwork. First, the presence of more than one person is pivotal to the very idea of relational art. Therefore, it’s assumed that in the absence of any of these elements the artwork ceases to exist in its full aesthetic dimension, because the system’s behavior is determined by all elements being present. The actions of people are interdependent, meaning that they are defined by the computer system’s behavior
c 2016 The Author(s)
c 2016 The Eurographics Association. Eurographics Proceedings
Physical presence is inherent to the social dimension in Bourriaud’s concept of relational aesthetics [Bou02], and therefore key to interpersonal relationships that might occur within a relational artwork. By adding an interactive dimension to relational art, we add a computation system in the relational loop mediating interpersonal relationships. In this intricate feedback loop, (shown in Figure 2), we must additionally account for behavior aspects in the variables regulating the participant’s physical presence. We now describe the main principles that shape these interaction dynamics, in an effort to establish a framework for the conception and execution of interactive relational artworks. To a certain extent the framework detailed here has many points in contact to Snibbe’s and Raffle’s concept of social immersive media [SR09], which highlights inherent social context present in interactive art. Yet, we further extend this concept by proposing an interaction model that establishes behavioral relationships between the computational system and the human element, instead of focusing on the specific narrative structure of an artwork.
N. Cabrita & G. Bernardes / Relational interactive art: A framework for interaction in a social context
interactive art. By following these five principles we acknowledge the foundations needed to build relational interactive artworks, allowing the establishment of social bonds between participants. This is accomplished by their interaction with the computational system and with one another. 6. Discussion and Conclusions
Figure 2: Relational Interaction Diagram (adapted from [Bon00]).
in relation to the interpersonal relations between humans (established either spontaneously or driven by the computational system response). These actions end up being interpreted by the system as a collaborative behavior, which guides its response and shapes the next instance of human action. In light of these relational interaction dynamics, the contactless tracking devices mentioned in Section 2.2 are ideal to enhance these relationships in a transparent way, without disrupting the continuous flux of information between human effectors and computational sensors. While being non-invasive, they are intrinsically connected to the sheer existence of the human body in physical space, which is the minimum requirement for the emergence of a social context. To the core computational system, its sensors provide valuable information that shapes the digital response of the artwork by data extracted from the physical world. This data is defined as the number of people detected in a confined space, the proximity between participants as well as their body structure, orientation, movement speed, and direction. Key principles that should be guaranteed in relational interactions are now reframed from Snibbe’s and Raffle’s concept of social immersive media [SR09]: i) continuously scalable: the artwork has an evident response to the participants’ actions, and the recursive nature of the interaction loop makes it build on top of its own behaviors; ii) socially scalable: the interaction must be shared between multiple participants (“if the exhibit fits more than one person, it must work with more than one person” [SR09]); iii) socially familiar: it augments and reinforces existing social behaviors through the mere presence of the body in space and its regular actions in an exhibition setting; iv) socially balanced: the interaction enhances the participant’s awareness of space, as well as their social relationship with others. To these principles, we add the need to view the humancomputer and human-human relationships as equally important and always present in the interaction loop, which is unique to relational
In the theoretical field of art criticism, the association of interactive and relational art is not a significantly explored issue. Although Bourriaud created the concept of relational aesthetics [Bou02], he didn’t specifically link it to the context in which artworks are exhibited. Technology is not mentioned, and perhaps this has to do with what he perceived to be its inability, at the time, to put in evidence the relationship between various people in an exhibition setting. More recently, Massumi [Mas11] and Bishop [Bis06] revealed themselves as very skeptical about the existence of interactive art in the relational realm. Massumi addresses a continuous creation potential between inter-given subjects as an essential feature of the relational possibilities of art, stating that interactive artworks merely promote a “going back and forth between actions, largely reduced to instrumental function” [Mas11] in a closed and clearly defined system. This turns participants into instrumentalized actors, serving their purpose only as players of a game with its own goals. On the other hand, Bishop states that there is only an “activation of the individual viewer” [Bis06] in relation to the interactive artwork, leaving the social dimension out of the equation in this setting. Nathaniel Stern [Ste13] states that this is a sign of a common misinterpretation from the theorist sphere towards interactive artistic expression. He argues that digital existence is addressed through the exploration of the body in a context where physical and virtual dimensions meet as art form. This has a social dimension, and even an emotional one: movement, sensation and affect can and are shown in this kind of artistic work, while interrogating “continuity, being and becoming” [Ste13] through its exploratory and temporary nature. The framework described in this paper is proposed as a means of guiding the practice of artists who are interested in developing interactive artworks that serve the purpose of establishing social bonds between participants in the interaction moment, through the use of non-invasive embodiment sensing methods. By acknowledging the inherent social context of interactive artwork exhibition, relational interactive art might enhance participant engagement with the artworks and overcome the interaction limitations encountered in this setting. Acknowledgments This research is financed by National Funds through the FCT - Portuguese Science Foundation post-doctoral grant SFRH/BPD/109457/2015. References [AS03] A SCOTT R., S HANKEN E. A.: Telematic embrace: Visionary theories of art, technology, and consciousness. Univ. of California Press, 2003. 2 c 2016 The Author(s)
c 2016 The Eurographics Association. Eurographics Proceedings
N. Cabrita & G. Bernardes / Relational interactive art: A framework for interaction in a social context [Bis06]
B ISHOP C.: Participation. The MIT Press, 2006. 4
[Bon00] B ONGERS B.: Physical interfaces in the electronic arts. Trends in gestural control of music (2000), 41–70. 2, 3, 4 [Bou02] B OURRIAUD N.: Relational Aesthetics. Les presses du reel, 2002. 1, 3, 4 [BV07] B ONGERS B., V EER G. C. V. D .: Towards a multimodal interaction space: categorisation and applications. Personal and Ubiquitous Computing 11, 7 (2007), 609–619. 2 [Deb95]
D EBORD G.: The society of the spectacle. Zone Books, 1995. 3
[Exi] E XILE: Ego Klaus Obermeier. URL: http://www.exile.at/ ego/. 2 [How] H OW S TUFF W ORKS: How microsoft kinect works. URL: http://electronics.howstuffworks.com/ microsoft-kinect.htm. 2 [Int] I NTEL: Intel RealSenseTM Technology. URL: https://www. leapmotion.com/. 2 [Klu10] K LUSZCZYNSKI R. W.: Strategies of interactive art. Journal of Aesthetics & Culture 2 (2010). 2 [LH] L OZANO -H EMMER: Rafael Lozano–Hemmer – Project ”Body Movies”. URL: http://www.lozano-hemmer.com/body_ movies.php. 2 [Mas11]
M ASSUMI B.: Into the diagram. ArtSpace, 2011. 4
[MOM12] MOMA: Rirkrit tiravanija: Cooking up an art experience, 2012. URL: http://www. moma.org/explore/inside_out/2012/02/03/ rirkrit-tiravanija-cooking-up-an-art-experience. 1, 3 [Mot] M OTION L.: Leap Motion | Mac and pc motion controller for games, design, virtual reality and more. URL: https://www. leapmotion.com/. 2 [MPS96] M ERLEAU -P ONTY M., S MITH C.: Phenomenology of perception. Motilal Banarsidass Publisher, 1996. 2 [Pen96] P ENNY S.: From a to d and back again: The emerging aesthetics of interactive art. Leonardo Electronic Almanac 4, 4 (1996), 4–7. 1, 2 [SHSHB13] S COTT S., H INTON -S MITH T., H ARMA V., B ROOME K.: Goffman in the gallery: Interactive art and visitor shyness. Symbolic Interaction 36, 4 (2013), 417–438. 1, 2, 3 [SR09] S NIBBE S. S., R AFFLE H. S.: Social immersive media–Pursuing best practices for multi-user interactive camera/projector exhibits. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2009). 2, 3, 4 [Ste13] S TERN N.: Interactive art and embodiment– The implicit body as performance. Gylphi Limited, 2013. 4 [Wal11] WALSH R.: Emergent narrative in interactive media. Narrative 19, 1 (2011). 2
c 2016 The Author(s)
c 2016 The Eurographics Association. Eurographics Proceedings
