Extending the boundaries of local television new production concepts and technologies for interactive services in an augmented public space Author, affiliation, etc. intentionally left blank Submission to the doctoral colloquium.. ABSTRACT

In my doctoral dissertation I aim to extend the boundaries for what today is called local television. The object of study is a local TV station in Helsinki which will be extended with new technologies thatconvert the city physically into an interactive media interface. The inhabitants of the city will be able to mediate their political opinions and at the visitors will be guided through the city through this interactive, content aware ubiquitous system. The system will be developed and ready for the year 2012 when Helsinki is the World Design Capital. Supervisor: professor Stefan University Helsinki – Finland.

Sonvilla-Weiss,

Aalto

Author keywords

Citizen journalism, context awareness, social video, visual markers, user experience, ubiquitous computing, iTV. ACM Classification Keywords

H5.2 User Interfaces Theory and methods. 1.INTRODUCTION

During the last decades many new technologies have emerged that can be used to radically alter the traditional (technological) notion of what we understand by television. Some of the more important ones being the mobile phones and tablets, camcorders, visual markers, augmented reality applications, YouTube, and interactive multiuser virtual worlds on the Internet. As the Internet expands to new terminals the possibilities for the citizen to mediate themselves through these new forms of media is growing fast. The implications of this paradigm shift, where large media houses are being challenged by individual media producers is interesting in that it opens up new possibilities for a truly democratic local media environment in the form of citizen generated content and DIY media production. (Figueiredo, 2009) However at the same time the concept of local television has in most countries remained just a traditional one way broadcasting service without frills and with content that largely goes unwatched and unnoticed, with little impact on the society.

In Helsinki, the current local TV channel will be relaunched on September 29, 2010. The new channel shall incorporate new and innovative technologies that allow for the people in Helsinki to mediate themselves and that let them participate in creating new interactive content that is context aware and connected to physical locations in the city. Herein lies my research problem, I’m responsible of developing these new services and technologies. 1.1 Research goals

In my research I will combine four related areas that together form a new citywide media platform that will extend the boundaries of traditional local TV. These areas are: (1) enabling technologies (visual markers, mobile phones, augmented reality technologies), (2) production system design (defining video metadata handling, building a broadcast backend), (3) production principles (best practices and new phone based production tools) and (4) programme concepts (new “TV” formats and multiplayer games). The systems will convert shared space physical objects (statues, buildings, trams, streets etc.) into video portals linking social video material and augmented reality based overlays to the object – thus creating an interactive media interface in the city. Once the technology works some new concepts for using these facilities will be developed and tested. In the empirical part of my work I will analyze and assess the way the users experience the media interface and how it will function as a means for the inhabitants to mediate their political opinions to the city municipals. 1.2 Relevance of research

From a media-technological perspective, the system created can be utilized as a new interactive citywide media platform both for locals and tourists, while at the same time also forming a base for sensory multiuser games and other media concepts. From a sociological view, the results will empower the potential citizen journalists of Helsinki and give them a

medium through which to express themselves towards the municipality.

qualified professional fields with regards to television, sociology, economy and ITC.

From pedagogic point of view the value lies in the resulting new didactical tools and the knowledge and inspiration they convey about the city and the people of Helsinki.

As a result of stage two I will have the specifications for the functionality of augmented local TV – which in turn allows to proceed to:

Finally the thesis generates proof of concept for a general citywide media infrastructure that could be replicated in other major capitals.

3.3 Stage 3: Constructing a model and a prototype

2. CURRENT AND PREVIOUS WORK

How should the broadcasting system be developed to allow for user generated content?

I’m currently in the stage of doing my literature review. I’m focusing on the work done in the following areas: visual markers, ubiquitous and pervasive computing, citizen journalism, ambient media, human centered design, user experience, news production., media ecology, postmodernism. I include some selected references below. 3. RESEARCH PROBLEM AND WORKFLOW

My primary research problem is: how can the boundaries of Local TV be extended using new media technologies? Why is this a problem? Because the current version of local TV is becoming obsolete, nobody watches it and within a few years it could disappear completely This question can be articulated in questions or stages:

six more specific

3.1 Stage 1: Ontology and epistemology

What is the form and nature of these new concepts in relation to local television – i.e. what aspects are we examining? (The ontological question). How can e.g. social video, ubiquity, visual markers, and other important concepts be understood in relation to what there is today: traditional TV and video on demand? How can the analytical relevance of these concepts be determined i.e. how can the process of DIY media production using new methods of production be operationalized in a way that allows the effects to be studied? (The epistemological question) What are the functional expectations: what could the notion of “augmented local television” be – if it could be all it could be? I.e. what are the non-articulated forecasts of what kind of TV people want? This stage leads to: 3.2 Stage 2: Expert opinions and predictions

What do media professionals think are the main desired qualitative attributes of augmented local TV? How do these interact with the traditional cable content? What does an ecological perspective allow us to understand about the impact of technological change on the economical and cultural aspects of normative television? These questions will be answered by conducting qualitative interviews in the form of focus group studies and in-depth interviews (grounded theory) with people from different

How can bar-codes and mobile terminals be interlinked to create an augmented version of local TV?

What are the packaging and metadata problems that need to be resolved? Once there is a prototype version that can be tested I will proceed to: 3.4 Stage 4: Pilot testing and gathering of empirical data.

How do digital natives and late adaptors compare in their view of the system functionality and how it conveys information? What kind of technical issues arise during the pilots? How can the feedback of the pilot results be used to improve on the model? This stage will hopefully be completed with a real implementation of the system. If the parts necessary to build this technology mash up aren’t available I will do the pilot test on a conceptual model and proceed to: 3.5 Stage 5: Analysing the results of the pilot test

Once the main empirical phase is completed. I will proceed to analysing the results using the methods described below. 3.6 Stage 6: SWAT analysis and conclusions

At this stage I should have the answer to the main question. I can now conclude and discuss further work by answering the following: What are the strengths, weaknesses, opportunities and threats to this new media system? I will then finish off with some forecasting from a media ecology pov: how will the introduced change in technology affect the economics and cultural influences of local TV? Here media ecology will be used as an assessment tool for studying the affects of the platform. Altogether, although these six stages present a multitude of questions – it should be feasible to answer most of them by breaking down the larger problem into these simpler subquerys In order to improve on my results I will most likely have to iterate the stages 3 and 4 a few times to develop the final version of both the conceptual and technological version of the system.

4. METHODS

I approach my research questions from an empirically holistic point of view. My principal methodology is that of pheonomenography as defined by Marton (1981). My motivation for the choice of method is that Phenomenography seeks to reveal understanding of a phenomena from the participant’s point of view rather than that of the researcher. The phenomenon is inseparably embedded in how the perceiver interacts with the environment – learns – and then re-evaluates his position in the matter (Marton, 1981) The empirical part of my work will be carried out through pheonomenographical studies including in-depth interviews, observations and as an important part – analysis of audiovisual material produced by the agents.

[10] Kawano, T., Ban, Y. & Uehara, K., 2003. A Coded Visual Marker for Video Tracking System Based on Structured Image Analysis. In ISMAR '03: Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality. Washington, DC, USA: IEEE Computer Society, p. 262. [11] Kishino, Y. et al., 2005. A Ubiquitous Computing Environment Composed by Cooperation between Visual Markers and Event-Driven Compact Devices. In UDM '05: Proceedings of the International Workshop on Ubiquitous Data Management. Washington, DC, USA: IEEE Computer [12] Klinenberg, E.: Convergence: News Production in a Digital Age The ANNALS of the American Academy of Political and Social Science 2005; 597; 48 [13] McLuhan, M.: Understanding Media: The Extensions of Man. New York: McGraw-Hill, 1964.

I will also use a case study. To allow me to purposefully sample the way people use the technologies developed – and assess how their learning experience develop during their contact with the system. Last but not least there is of course the developing of the prototype itself.

[14] Marton, F. (1981). Phenomenography: Describing conceptions of the world around us [online version]. Instructional Science, 10, 177-200. Retrieved March 11, 2010, from http://www.ped.gu.se/biorn/phgraph/misc/constr/phegraph.ht ml

5. REFERENCES [1] Aroyo, L. et al., 2007. Personalized ambient media experience: move.me case study. In Proceedings of the 12th international conference on Intelligent user interfaces. Honolulu, Hawaii, USA: ACM, pp. 298-301

[15] Ong, W.: Orality and Literacy: The Technologizing of the Word. London, New York: Methuen, 1982

[2] Bauman, Z.: Liquid Modernity. Cambridge: Polity,2000 [3] Cesar P., Dick C. A. Bulterman, and Luiz Fernando Gomes Soares, “Introduction to special issue: Human-centered television;directions in interactive digital television research,” ACM Trans. Multimedia Comput. Commun. Appl. 4, no. 4 (2008): 1-7. [4] Costanza, E. & Huang, J., 2009. Designable visual markers. In CHI '09: Proceedings of the 27th international conference on Human factors in computing systems. New York, NY, USA: ACM, pp. 1879–1888. [5] Dijk van, J. and and de Vos, L.: Searching for the Holy Grail: Images of Interactive Television, New Media Society, 2001. [6] Eriksson, E., Hansen, T.R. & Lykke-Olesen, A., 2007. Reclaiming public space: designing for public interaction with private devices. In Proceedings of the 1st international conference on Tangible and embedded interaction. Baton Rouge, Louisiana: ACM, pp. 31-38. [7] Figueiredo, M.A. et al., 2009. Empowering rural citizen journalism via web 2.0 technologies. In Proceedings of the fourth international conference on Communities and technologies. University Park, PA, USA: ACM, pp. 77-84. [8] Gere, C., Digital Culture, 2003. Reaktion books Ltd. [9] Jenkins, H., Convergence Culture: Where Old and New Media Collide. New York University Press 2006

[16] Postman, N.: Amusing Ourselves to Death: Public Discourse in the Age of Show Business. New York: Viking, 1985. [17] Roibás, A.C. et al., 2006. Mobile iTV: new challenges for the design of pervasive multimedia systems. In CHI '06 extended abstracts on Human factors in computing systems. Montréal, Québec, Canada: ACM, pp. 407-4 [18] Sonvilla-Weiss (ed), S.: (e)Pedagogy Р Visual Knowledge Building Rethinking Art and New Media in Education. Peter Lagn, 2005. [19] Storsul, T. and Stuedahl, D.: Ambivalence towards Convergence. Göteborg University, Nordicom 2007. [20] Trekreem, A.: Dako 2: Forskningsmetodik Del II: Kvalitativ ansats och metodik. Ѓbo Akademi. [21] Uljens, M. (1996). On the philosophical foundation of phenomenography. In G. Dall’Alba & B. Hasselgren (Ed.), Reflections on Phenomenography (pp. 105Р130). Goteborg: Acta Universitatis Gothenburgensis. [22] Zavoina, S. and Reichert, T. : Media Convergence/Management Change: The Evolving Workflow for Visual Journalists’, Journal of Media Economics 13(2): 2000;143Р51. [23] Zhang, X., Fronz, S. & Navab, N., 2002. Visual Marker Detection and Decoding in AR Systems: A Comparative Study. In ISMAR '02: Proceedings of the 1st International Symposium on Mixed and Augmented Reality. Washington, DC, USA: IEEE Computer Society, p. 97.