entertainment Archives | Page 2 of 3

Summary So Far

In summary, Mobile AR has many paths leading to it. It is this convergence of various paths that makes a true historical appraisal of this technology difficult to achieve. However, I have highlighted facets of its contributing technologies that assist in the developing picture of the implications that Mobile AR has in store. A hybridisation of a number of different technologies, Mobile AR embodies the most gainful properties of its three core technologies: This analyst sees Mobile AR as a logical progression from VR, but recognises its ideological rather than technological founding. The hardware basis of Mobile AR stems from current mobile telephony trends that exploit the growing capabilities of Smartphone devices. The VR philosophy and the mobile technology are fused through the Internet, the means for enabling context-based, live-updating content, and housing databases of developer-built and user-generated digital objects and elements, whilst connecting users across the world.

I have shown that where the interest in VR technologies dwindled due to its limited real-world applicability, Mobile Internet also lacks in comparison to Mobile AR and its massive scope for intuitive, immersive and realistic interpretations of digital information. Wearable AR computing shares VR’s weaknesses, despite keeping the user firmly grounded in physical reality. Mobile AR offers a solution that places the power of these complex systems into a mobile telephone: the ubiquitous technology of our generation. This new platform solves several problems at once, most importantly for AR developers and interested Blue-chip parties, market readiness. Developing for Mobile AR is simply the commercially sensible thing to do, since the related industries are already making the changes required for its mass-distribution.

Like most nascent technologies, AR’s success depends on its commercial viability and financial investment, thus most sensible commercial developers of AR technologies are working on projects for the entertainment and advertising industries, where their efforts can be rewarded quickly. These small-scale projects are often simple in concept, easily grasped and thus not easily forgotten. I claim here that the first Mobile AR releases will generate early interest in the technology and entertainment markets, with the effect that press reportage and word-of-mouth behaviour assist Mobile AR’s uptake. I must be careful with my claims here however, since there is no empirical evidence to suggest that this will occur for Mobile AR. Looking at the emergence of previous technologies, however, the Internet and mobile telephony grew rapidly and to massive commercial success thanks to some strong business models and advancements in their own supporting technologies. It is strongly hoped by developers like Gameware and T-Immersion that Mobile AR can enjoy this same rapid lift-off. Both technologies gained prominence once visible in the markets thanks to a market segment called early adopters. This important group gathers their information from specialist magazine sources and word of mouth. Mobile AR developers would do well to recognise the power of this group, perhaps by offering shareware versions of their AR software that encourage a form of viral transmission that exploit text messaging.

Gameware have an interesting technique for the dissemination of their HARVEE software. They share a business interest with a Bluetooth technology firm, which has donated a prototype product the Bluetooth Push Box, which scans for local mobile devices and automatically sends files to users in acceptance. Gameware’s Push Box sends their latest demo to all visitors to their Cambridge office. This same technology could be placed in public places or commercial spaces to offer localised AR advertising, interactive tourist information, or 3D restaurant menus, perhaps.

Gameware, through its Nokia projects and HARVEE development program is well placed to gain exposure on the back of a market which is set to explode as mobile offerings become commercially viable, ‘social’, powerful, multipurpose and newsworthy. Projects like HARVEE are especially interesting in terms of their wide applicability and mass-market appeal. It is its potential as a revolutionary new medium that inspires this very series.

Mobile Telephone

The Internet and the mobile phone are two mighty forces that have bent contemporary culture and remade it in their form. They offer immediacy, connectivity, and social interaction of a wholly different kind. These are technologies that have brought profound changes to the ways academia consider technoscience and digital communication. Their relationship was of interest to academics in the early 1990’s, who declared that their inevitable fusion would be the beginning of the age of Ubiquitous Computing: “the shift away from computing which centered on desktop machines towards smaller multiple devices distributed throughout the space” (Weiser, 1991 in Manovich, 2006). In truth, it was the microprocessor and Moore’s Law- “the number of transistors that can be fit onto a square inch of silicon doubles every 12 months” (Stokes, 2003) that led to many of the technologies that fall under this term: laptops, PDA’s, Digital Cameras, flash memory sticks and MP3 players. Only recently have we seen mobile telephony take on the true properties of the Internet.

The HARVEE project is partially backed by Nokia Corp. which recognises its potential as a Mobile 2.0 technology: user-generated content for mobile telephony that exploits web-connectivity. Mobile 2.0 is an emerging technology thematically aligned with the better established Web 2.0. Nokia already refer to their higher-end devices as multimedia computers, rather than as mobile phones. Their next generation Smartphones will make heavy use of camera-handling systems, which is predicated on the importance of user-generated content as a means to promote social interaction. This strategic move is likely to realign Nokia Corp.’s position in the mobile telephony and entertainment markets.

Last year, more camera phones were sold than digital cameras (Future Image, 2006). Nokia have a 12 megapixel camera phone ready for release in 2009, and it will be packaged with a processing unit equal to the power of a Sony PSP (Nokia Finland: non-public product specification document). MP3 and movie players are now a standard on many handsets, stored on plug-in memory cards and viewed through increasingly higher resolution colour screens. There is a growing mobile gaming market, the fastest growing sector of the Games Industry (Entertainment & Leisure Software Publishers Association (ELSPA) sales chart). The modern mobile phone receives its information from wide-band GPRS networks allowing greater network coverage and faster data transfer. Phone calls are the primary function, but users are exploiting the multi-media capabilities of their devices in ways not previously considered. It is these factors, technologic, economic and infrastructural that provide the perfect arena for Mobile AR’s entry into play.

Mobile Internet is the natural convergence of mobile telephony and the World Wide Web, and is already a common feature of new mobile devices. Mobile Internet, I would argue, is another path leading to Mobile AR, driven by mobile users demanding more from their handsets. Mobile 2.0 is the logical development of this technology- placing the power of location-based, user-generated content into a new real-world context. Google Maps Mobile is one such application that uses network triangulation and its own Google Maps technologies to offer information, directions, restaurant reviews or even satellite images of your current location- anywhere in the world. Mobile AR could achieve this same omniscience (omnipresence?) given the recent precedent for massively multi-user collaborative projects such as Wikipedia, Flickr and Google Maps itself. These are essentially commercially built infrastructures designed to be filled with everybody’s tags, comments or other content. Mobile AR could attract this same amount of devotion if it offered such an infrastructure and real-world appeal.

There is a growing emphasis on Ubiquitous Computing devices in our time-precious world, signified by the increased sales in Smartphones and WiFi enabled laptops. Perhaps not surprisingly, Mobile Internet use has increased as users’ devices become capable of greater connectivity. Indeed, the mobile connected device is becoming the ubiquitous medium of modernity, as yet more media converge in it. It is the mobile platform’s suitability to perform certain tasks that Mobile AR can take advantage of, locating itself in the niche currently occupied by Mobile Internet. Returning to my Mixed Reality Scale, Mobile AR serves the user better than Mobile Internet currently can: providing just enough reality to exploit virtuality, Mobile AR keeps the user necessarily grounded in their physical environment as they manipulate digital elements useful to their daily lives.

The Internet

The Internet, or specifically the World Wide Web, requires a limited virtuality in order to do its job. The shallow immersion offered to us by our computer screens actually serves our needs very well, since the Internet’s role in our lives is to connect, store and present information in accessible, searchable, scannable, and consistent form for millions of users to access simultaneously, to be dived in and out of quickly or to surround ourselves in the information we want. The naturally-immersive VR takes us partway towards Mobile AR, but its influence stops at the (admittedly profound) concept of real-time interaction with 3D digital images. What the Internet does is bring information to us, but VR forces us to go to it.

This is a function of the Mixed Reality Scale, and the distance of each from The Real. The closer we can bring artefacts from The Virtual to The Real, the more applicable these can be in our everyday lives. The self-sufficient realm of The Virtual does not require grounding in physical reality in order to exist, whereas the Internet and other MR media depend on The Real to operate. AR is the furthest that a virtual object can be ‘stitched into’ our reality, and in doing so we exploit our power in this realm to manipulate and interact with these digital elements to suit our own ends, as we currently do with the World Wide Web.

The wide-ranging entertainment resources offered by the Internet are having a profound effect on real-world businesses, a state of flux that Mobile AR could potentially exploit. There is a shift in the needs of consumers of late that is forcing a change in the ways that many blue-chip organisations are handling their businesses: Mobile data carriers (operators), portals, publishers, content owners and broadcasters are all seeking new content types to face up to the threat of VOIP (Voice Over Internet Protocol) – which is reducing voice traffic; and Web TV/ Internet – reducing (reduced?) TV audiences, particularly in the youth market.

T-Mobile, for example, seeks to improve on revenues through offering unique licensed mobile games, themes, ringtones and video-clips on their T-Zones Mobile Internet Portal; NBC’s hit-series ‘Heroes’ is the most downloaded show on the Internet, forcing NBC to offer exclusive online comics on their webpage, seeking to recoup advertising revenue losses through lacing the pages of these comics with advertising. Mobile AR represents a fresh landscape for these businesses to mine. It is no surprise, then, that some forward-thinking AR developers are already writing software specifically for the display of virtual advertisement billboards in built-up city areas (T-Immersion).

The Internet has changed the way we receive information about the world around us. This hyper-medium has swallowed the world’s information and media content, whilst continuing to enable the development of new and exciting offerings exclusive to the desktop user. The computing capacity required to use the Internet has in the past constrained the medium to the desktop computer, but in the ‘Information Age’ the World Wide Web is just that: World Wide.

Gameware: A Case-Study in AR Development

I have been aided in this series by a connection with Gameware Development Limited, a Cambridge-based commercial enterprise working in the entertainment industry. Gameware was formed in May 2003 from Creature Labs Ltd, developing for the PC games market which produced the market leading game in Artificial Intelligence (AI), Creatures. When Gameware was formed, a strategic decision was made to move away from retail products and into the provision of technical services. They now work within the Broadcasting and Mobile Telephony space in addition to the traditional PC market. I use this business as a platform to launch into a discussion of the developments current and past that could see AR become a part of contemporary life, and just why AR is such a promising technology.

Gameware’s first explorations into AR came when they were commissioned by the BBC to develop an AR engine and software toolkit for a television show to be aired on the CBBC channel. The toolkit lets children build virtual creatures or zooks at home on their PCs which are uploaded back to the BBC and assessed:

A typical Zook, screenshot taken from Gameware's Zook Kit which lets children build virtual creatures

The children with the best designs are then invited to the BAMZOOKi studio to have their virtual creatures compete against each other in a purpose-built arena comprised of real and digital elements. The zooks themselves are not real, of course, but the children can see silhouettes of digital action projected onto the arena in front of them. Each camera has an auxiliary camera pointed at AR markers on the studio ceiling, meaning each camera’s exact location in relation to the simulated events can be processed in real time. The digital creatures are stitched into the footage, and are then navigable and zoomable as if they were real studio elements. No post-production is necessary. BAMZOOKi is currently in its fourth series, with repeats aired daily:

BAMZOOKi, BBC's AR game show where children’s zooks compete in a studio environment

BAMZOOKi has earned Childrens BBC some of its highest viewing figures (up to 1.2 million for the Monday shows on BBC1 and around 100,000 for each of the 20 episodes shown on digital Children’s BBC), which represents a massive milestone for AR and its emergence as a mainstream media technology. The evidence shows that there is a willing audience already receptive to contemporary AR applications. Further to the viewing figures the commercial arm of the BBC, BBC Worldwide, is in talks to distribute the BAMZOOKi format across the world, with its AR engine as its biggest USP. Gameware hold the rights required to further develop their BAMZOOKi intellectual property (IP), and are currently working on a stripped down version of their complex AR engine for the mobile telephony market.

I argue, however, that Broadcast AR is not the central application of AR technologies, merely an enabler for its wider applicability in other, more potent forms of media. Mobile AR offers a new channel of distribution for a variety of media forms, and it is its flexibility as a platform that could see it become a mainstream medium. Its successful deployment and reception is reliant on a number of cooperating factors; the innovation of its developers and the quality of the actual product being just part of the overall success the imminent release.

As well as their AR research, Gameware creates innovative digital games based on their Creatures AI engine. They recently produced Creebies; a digital game for Nokia Corp. Creebies is one of the first 3D games which incorporates AI for mobile phones. Gameware’s relationship with Nokia was strengthened when Nokia named them Pro-Developers. This is a title that grants Gameware a certain advantage: access to prototype mobile devices, hardware specifications, programming tools and their own Symbian operating system (Symbian OS) for mobile platforms. It was this development in combination with their experiences with BAMZOOKi and a long-standing collaboration with Cambridge University which has led to the idea for their HARVEE project. HARVEE stands for Handheld Augmented Reality Virtual Entertainment Engine.

Their product allows full 3D virtual objects to co-exist with real objects in physical space, viewed through the AR Device, which are animated, interactive and navigable, meaning the software can make changes to the objects as required, providing much space for interesting digital content. The applications of such a tool range from simple toy products; advertising outlets; tourist information or multiplayer game applications; to complex visualisations of weather movements; collaborating on engineering or architectural problems; or even implementing massive city-wide databases of knowledge where users might ‘tag’ buildings with their own graphical labels that might be useful to other AR users. There is rich potential here.

In HARVEE, Gameware attempt to surmount the limitations of current AR hardware in order to deliver the latest in interactive reality imaging to a new and potentially huge user base. Indeed, Nokia’s own market research suggests that AR-capable Smartphones will be owned by 25% of all consumers by 2009 (Nokia Research Centre Cambridge, non-public document). Mobile AR of the type HARVEE hopes to achieve represents not only a significant technical challenge, but also a potentially revolutionary step in mobile telephony technologies and the entertainment industry.

Gameware’s HARVEE project is essentially the creation of an SDK (Software Development Kit) which will allow developers to create content deliverable via their own Mobile AR applications. The SDK is written with the developer in mind, and does the difficult work of augmenting images and information related to the content. This simple yet flexible approach opens up a space for various types of AR content created at low cost for developers and end-users. I see Mobile AR’s visibility on the open market the only impediment to its success, and I believe that its simplicity of concept could see it become a participatory mass-medium of user-generated and mainstream commercial content.

Introduction

Augmented Reality (AR) is a theme of computer research which deals with a combination of real world and computer generated data. AR is just one version of a Mixed Reality (MR) technology, where digital and real elements are mixed to create meaning. In essence AR is any live image that has an overlay of information that augments the meaning of these images.

Digital graphics are commonly put to work in the entertainment industry, and ‘mixing realities’ is a common motif for many of today’s media forms. There are varying degrees to which The Real and The Virtual can be combined. This is illustrated in my Mixed Reality Scale:

mixed-reality-scale — My Mixed Reality Scale, a simplified version of Milgram & Kishino’s (1994) Virtuality Continuum

This is a simplified version of Milgram and Kishino’s (1994) Virtuality Continuum; simplified, because their research is purely scientific, without an explicit interest in media theory or effects, therefore not wholly applicable to my analysis. At the far left of my Mixed Reality Scale lies The Real, or physical, every-day experiential reality. For the longest time we lived solely in this realm. Then, technological innovation gave rise to the cinema, and then television. These media are located one step removed from The Real, a step closer to The Virtual, and can be considered a window on another world. This world is visually similar to our own, a fact exploited by its author to narrate believable, somewhat immersive stories. If willing, the viewer is somewhat ‘removed’ from their grounding here in physical reality, allowing them to participate in the construction of a sculpted, yet static existence. The viewer can only observe this contained reality, and cannot interact with it, a function of the viewing apparatus.

Later advancements in screen media technologies allowed the superimposition of graphical information over moving images. These were the beginnings of AR, whereby most of what is seen is real with some digital elements supplementing the image. Indeed, this simple form of AR is still in wide use today, notably in cases where extra information is required to make sense of a subject. In the case of certain televised sports, for example, a clock and a scoreboard overlay a live football match, which provides additional information that is useful to the viewer. Television viewers are already accustomed to using information that is displayed in this way:

Simple Augmented Reality, televised football matches augment meaning with digital graphics

More recently, computing and graphical power gave designers the tools to build wholly virtual environments. The Virtual is a graphical representation of raw data, and the furthest removed from physical reality on my Mixed Reality Scale. Here lies the domain of Virtual Reality (VR), a technology that uses no real elements except for the user’s human senses. The user is submersed in a seemingly separate reality, where visual, acoustic and sometimes haptic feedback serve to transpose them into this artificial, yet highly immersive space. Notice the shift from viewer to user: this is a function of the interactivity offered by digital space. VR was the forerunner to current AR research, and remains an active realm of academic study.

Computer graphics also enhanced the possibilities offered by television and cinema, forging a new point on the Mixed Reality Scale. I refer to the Augmented Virtuality (AV) approach, which uses mainly digital graphics with some real elements superimposed. For example, a newsreader reporting from a virtual studio environment is one common application. I position AV one step closer towards The Virtual to reflect the ratio of real to virtual elements:

An Augmented Virtuality, the ITV newscasters sit at a real table in a virtual studio

There is an expansive realm between AV and VR technologies, media which offer the user wholly virtual constructions that hold potential for immersion and interactivity. I refer to the media of video games and desktop computers. Here the user manipulates visually depicted information for a purpose. These media are diametrically opposed to their counterpart on my scale, the cinema and television, because they are windows this time into a virtual world, actively encouraging (rather than denying) user interactivity to perform their function. Though operating in virtuality, the user remains grounded in The Real due to apparatus constraints.

Now, further technological advancements allow the fusion of real and virtual elements in ways not previously possible. Having traversed our way from The Real to The Virtual, we have now begun to make our way back. We are making a return to Augmented Reality, taking with us the knowledge to manipulate wholly virtual 3D objects and the computing power to integrate digital information into live, real world imagery. AR is deservedly close to The Real on my scale, because it requires physicality to function. This exciting new medium has the potential to change the way we perceive our world, forging a closer integration between our two binary worlds. It is this potential as an exciting and entirely new medium that has driven me to carry out the following work.

To begin, I address the science behind AR and its current applications. Next, I exploit an industry connection to inform a discussion of AR’s development as an entertainment medium. Then, I construct a methodology for analysis from previous academic thought on emergent technologies, whilst addressing the problems of doing so. I use this methodology to locate AR in its wider technologic, academic, social and economic context. This discussion opens ground for a deeper analysis of AR’s potential socio-cultural impact, which makes use of theories of media and communication and spatial enquiry. I conclude with a final critique that holds implications for the further analysis of Mixed Reality technology.