real world Archives | Digital Cortex

Music is the Virus

Airborne, a potentially disruptive start-up in the music sector, hopes to “cure the music industry of its sickness” with their upcoming launch.

Their cloud-based music sharing platform places fans and artists in direct symbiosis. It’s an interesting model, so take a look:

Beyond all the virus metaphors (they even go so far as to call songs ‘strains’) the core idea is quite simple:

Cut out traditional distributors
Enable artists to monetise via a system of micropayments
Give fans distribution rights instead, and empower them to share as much as possible, thus helping to secure further micropayments

It’s a model that I think could work particularly well for electronic music, whose artists tend to release one track or remix at a time, as opposed to a band who might release one album a year. Airborne will work best when artists can trickle content to their audience to keep them subscribed.

Looking on SoundCloud, my current favourite producer/DJ has 3,934 followers, which would net $3,934 per month on Airborne. Give those early adopting, high-class listeners some viral functionality and the impetus to share with friends and that figure could easily grow to $10,000/month – a healthy supplement to any unsigned musician, I’m sure you’ll agree.

Airborne have an interesting blog, The Music Industry is Sick, which looks at the challenges faced by listeners, musicians and labels today. In an ecology where artists need their stuff streamed four million times just to reach minimum wage, it’s platforms like Airborne that’ll help the system fix itself.

Applying Benjamin

AR technology grants virtual objects presence in physicality. This is a concept ripe with potential for academic study. Baudrillardian thought states that we would seek to assign these objects similar values to other, real world objects. The School of Economy offers the truth that scarcity creates value. Our physical, tangible world is finite, but The Virtual is infinite. Now, any object in space denies the opportunity for another object to exist there, however we know this not to be true in an Augmented Reality. I proffer that virtual objects taking up space in a finite world hold economic value, but Benjamin might argue against this view.

In his piece The Work of Art in the Age of Mechanical Reproduction (1968) which was written in the thirties, Walter Benjamin argues that film and other ‘reproductive’ media diminished or destroyed the aura that had belonged to earlier art. As Bolter et al. (2006: 24) put it, “Aura belongs to works of art that are unique, as most art was before technologies of mechanical reproduction. Aura is the sense of the ‘here and now’ that each such work possesses because of its history of production and transmission. This uniqueness lends to each painting or sculpture a special quality, which can in turn evoke an attitude of reverence on the part of the viewer”. Benjamin (1968: 222-223) said that:

“We define the aura as the unique phenomenon of a distance, however close it may be. If, while resting on a summer afternoon, you follow with your eyes a mountain range on the horizon or a branch which casts its shadow over you, you experience the aura of those mountains, of that branch.”

Benjamin (1968: 222-223)

In this we see that Benjamin equates Aura with immediacy and the uniqueness of the moment of an artefact’s viewing. Since each moment is unique for each of us, it is economically valuable, auratic even. The information age of perfect digital reproduction does not detract from the personal uniqueness of experiential reality, only diminishes the aura offered by certain mass-produced elements within it. In Benjamin’s time, art was sanctified and extremely auratic. Now, most view classical art as a digital copy or encyclopaedia reference, which embodies little of the aura of the original piece. Travelling to the Louvre offers only the added auratic experience not gained from a home computer, where one’s experience is more like anybody else’s experience of the subject, but visitors must get very close to a works in order to experience what Aura remains there.

Benjamin might say that in a world of perfect digital reproduction, virtual objects can have no Aura. The Magic Lens is not spiritually void, however, and my Reality 2.0 is not a stagnant place. It buzzes with an aura of its own, borne from the uniqueness of each interaction, live and in context. A virtual object can be duplicated infinitely, but its location in space and the immediacy of its presence in that moment that helps shape its unique meaning for every AR denizen. Physicality offers the backdrop to the unique experiences offered by virtuality, indeed, experiential reality is improved through the wider opportunity for unique moments.

Returning to the economics of virtual space, virtual objects cannot hold currency in and of themselves, but the context surrounding them has value. Advertisers could exploit this if they act fast in commercialising AR space. More interesting to me is the potential for unique virtual art objects and sculptures that could hold purely aesthetic value. In this case, interested parties would seek out the object and view its intricacies and movements in the space it was designed to fill. There is much room for spatial enquiry in this field, some of which I will come onto in my next section.

Applying Benjamin

Applying McLuhan

I begin with McLuhan, whose Laws of Media or Tetrad offers greater insights for Mobile AR, sustaining and developing upon the arguments developed in my assessment of the interlinking technologies that meet in Mobile AR, whilst also providing the basis to address some of this man’s deeper thoughts.

The tetrad can be considered an observation lens to turn upon one’s subject technology. It assumes four processes take place during each iteration of a given medium. These processes are revealed as answers to these following questions, taken from Levinson (1999):

“What aspect of society or human life does it enhance or amplify? What aspect, in favour or high prominence before the arrival of the medium in question, does it eclipse or obsolesce? What does the medium retrieve or pull back into centre stage from the shadows of obsolescence? And what does the medium reverse or flip into when it has run its course or been developed to its fullest potential?”

(Digital Mcluhan 1999: 189).

To ask each of these it is useful to transfigure our concept of Mobile AR into a more workable and fluid term: the Magic Lens, a common expression in mixed reality research. Making this change allows the exploration of the more theoretical aspects of the technology free of its machinic nature, whilst integrating a necessary element of metaphor that will serve to illustrate my points.

To begin, what does the Magic Lens amplify? AR requires the recognition of a pre-programmed real-world image in order to augment the environment correctly. It is the user who locates this target, it is important to mention. It could be said that the Magic Lens more magnifies than amplifies an aspect of the user’s environment, because like other optical tools the user must point the device towards it and look through, the difference with this Magic Lens is that one aspect of its target, one potential meaning, is privileged over all others. An arbitrary black and white marker holds the potential to mean many things to many people, but viewed through an amplifying Magic Lens it means only what the program recognises and consequently superimposes.

This superimposition necessarily obscures what lies beneath. McLuhan might recognise this as an example of obsolescence. The Magic Lens privileges virtual over real imagery, and the act of augmentation leaves physical space somewhat redundant: augmenting one’s space makes it more virtual than real. The AR target undergoes amplification, becoming the necessary foundation of the augmented reality. What is obsolesced by the Magic Lens, then, is not the target which it obscures, but everything except the target.

I am reminded of McLuhan’s Extensions of Man (1962: 13), which offers the view that in extending ourselves through our tools, we auto-amputate the aspect we seek to extend. There is a striking parallel to be drawn with amplification and obsolescence, which becomes clear when we consider that in amplifying an aspect of physical reality through a tool, we are extending sight, sound and voice through the Magic Lens to communicate in wholly new ways using The Virtual as a conduit. This act obsolesces physical reality, the nullification effectively auto-amputating the user from their footing in The Real. So where have they ‘travelled’? The Magic Lens is a window into another reality, a mixed reality where real and virtual share space. In this age of Mixed Realities, the tetrad can reveal more than previously intended: new dimensions of human interaction.

The third question in the tetrad asks what the Magic Lens retrieves that was once lost. So much new ground is gained by this technology that it would be difficult to make a claim. However, I would not hold belief in Mobile AR’s success if I didn’t recognise the exhumed, as well as the novel benefits that it offers. The Magic Lens retrieves the everyday tactility and physicality of information engagement, that which was obsolesced by other screen media such as television, the Desktop PC and the games console. The Magic Lens encourages users to interact in physicality, not virtuality. The act of actually walking somewhere to find something out, or going to see someone to play with them is retrieved. Moreover, we retrieve the sense of control over our media input that was lost by these same technologies. Information is freed into the physical world, transfiguring its meaning and offering a greater degree of manipulative power. Mixed Reality can be seen only through the one-way-glass of the Magic Lens, The Virtual cannot spill through unless we allow it to. We have seen that certain mainstream media can wholly fold themselves into reality and become an annoyance- think Internet pop-ups and mobile ringtones- through the Magic Lens we retrieve personal agency to navigate our own experience. I earlier noted that “the closer we can bring artefacts from The Virtual to The Real, the more applicable these can be in our everyday lives”; a position that resonates with my growing argument that engaging with digital information through the Magic Lens is an appropriate way to integrate and indeed exploit The Virtual as a platform for the provision of communication, leisure and information applications.

It is hard to approximate what the Magic Lens might flip into, since at this point AR is a wave that has not yet crested. I might suggest that since the medium is constrained to success in its mobile device form, its trajectory is likely entwined with that medium. So, the Magic Lens flips into whatever the mobile multimedia computer flips into. Another possibility is that the Magic Lens inspires such commercial success and industrial investment that a surge in demand for Wearable Computers shifts AR into a new form. This time, the user cannot dip in and out of Mixed Reality as they see fit, they are immersed in it whenever they wear their visor. This has connotations all of its own, but I will not expound my own views given that much cultural change must first occur to implement such a drastic shift in consumer fashions and demands. A third way for the Magic Lens to ‘flip’ might be its wider application in other media. Developments in digital ink technologies; printable folding screens; ‘cloud’ computing; interactive projector displays; multi-input touch screen devices; automotive glassware and electronic product packaging could all take advantage of the AR treatment. We could end up living far more closely with The Virtual than previously possible.

In their work The Global Village, McLuhan and Powers (1989) state that:

“The tetrad performs the function of myth in that it compresses past, present, and future into one through the power of simultaneity. The tetrad illuminates the borderline between acoustic and visual space as an arena of the spiralling repetition and replay, both of input and feedback, interlace and interface in the area of imploded circle of rebirth and metamorphosis”

(The Global Village 1989: 9)

I would be interested to hear their view on the unique “simultaneity” offered by the Magic Lens, or indeed the “metamorphosis” it would inspire, but I would argue that when applied from a Mixed Reality inter-media perspective, their outlook seems constrained to the stringent and self-involved rules of their own epistemology. Though he would be loath to admit it, Baudrillard took on McLuhan’s work as the basis of his own (Genosko, 1999; Kellner, date unknown), and made it relevant to the postmodern era. His work is cited by many academics seeking to forge a relationship to Virtual Reality in their research…

What is AR and What is it Capable Of?

Presently, most AR research is concerned with live video imagery and it’s processing, which allows the addition of live-rendered 3D digital images. This new augmented reality is viewable through a suitably equipped device, which incorporates a camera, a screen and a CPU capable of running specially developed software. This software is written by specialist software programmers, with knowledge of optics, 3D-image rendering, screen design and human interfaces. The work is time consuming and difficult, but since there is little competition in this field, the rare breakthroughs that do occur are as a result of capital investment: something not willingly given to developers of such a nascent technology.

What is exciting about AR research is that once the work is done, its potential is immediately seen, since in essence it is a very simple concept. All that is required from the user is their AR device and a real world target. The target is an object in the real world environment that the software is trained to identify. Typically, these are specially designed black and white cards known as markers:

An AR marker, this one relates to a 3D model of Doctor Who's Tardis in Gameware's HARVEE kit

These assist the recognition software in judging viewing altitude, distance and angle. Upon identification of a marker, the software will project or superimpose a virtual object or graphical overlay above the target, which becomes viewable on the screen of the AR device. As the device moves, the digital object orients in relation to the target in real-time:

armarker2 — Augmented Reality in action, multiple markers in use on the HARVEE system on a Nokia N73

The goal of some AR research is to free devices from markers, to teach AR devices to make judgements about spatial movements without fixed reference points. This is the cutting edge of AR research: markerless tracking. Most contemporary research, however, uses either marker-based or GPS information to process an environment.

Marker-based tracking is suited to local AR on a small scale, such as the Invisible Train Project (Wagner et al., 2005) in which players collaboratively keep virtual trains from colliding on a real world toy train track, making changes using their touch-screen handheld computers:

crw_80271 — The Invisible Train Project (Wagner et al., 2005)

GPS tracking is best applied to large scale AR projects, such as ARQuake (Thomas et al, 2000), which exploits a scale virtual model of the University of Adelaide and a modified Quake engine to place on-campus players into a ‘first-person-shooter’. This application employs use of a headset, wearable computer, and a digital compass, which offer the effect that enemies appear to walk the corridors and ‘hide’ around corners. Players shoot with a motion-sensing arcade gun, but the overall effect is quite crude:

100-0007_img_21 — ARQuake (Thomas et al, 2000)

More data input would make the game run smoother and would provide a more immersive player experience. The best applications of AR will exploit multiple data inputs, so that large-scale applications might have the precision of marker-based applications whilst remaining location-aware.

Readers of this blog will be aware that AR’s flexibility as a platform lends applicability to a huge range of fields:

Current academic work uses AR to treat neurological conditions: AR-enabled projections have successfully cured cockroach phobia in some patients (Botella et al., 2005);
There are a wide range of civic and architectural uses: Roberts et al. (2002) have developed AR software that enables engineers to observe the locations of underground pipes and wires in situ, without the need schematics
AR offers a potentially rich resource to the tourism industry: the Virtuoso project (Wagner et al., 2005) is a handheld computer program that guides visitors around an AR enabled gallery, providing additional aural and visual information suited to each artefact;

The first commercial work in the AR space was far more playful, however: AR development in media presentations for television has led to such primetime projects as Time Commanders (Lion TV for BBC2, 2003-2005) in which contestants oversee an AR-enabled battlefield, and strategise to defeat the opposing army, and FightBox (Bomb Productions for BBC2, 2003) in which players build avatars to compete in an AR ‘beat-em-up’ that is filmed in front of a live audience; T-Immersion (2003- ) produce interactive visual installations for theme parks and trade expositions; other work is much more simple, in one case the BBC commissioned an AR remote-control virtual Dalek meant for mobile phones, due for free download from BBC Online:

A Dalek, screenshot taken from HARVEE's development platform (work in progress)

The next entry in this series is a case study in AR development. If you haven’t already done so, please follow me on Twitter or grab an RSS feed to be alerted when my series continues.

Introduction

Augmented Reality (AR) is a theme of computer research which deals with a combination of real world and computer generated data. AR is just one version of a Mixed Reality (MR) technology, where digital and real elements are mixed to create meaning. In essence AR is any live image that has an overlay of information that augments the meaning of these images.

Digital graphics are commonly put to work in the entertainment industry, and ‘mixing realities’ is a common motif for many of today’s media forms. There are varying degrees to which The Real and The Virtual can be combined. This is illustrated in my Mixed Reality Scale:

mixed-reality-scale — My Mixed Reality Scale, a simplified version of Milgram & Kishino’s (1994) Virtuality Continuum

This is a simplified version of Milgram and Kishino’s (1994) Virtuality Continuum; simplified, because their research is purely scientific, without an explicit interest in media theory or effects, therefore not wholly applicable to my analysis. At the far left of my Mixed Reality Scale lies The Real, or physical, every-day experiential reality. For the longest time we lived solely in this realm. Then, technological innovation gave rise to the cinema, and then television. These media are located one step removed from The Real, a step closer to The Virtual, and can be considered a window on another world. This world is visually similar to our own, a fact exploited by its author to narrate believable, somewhat immersive stories. If willing, the viewer is somewhat ‘removed’ from their grounding here in physical reality, allowing them to participate in the construction of a sculpted, yet static existence. The viewer can only observe this contained reality, and cannot interact with it, a function of the viewing apparatus.

Later advancements in screen media technologies allowed the superimposition of graphical information over moving images. These were the beginnings of AR, whereby most of what is seen is real with some digital elements supplementing the image. Indeed, this simple form of AR is still in wide use today, notably in cases where extra information is required to make sense of a subject. In the case of certain televised sports, for example, a clock and a scoreboard overlay a live football match, which provides additional information that is useful to the viewer. Television viewers are already accustomed to using information that is displayed in this way:

Simple Augmented Reality, televised football matches augment meaning with digital graphics

More recently, computing and graphical power gave designers the tools to build wholly virtual environments. The Virtual is a graphical representation of raw data, and the furthest removed from physical reality on my Mixed Reality Scale. Here lies the domain of Virtual Reality (VR), a technology that uses no real elements except for the user’s human senses. The user is submersed in a seemingly separate reality, where visual, acoustic and sometimes haptic feedback serve to transpose them into this artificial, yet highly immersive space. Notice the shift from viewer to user: this is a function of the interactivity offered by digital space. VR was the forerunner to current AR research, and remains an active realm of academic study.

Computer graphics also enhanced the possibilities offered by television and cinema, forging a new point on the Mixed Reality Scale. I refer to the Augmented Virtuality (AV) approach, which uses mainly digital graphics with some real elements superimposed. For example, a newsreader reporting from a virtual studio environment is one common application. I position AV one step closer towards The Virtual to reflect the ratio of real to virtual elements:

An Augmented Virtuality, the ITV newscasters sit at a real table in a virtual studio

There is an expansive realm between AV and VR technologies, media which offer the user wholly virtual constructions that hold potential for immersion and interactivity. I refer to the media of video games and desktop computers. Here the user manipulates visually depicted information for a purpose. These media are diametrically opposed to their counterpart on my scale, the cinema and television, because they are windows this time into a virtual world, actively encouraging (rather than denying) user interactivity to perform their function. Though operating in virtuality, the user remains grounded in The Real due to apparatus constraints.

Now, further technological advancements allow the fusion of real and virtual elements in ways not previously possible. Having traversed our way from The Real to The Virtual, we have now begun to make our way back. We are making a return to Augmented Reality, taking with us the knowledge to manipulate wholly virtual 3D objects and the computing power to integrate digital information into live, real world imagery. AR is deservedly close to The Real on my scale, because it requires physicality to function. This exciting new medium has the potential to change the way we perceive our world, forging a closer integration between our two binary worlds. It is this potential as an exciting and entirely new medium that has driven me to carry out the following work.

To begin, I address the science behind AR and its current applications. Next, I exploit an industry connection to inform a discussion of AR’s development as an entertainment medium. Then, I construct a methodology for analysis from previous academic thought on emergent technologies, whilst addressing the problems of doing so. I use this methodology to locate AR in its wider technologic, academic, social and economic context. This discussion opens ground for a deeper analysis of AR’s potential socio-cultural impact, which makes use of theories of media and communication and spatial enquiry. I conclude with a final critique that holds implications for the further analysis of Mixed Reality technology.