I read about Foldit in Wired US yesterday, a game that takes the foundations laid by SETI@home, which uses thousands of computers’ idle time to decode frequencies from Space, and crowdsources solutions to the protein folding problems that are currently baffling the smartest machines in the world.
The difference with Foldit is that it’s not PC idle time that is tapped into here, but players’ idle time. There is no algorithm that can yet match humans’ depth perception; natural ability to recognise patterns; and see causal links in their actions. These traits make us humans the ideal CPU to solve these ‘protein-puzzles’:
Foldit provides a series of tutorials in which the player manipulates simple protein-like structures, and a periodically updated set of puzzles based on real proteins. The application displays a graphical representation of the protein’s structure which the user is able to manipulate with the aid of a set of tools.
As the structure is modified, a “score” is calculated based on how well-folded the protein is, based on a set of rules. A list of high scores for each puzzle is maintained. Foldit users may create and join groups, and share puzzle solutions with each other; a separate list of group high scores is maintained.
Indeed, the creators report that groups working together have led to breakthroughs not matched by either individuals or heavy-duty computing power. It is the power of the engaged-masses that the Baker Lab, research team behind the game are hoping will bring forth potential cures for HIV/AIDS, Cancer and Alzheimer’s.
More info on the game and it’s background on their Science Portal.
A great bit of work from SPRX Mobile has come to my attention:
Man, I’m looking forward to the Slope of Enlightenment!
However, I reckon we’re already at the Peak of Inflated Expectations, due a worldwide crash into the Trough of Dissillusionment within the next six months.
You’ve probably read about Google Latitude, and maybe even used it yourself. I’ve been using it mostly without meaning to, because I activated the service on my N95’s Google Maps and the bloody thing never turns off. Here’s where I am right now:
Locative technologies are a growing area of interest for me. I believe that GPS, cell-tower triangulation and even good old Bluetooth will play a large part in making cloud-computing extra-relevant to consumers.
I know that people get a bit funny with the blend of real locations and virtual space (see Google Street View debacle) but once we’re all using our next-gen pieces of UI, your networked device could begin to act as a portal to new layers of information useful to you about the city, street, or shop you are in.
I am talking about location-based advertising. An implementational nightmare, but it is foreseeable that Semantic technologies could serve geographically relevant messages, charging advertisers on a cost per impact basis. Google kind of do this with their local search results. It’s a bit shit at the moment though.
The nearest we have to the kind of next-gen solution I’m thinking of is lastminute.com’s free service NRU, available on the Android OS. It lets you scan around your environment with your phone acting as a viewfinder, where cinemas, restaurants and theatres are overlaid in a sonar-like interface. These services pay a small amount to lastminute.com on an affiliate basis, or are paid inclusions:
NRU for Android, from lastminute on the G1
There’s one locative service I’m disappointed never took off in the UK, despite being around for a while. BrightKite is a kind of location-based Twitter, and it had real promise until Google came stomping all over them with the release of Latitude.
If I were to ‘check in’ at The Queens Larder on Russell Square, BrightKite users would see my marker and message on a map of the area, as well as other people checked in nearby. The potential for social interaction is high, because through using the service one feels proximity with other users.
With all this in mind, I’d like my readers to ‘feel closer’ to me, so as well as in this post I’ll be placing my Latitude Location Badge on my Contact Page. If you’re in the vicinity, go ahead and either serve me an advert or say hello. I won’t mind which.
AR is considered by some to be a logical progression of VR technologies (Liarokapis, 2006; Botella, 2005; Reitmayr & Schmalstieg, 2001), a more appropriate way to interact with information in real-time that has been granted only by recent innovations. Thus, one could consider that a full historical appraisal would pertain to VR’s own history, plus the last few years of AR developments. Though this method would certainly work for much of Wearable AR- which uses a similar device array- the same could not be said for Mobile AR, since by its nature it offers a set of properties from a wholly different paradigm: portability, connectivity and many years of mobile development exclusive of AR research come together in enhancing Mobile AR’s formal capabilities. Despite the obvious mass-market potential of this technology, most AR research continues to explore the Wearable AR paradigm. Where Mobile AR is cousin to VR, Wearable AR is sister. Most published works favour the Wearable AR approach, so if my assessment of Mobile AR is to be fair I cannot ignore its grounding in VR research.
As aforementioned, VR is the realm at the far right of my Mixed Reality Scale. To explore a Virtual Reality, users must wear a screen array on their heads that cloak the user’s vision with a wholly virtual world. These head-mounted-displays (HMD’s) serve to transpose the user into this virtual space whilst cutting them off from their physical environment:
A Virtual Reality HMD, two LCD screens occupy the wearer's field of vision
The HMD’s must be connected to a wearable computer, a Ghostbusters-style device attached to the wearer’s back or waist that holds a CPU and graphics renderer. To interact with virtual objects, users must hold a joypad. Aside from being a lot to carry, this equipment is restrictive on the senses and is often expensive:
A Wearable Computer array, this particular array uses a CPU, GPS, HMD, graphics renderer, and human-interface-device
It is useful at this point to reference some thinkers in VR research, with the view to better understanding The Virtual realm and its implications for Mobile AR’s Mixed Reality approach. Writing on the different selves offered by various media, Lonsway (2002) states that:
“With the special case of the immersive VR experience, the user is (in actual fact) located in physical space within the apparatus of the technology. The computer-mediated environment suggests (in effect) a trans-location outside of this domain, but only through the construction of a subject centred on the self (I), controlling an abstract position in a graphic database of spatial coordinates. The individual, of which this newly positioned subject is but one component, is participant in a virtuality: a spatio-temporal moment of immersion, virtualised travel, physical fixity, and perhaps, depending on the technologies employed, electro-magnetic frequency exposure, lag-induced nausea, etc.”
Lonsway (2002: 65)
Despite its flaws, media representations of VR technologies throughout the eighties and early nineties such as Tron (Lisberger, 1982), Lawnmower Man (Leonard, 1992) and Johnny Mnemonic (Longo, 1995) generated plenty of audience interest and consequent industrial investment. VR hardware was produced in bulk for much of the early nineties, but it failed to become a mainstream technology largely due to a lack of capital investment in VR content, a function of the stagnant demand for expensive VR hardware (Mike Dicks of Bomb Productions: personal communication). The market for VR content collapsed, but the field remains an active contributor in certain key areas, with notable success as a commonplace training aid for military pilots (Baumann, date unknown) and as an academic tool for the study of player immersion and virtual identity (Lonsway, 2002).
Most AR development uses VR’s same array of devices: a wearable computer, input device and an HMD. The HMD is slightly different in these cases; it is transparent and contains an internal half-silvered mirror, which combines images from an LCD display with the user’s vision of the world:
An AR HMD, this model has a half-mirrored screen at 45 degrees. Above are two LCDs that reflect into the wearer's eyes whilst they can see what lies in front of them
What Wearable AR looks like, notice the very bright figure ahead. If he was darker he would not be visible
There are still many limitations placed on the experience, however: first, the digital graphics must be very bright in order to stand out against natural light; second, they require the use of a cumbersome wearable computer array; third, this array is at a price-point too high for it to reach mainstream use. Much of the hardware used in Wearable AR research is bought wholesale from liquidized VR companies (Dave Mee of Gameware: personal communication), a fact representative of the backward thinking of much AR research.
In their work New Media and the Permanent Crisis of Aura Bolter et al. (2006) apply Benjamin’s work on the Aura to Mixed Reality technologies, and attempt to forge a link between VR and the Internet. This passage offers a perspective on the virtuality of the desktop computer and the World Wide Web:
“What we might call the paradigm of mixed reality is now competing successfully with what we might call ‘pure virtuality’ – the earlier paradigm that dominated interface design for decades.
In purely virtual applications, the computer defines the entire informational or perceptual environment for the user … The goal of VR is to immerse the user in a world of computer generated images and (often) computer-controlled sound. Although practical applications for VR are relatively limited, this technology still represents the next (and final?) logical step in the quest for pure virtuality. If VR were perfected and could replace the desktop GUI as the interface to an expanded World Wide Web, the result would be cyberspace.”
Bolter et al. (2006: 22)
This account offers a new platform for discussion useful for the analysis of the Internet as a component in Mobile AR: the idea that the Internet could exploit the spatial capabilities of a Virtual Reality to enhance its message. Bolter posits that this could be the logical end of a supposed “quest for pure virtuality”. I would argue that the reason VR did not succeed is the same reason that there is no “quest” to join: VR technologies lack the real-world applicability that we can easily find in reality-grounded media such as the Internet or mobile telephone.
I have been aided in this series by a connection with Gameware Development Limited, a Cambridge-based commercial enterprise working in the entertainment industry. Gameware was formed in May 2003 from Creature Labs Ltd, developing for the PC games market which produced the market leading game in Artificial Intelligence (AI), Creatures. When Gameware was formed, a strategic decision was made to move away from retail products and into the provision of technical services. They now work within the Broadcasting and Mobile Telephony space in addition to the traditional PC market. I use this business as a platform to launch into a discussion of the developments current and past that could see AR become a part of contemporary life, and just why AR is such a promising technology.
Gameware’s first explorations into AR came when they were commissioned by the BBC to develop an AR engine and software toolkit for a television show to be aired on the CBBC channel. The toolkit lets children build virtual creatures or zooks at home on their PCs which are uploaded back to the BBC and assessed:
A typical Zook, screenshot taken from Gameware's Zook Kit which lets children build virtual creatures
The children with the best designs are then invited to the BAMZOOKi studio to have their virtual creatures compete against each other in a purpose-built arena comprised of real and digital elements. The zooks themselves are not real, of course, but the children can see silhouettes of digital action projected onto the arena in front of them. Each camera has an auxiliary camera pointed at AR markers on the studio ceiling, meaning each camera’s exact location in relation to the simulated events can be processed in real time. The digital creatures are stitched into the footage, and are then navigable and zoomable as if they were real studio elements. No post-production is necessary. BAMZOOKi is currently in its fourth series, with repeats aired daily:
BAMZOOKi, BBC's AR game show where children’s zooks compete in a studio environment
BAMZOOKi has earned Childrens BBC some of its highest viewing figures (up to 1.2 million for the Monday shows on BBC1 and around 100,000 for each of the 20 episodes shown on digital Children’s BBC), which represents a massive milestone for AR and its emergence as a mainstream media technology. The evidence shows that there is a willing audience already receptive to contemporary AR applications. Further to the viewing figures the commercial arm of the BBC, BBC Worldwide, is in talks to distribute the BAMZOOKi format across the world, with its AR engine as its biggest USP. Gameware hold the rights required to further develop their BAMZOOKi intellectual property (IP), and are currently working on a stripped down version of their complex AR engine for the mobile telephony market.
I argue, however, that Broadcast AR is not the central application of AR technologies, merely an enabler for its wider applicability in other, more potent forms of media. Mobile AR offers a new channel of distribution for a variety of media forms, and it is its flexibility as a platform that could see it become a mainstream medium. Its successful deployment and reception is reliant on a number of cooperating factors; the innovation of its developers and the quality of the actual product being just part of the overall success the imminent release.
As well as their AR research, Gameware creates innovative digital games based on their Creatures AI engine. They recently produced Creebies; a digital game for Nokia Corp. Creebies is one of the first 3D games which incorporates AI for mobile phones. Gameware’s relationship with Nokia was strengthened when Nokia named them Pro-Developers. This is a title that grants Gameware a certain advantage: access to prototype mobile devices, hardware specifications, programming tools and their own Symbian operating system (Symbian OS) for mobile platforms. It was this development in combination with their experiences with BAMZOOKi and a long-standing collaboration with Cambridge University which has led to the idea for their HARVEE project. HARVEE stands for Handheld Augmented Reality Virtual Entertainment Engine.
Their product allows full 3D virtual objects to co-exist with real objects in physical space, viewed through the AR Device, which are animated, interactive and navigable, meaning the software can make changes to the objects as required, providing much space for interesting digital content. The applications of such a tool range from simple toy products; advertising outlets; tourist information or multiplayer game applications; to complex visualisations of weather movements; collaborating on engineering or architectural problems; or even implementing massive city-wide databases of knowledge where users might ‘tag’ buildings with their own graphical labels that might be useful to other AR users. There is rich potential here.
In HARVEE, Gameware attempt to surmount the limitations of current AR hardware in order to deliver the latest in interactive reality imaging to a new and potentially huge user base. Indeed, Nokia’s own market research suggests that AR-capable Smartphones will be owned by 25% of all consumers by 2009 (Nokia Research Centre Cambridge, non-public document). Mobile AR of the type HARVEE hopes to achieve represents not only a significant technical challenge, but also a potentially revolutionary step in mobile telephony technologies and the entertainment industry.
Gameware’s HARVEE project is essentially the creation of an SDK (Software Development Kit) which will allow developers to create content deliverable via their own Mobile AR applications. The SDK is written with the developer in mind, and does the difficult work of augmenting images and information related to the content. This simple yet flexible approach opens up a space for various types of AR content created at low cost for developers and end-users. I see Mobile AR’s visibility on the open market the only impediment to its success, and I believe that its simplicity of concept could see it become a participatory mass-medium of user-generated and mainstream commercial content.
