Mobile, Locative, Augmented Reality

Two new services have come up in the last week that really grabbed my attention - and that of a few people I've shown them to. From different stables but with some similar concepts and intent.

These are Layar from SPRXMobile and Seer from IBM which was done as part of their mobile services for Wimbledon. A few commonalities first and some explanation.

Augmented reality is the term given when an object in the real world (a QR code, a tag of some sort, an image, an icon, a face or a building or a landscape) - the 'reality' is somehow made richer when viewed through a screen of some sort which add to or 'augments' the reality.

Classically, you take a trigger of some sort (such as the QR codes, tags or images mentioned before), hold them up to your web cam, and then see a 3D image play out over the static flat image you are holding up to the camera. Augmented Reality (or AR) can be run as fully life sized (there are some nice examples of cars in showrooms which are not really there); can be interacted with by you clicking options on the screen or, even sexier, pressing virtual elements in the augmentation itself. In my next blog, I'll be going into this is more detail - but I just wanted to set the scene.

Mobile is even more interesting, in part because the camera on mobile phones is usually at the back - giving an uninterrupted visual display (seeing around the image held up to a web cam can be a pain); in part because the screen is so personal to us; but even more so because location can make this much more personally relevant.

Layar and Seer both take the visual information that is present where you are, and augment this with additional information which adds value. This might be showing you the menu for a restaurant that you are viewing; placing a mark on a scene to tell you where an address is; giving you opening hours for a shop etc. In order to provide this, three things come into play: a camera on the phone (to stream the image): GPRS (where you are) and a Magnetometer (a digital compass to identify the direction you are pointing). IN true augmented reality, there would also be a fast, back-end image recognition facility (to match the visual reality being passed through the camera with the stored images for recognition and triggering of the augmentation). Given the need for the magnetometer, right now, these service are only developed for Android handset, however the new iPhone 3G S includes (for the first time) a magnetometer, so expect to see these service extend to here soon.

Of course, a pretty fast, ideally browser based application is required to run behind all this which has a rich set of content. This rich content needs to both be able to recognise what it might be you are looking at (based on your location and the direction you are looking) as well as provide some value to the visual experience. While both Seer and Layar are version of 'mobile augmented reality', both use plotting (GPS + Direction) to determine what should be shown in the augmentation and, unlike most augmented reality, no not recognise the landmarks to determine the trigger. Apart from the visual clues of seeing the plotting over the visual camera feed, the camera is effectively redundant and the information could be augmented to Google Street View or a highly detailed mapping service. That said - augmented locative information is new, consumer friendly, and likely to become commonplace soon.

In the case of Layar, which currently has content (Layars) only for The Netherlands, each content partner provides the positioning information, with each partner having their own 'layar' - which is effectively a visual overlay which uses the GPS + Direction to place relevant information at the location coordinates. By tapping on the side of the screen, the use can 'swap' between the 'layars' they are seeing over the background (reality) image. Look for this coming to the US and UK later this year.

Seer is, in some ways, a little more interesting. While the location here is even more limited - Wimbledon over the tennis season - they've add a very interesting addition by being able to have content drive from a live database. So, at Wimbledon, point your phone towards the restaurant - and you'll get the menu; to the stadium - and you'll get a map, venue information and the location of the toilets. Better still - point your phone at Centre Court - and you'll be told about the match that is currently playing, the score and given access to the previous results. Database driven systems, just like content management systems, can dramatically reduce the cost of developing regularly changing campaigns; even scarier, linking this with handset specific data (user, mobile number, Facebook friends, demographics) - the potential for personally targeted messages really comes into its own.

There are real Augmented Reality tools available, including for mobile phones, with ARToolkit having an iPhone version. Plotting is fun (in many ways!) but I think real mobile augmented reality is even more interesting, especially from a game play element, but again, more on that later.

Minority Report, we're not going to scan your retina - your (reality augmenting) mobile will do it for you.