Abstract Reality

[image by flickr user Drewje]

In preparation for Where 2.0 next week, where I’ll be on a panel called “Mobile Reality,” I wanted to jot down some recent thoughts on the matter, both as a way of helping me better understand what I might say next week, but also a kind of beginnings of a more deliberate effort to think through some of the projects I’ve been involved with recently, both at work, and on my own stuff.

(I should mention that all of what comes next is entirely in the context of my personal work, and in no way is endorsed by or has anything to do with my current employer.)

I guess Mobile Reality is an attempt to define some different kind of Augmented Reality (AR) that is about focusing more on real spaces and location awareness, and stuff that runs on mobile devices. (Although AR in general I think is based around the same ideas idea of “enhanced” mobility). So perhaps this new variation is slightly redundant, except maybe to contextualize this stuff as being available on “today’s” mobile devices instead of on some crazy setup with a 100 pound backpack full of gear. In any case, I’m going to just refer to this stuff as Augmented Reality, and completely ignore the whole hologram side of this, where you show your computer’s camera a AR Code, and some 3D model is magically displayed on top of the code. That’s just called magic. [note: see this]

So most instances of Augmented Reality don’t interest me a whole lot. Julian’s been writing about a bunch of overlapping ideas, so I’m not going to spend much time discussing why I’m not 100% on board for this particular vision of the future. As He writes,

All [this] kind of stuff that would turn spatial experiences into some kind of database inquiry seem very much different from what I enjoy about the world when it is mixed with humans — curious interpretations of objects and moments that are not salted with uniform resource locators, pop-ups, soft synthesized voices telling me that I’ve got mail or to turn right at the next intersection.

What he’s getting at here is important, at least to me. The entire idea of many types of AR is to ostensibly make our lives easier, to help us learn about our environment. See Wikitude for a pretty insane example of how this kind of future might look. But in this entire process of becoming more at one with the world through this technology, the world becomes entirely mediated by a computer. It literally becomes a simulacra of itself, as though we weren’t close enough already. In some cases, AR requires use of special glasses that beam the real world into your vision through a camera mounted on the glasses, effectively taking the human part of you completely out of the physical world that the system is supposedly augmenting.

There’s a kind of Uncanny Valley effect that goes on in these experiences. The more that reality is augmented, the less real it becomes. The similarities to the Uncanny Valley here is not surprising considering that AR is a descendant of virtual reality, which itself was an offshoot of computer graphics, where the term comes from.

Clearly in this genealogy, there is a tendency to try and make virtual things real. AR I think is a little different in that it aims to (by definition) make the Real World itself more real, or at least more defined and accessible. In all cases, the inclination is always, “wouldn’t it be cool if…”

Ok, so that’s enough of that. But what I, and I think probably lots of other folks are interested in, is the idea of applying Augmented Reality to virtual spaces. This is the idea of lots of pervasive games and stuff, layering virtual worlds on top of the real world. This concept video from HP is a pretty archetypal version of what this kind of entertainment looks like. A kid, using his HP device, sees a virtual world layered on top of the real world. He’s running around, dodging virtual enemies in the streets, navigating the normal, real world which acts a kind of game field. So this is kind of one of the promises of AR as well, a way to create mobile and pervasive game spaces that blend (or mix) realities.

I think this stuff is pretty cool, in general. I think everyone is on board with the idea that your real life can kind of turn into a game at any moment, your cubicle or whatever turning into some kind of evil lair. If you haven’t seen The Game, or heard about Majestic, maybe you should check those out if you are interested in this kind of thing.

So, most of these type of things are based on this idea of spatial correspondence. If I’m in a park in the real world, there is some virtual space I’m walking around at the same time, that shares the spatial characteristics of the real world. The virtual world is the real world, reskinned, with enemies and obstacles that conform to the same constraints as you would in the real world. The HP video linked above is full of examples of this. I was also part of an early attempt to do similar things (albeit much weirder). Using spatially correspondent worlds makes sense I guess if you really want to turn the real world into a game board. Paradoxically, this kind of Augmented Reality really does more to connect you with the real world, forcing a more complete connection with physical space as you navigate the virtual layer.

In trying to define this kind of stuff, I’m realizing that the kind of stuff that I most interested in is similar to the virtual worlds stuff, but much less concerned with spatial correspondence.

I’m really compelled by the idea that mobile games or other experiences can be used to further connect people to physical environments. At the same time, I’m realizing that my work, and the work of others that I find the most interesting in regards to this, is really experimenting more with the idea of Abstract, or Abstracted Reality. To me, this is more like using aspects of the real world as game pieces rather than using it as the game board itself. There are any number of components that define the real world, from concrete objects like trees or buildings, but also less tangible things about how we interact with the real world like distance, speed, if this house is my house or this bar is where I meet my friends on the weekend.

This is one of the nice things about foursquare, for instance. On the face of it, it’s a type of mobile social network where you can see where your friends are. The thing that’s interesting to me about it though, is that it begins to use some of these more abstract layers of space (call it place, if you want) in a game context. For example, if I check into my bagel shop 3 times in one week, I unlock the “local” badge, or if I am at my local pub more than anyone else, I’m deemed “the mayor” by the game. In this way, dennis and naveen are using a more abstract concept of the physical world, one that connects me to my world a lot more than seeing some pop up overlay of the pub menu, or a little sound blip that tells me where the nearest place to get beer x is. (clearly, it’s getting closer to the beer part of the day…)

It’s probably also useful to look at how traditional games go about getting players to figure out the game environment. Games are often all about figuring out an environment, whether it’s a puzzle, or figuring out how you can get up to the coin on that high platform. But that’s probably enough of this for now.

Where 2.0!

Scheduling note: I’m speaking on a panel at this year’s Where 2.0 Conference, which should be a lot of fun. The panel is based around the idea of Mobile Realities, and I think a lot of it will be talk about how people are pragmatically creating alternative realities using the mobile platforms that are with us today instead of the laptop+backpack+differential GPS setup of days past, present (and unfortunately) future. Instead of dreaming about circa 1987 views of the future like 100% spatial corresponding “mirror worlds,” I hope we can focus more on what people are actually building now, and and how they are using various constraints with the current systems create cool new interactions and experiences.

If you want to come (and you should), use the code: WHR09FSP for a 25% discount!

Constraints and Innovation: LBS Edition

For just about as long as I can remember, the top item on most mobile/location nerds’ wish lists has been access to real time, always on GPS information. For many years, this dream was stuck in *5-years out* purgatory, mostly taken up by researchers wearing robot suits or less-than-casual citizens geared up with specialized PDA hardware.

At some point, the N95 came out, which allowed developers to access GPS information to some extent. But the N95 was still super expensive (relegating it to PDA status), and developing stuff for it was still kind of a pain in the ass. Then some version of iPhone software came out which used the Skyhook Wireless SDK to allow your phone to have a flavor of GPS based on WIFI triangulation. Even later, the v2. iPhone came out, which had full GPS integration. But we were still not satisfied. We need GPS apps to run in the background! To make matters worse, the Skyhook data wasn’t accurate enough to really track us in real time. Jeez!

But it’s been interesting to see some of the stuff that’s worked and not worked with this constraint. One thing that immediately becomes clear is that LBS apps tend to be more simplistic and less grand in their ambitions. Not being able to track every GPS coordinate you stumble upon, at every instance, has forced people to think a little harder about exactly why they want to use location information in the first place. Another thing that the constraint highlights is that apps that are designed around “where am I now” persistence don’t function very well. The Brightkite app is an example of this, as I would imagine other “here now” social location apps. Instead of embracing the constraint of an articulated location, chosen at the discretion of a user, these apps ignore it, hoping that in the future the always on gps will fix the problem. This may well work for them, but it’s a failure in terms of getting people engaged in the service in the short term.

What’s more interesting is an application like Foursquare, which focuses not so much on what latitude and longitude you are at (the space) but on the venue, event, etc. you are engaged in (the place). This works much better in the context of an articulated location mechanism. If I want my friends to know I’m at this bar, I can open the app, find nearby venues, and then tell the system which one I’m at. This may seem like a process that, using the powers of technology, could be easily automated. But when done right, a system like Foursquare I would argue functions better when this level of interaction is required.

Clearly there’s more examples of where this works and doesn’t, but frankly I’m running out of steam (way over 140 characters at this point).

And, of course not meant to dismiss the idea that persistent tracking can lead to some really amazing applications, stuff that lots of us have been scheming on for years. Locale, for Android (which supports background processes), is one early idea that has a lot of potential. Instead, it seems warranted to revisit the idea of articulated location, and look at it not simply as a bridge towards full, persistent tracking, but as a valid interaction scheme in it’s own right.

3D Cloth UI

Eduardo and Michael from Protohaus just released a nice concept for a 3D Mobile UI.

At Protohaus we believe that 3D UIs are here to stay. Many 3D concepts borrow very tired metaphors for navigating through spaces. This metaphor ends up really being a series of transitions between elements of a UI. We choose the metaphor of a cloth because of it’s natural dynamic beauty. If you look through the history of Art you will see that representations of cloth are everywhere so the idea is not new. The other aspect of a UI we feel is important and often neglected is its playful nature. The ability to poke, flip, twist, and twirl something gives us great joy.

App Store Refunds?

Update: Looks like the uproar from the article is largely bullshit. While Apple does have a clause about refunds in the contract, it’s still pretty difficult to get a refund, as you have to have a legitimate claim, not just “I don’t like this.”

If this is true, it’s kind of frightening, particularly for game developers.  According to Techcrunch (which is totally an entirely unreliable source), people can ask for a full refund for anything they buy, as long as it’s within 90 days.

So here’s a scenario for, let’s say ngmoco with rolando, step by step:

• I buy the game
• I like the game
• I finish the game
• I return the game within 90 days

It makes some sense to offer refunds. But c’mon, the fucking app is $1.99. Suck it up. If something like this is implemented, there really should be some tracking of the number of times the app has been run. If you run it once or twice… ok, return it. After that? Then it’s just bad news for devs.

GDC Mobile wrap

This season, I had to limit myself to one convention.  I chose GDC, or specifically GDC Mobile. This was a mistake.

GDC mobile is technically a summit that takes place in the pre-dawn of the more hyper-capitalistic main show.  On paper, it should have been a amazing place to showcase all the building excitement around a mobile games industry that until about a year ago, was completely lost in the woods, handicapped by poor controls, impossible business models, and a general lack of creativity.  As Neil Young said in his opening keynote, the iPhone “has changed everything.”  Strangely enough then, despite all this innovation, the selected sessions chose to more or less ignore that there was anything interesting going on in this area.

So maybe that’s a problem, or maybe it’s more indicative that the interesting things in mobile are not necessarily *mobile* problems any more.  They are design problems, which need design solutions.  Mobile games are no longer just these weird little things with bad graphics and poor experiences that you port IP to the same time you are figuring out which logo to use for the t-shirt.  No, with new platforms (not just the iPhone), now this is a platform that can be leveraged to build interesting new things, and as a community, we need to start learning to talk about these things in a passionate and interesting way.

It’s nearly unfathomable to me, for example, that Fieldrunners, a simple tower defense clone won the “best game” category.  That’s not really to take anything away from the developers, who executed the thing nicely, and the game is probably super fun and everything.  But this seems to me like a good place to highlight innovation, and all apologies, but I don’t think that adding tilt controls to something that could be experienced just as well on my desktop counts as innovation in a mobile game.  It’s not really taking advantage of all, or really any, of what the platform has to offer.

But I should stop before this becomes a rant against casual games in a mobile context and mention that my favorite, at least most well-worn, iPhone game is Drop7, area/code’s soduku style stacking game.  Not all games have to be GPS enabled, use the camera, or have some wild concept with the word “pervasive” or “ubiquitous” in it.

In fact, I found that the so-called “pervasive” mobile sessions were equally lacking – just in the reverse direction.  Sessions exploring AR (augmented reality) games, or the use of sensing devices in gameplay, were completely devoid of passion and fun, and demonstrated an alarming willingness to elevate technological novelty over engaging design.  It’s hard to scold Fieldrunners for being boring when the so called “cutting edge” of Augmented Reality highlighted a tower defense game ( it just happened to be one that you had to have a whole bunch of technology to play).  At least Fieldrunners looks kind of fun.

Of course, maybe this isn’t really the point.  Researchers are doing research, not necessarily game design, and there’s a whole other set of stuff going on that informs the output.

All of this was demonstrated nicely by the fact that the best talk on mobile game design of the week was given by Frank Lantz as a keynote for the Serious Games summit.

The real issue here is that I wanted to hear from people that were making mobile games, particularly mobile games that weren’t a port of a popular console game to the iPhone.  It would be cool to hear from designers, instead of corporate entities or researchers.  It’s not so much that talks from those spheres are terrible as it is that they aren’t supplemented by sessions from people that exhibit a high degree of passion about what they are building.

I don’t really have any answers to how to fix these issues.  People are building things, even if 99% of those things are failures on some level.  But these people, for better or worse, are forming the industry, and it’s disappointing to not have a venue to hear about their experience so that discussion and community can emerge.  The mobile industry has always been obsessed with the future, but at least at GDC, it’s not necessarily interested in the people who are building it.

Elephant Paths in Animal Crossing

Not really sure the extent to which this happens in other virtual environments, but one of the really great new aspects of the new Animal Crossing game is it’s support of Elephant Paths. The basic idea of the Elephant path is a visible path, formed by “unofficial” movements through space. In this sense, unofficial is defined as outside of established path infrastructure like paved pathways or roads. In many virtual worlds, Animal Crossing included, there aren’t many official pathways to begin with. The larger the world, the more frustrating it can be to navigate. While the world of Animal Crossing really isn’t that huge to begin with, the game requires that you take many of the same routes to accomplish certain tasks. The designers, in this revision of the game, keep track of the movements of your character, and if you trod along the same ground for a few days in a row, a visually distinguishable path will form. These paths enable much quicker and more decisive movement through the space.

The larger point to all this is perhaps thinking about how elephant paths can be created and leveraged in other environments. Certainly implementing such paths in larger online worlds seems obvious. But what about other stuff? What about web browsing trajectories? What about more semantic paths? Most elephant paths are about the quickest way to get from point A to point B. With a more layered approach, we could try and account for running paths, shopping paths, art paths, or other paths that had more encoded information.

Accelerometer remote control

Quick proof of concept code snippet for the USC mobile class I’m teaching in the spring. iPhone app allows you to control a cursor with the accelerometer, which is mapped into a simple game in processing. Will post the code after I clean it up a bit. This isn’t really doing anything super interesting yet, but I think it’s fun to think about some cool ways to use this basic interaction model. I’m especially thinking about a play mechanic where you have separate game spaces on the phone and on the desktop.

Upicomp Needs Hints

One of my favorite visual and conceptual part of Mirror’s Edge is the “red” interactive objects that pop starkly from the acid-washed urban environment. It’s simple, but genius: you can do something with this door, it’s different than that other door.”

It struck me that these kind of visual cues are exactly what we need to think about as we develop “interactive” landscapes. What is the language going to be that tells me “hey, you can pop open your mobile and interact with me.” Right now you have a pretty limited and boring way of “seeing” the interactive environment. Is it going to be the drab version we have now, where wireless networks (bluetooh, bonjour, wifi) allow you to scan for available nodes, which you can then take some action on (connect, download an image, send a file). Or is it going to be something more dramatic like the beckoning redness of the Mirror’s Edge city? While it’s objectively silly to think about a bunch of red doors and pipes around your town, I think the general idea of making interactive objects somehow visible (rather of lurking about the wireless spectrum) is a powerful concept, and one that could reshape the way our future cities look and operate.

Pink Ponies: None of this is possible

I love j-pony’s writing style, since it reminds me of talking directly to him when he’s got this kind of flabbergasted look in his eye. His take here on a Garmin GPS unit is perfect.