Designing AR Experiences: Lean Forward or Lean Back?

Presently, for the most part, I believe Augmented Reality (AR) is a lean back experience and we need to move towards a lean forward model to drive AR into the future as a compelling, engaging new medium.

So what distinguishes these two media models? It comes down to a passive (lean back) versus active (lean forward) user experience. In 2008, Jakob Nielsen applied these terms to discuss the differences between the Web and television. He described the Web as an active lean forward medium, where “users are engaged and want to go places and get things done.” In comparison, he described the television as a lean back passive medium where, “viewers want to be entertained. They are in relaxation mode and vegging out; they don’t want to make choices.” In a 2011 talk at the mCommerce Summit in New York Steve Yankovich, Vice President of eBay Mobile, referred to the iPad as a “lean back experience (the lean back on the sofa device).” Yankovich described this as the “don’t make me work” and “entertain me!” mode.

Has AR to date largely become a passive lean back entertain and dazzle me / bombard me with information experience?

I thought of this as I read Fast Company’s Co. Design article on Michaël Harboun’s AR thesis project, Transcendenz. Harboun, now a designer at IDEO, states in the article, “Regular AR applications add a layer of objective data, informing us about our surroundings. They give us an instant answer, so that we immediately know what we see.” In this immediacy, we’ve mainly become passive spectators with visuals and data ready to hand (FYI, an interesting tangent here on AR decreasing reliance on memory).

Harboun distinguishes Transcendenz as not giving answers, but asking questions. “It believes in the user’s ability to put the world around him into question, and to not content himself eating instant available data.” We can think of the participant in Transcendenz then as partaking in a lean forward media experience.

Transcendenz is driven by creating an empathetic experience for the user in AR (an arena I’ve been very interested in since I began working with AR seven years ago). My thoughts turn to Steve Mann’s work on Mediated Reality from the early 90’s. In an article in the Linux Journal, Mann states, “Mediated Reality sets forth a new computational framework in which the visual interpretation of reality is finely customized to the needs of each individual wearer of the apparatus”. As such, he furthers, “Just as you would not want to wear undergarments or another person’s mouth guard, you may not want to find yourself wearing another person’s computer”. Years ago as I read this I often thought, perhaps in the future you will want to filter your reality through someone else’s perspective, be it a friend in a social network, someone you admire or idolize, or perhaps even a complete stranger. Mediated Reality could be applied to build empathy, by enabling someone to see the world through another’s eyes.

In the Transcendenz video, we see the main character jaunted by how everyone now quite literally resembles him, taking on his physical appearance, within the Empathy prism. The narration is translated: “It’s amazing how a perfect stranger can suddenly seem so familiar. It’s as if one would project our own life on others and even the most annoying persons [sic] now make us smile.” Transcendenz is depicted as a tool to enable empathy not only towards other human beings, but to nature and our environment as well. Philosopher Immanuel Kant is referenced in the narration, “Empathy is not only about projecting ourselves on our fellows, but also on the world around us.” (I’ll spare you the stardust part.)

While in the Empathy prism, the user is actively engaged in his environment in a lean forward mode. The experience is initially accessed through the act of meditation as seen in the video at 1:18. The program indicates to the user, “You’re too excited to enter the interconsciousness. Try to meditate a bit.” There’s something curious happening in this act where a lean forward mode is entered via a lean back relaxation mode in meditation.

A couple of references here enter my thoughts. Firstly, non-traditional video games developed at USC’s Game Innovation Lab including Cloud, Journey, Flower, and Bill Viola’s The Night Journey. Game designer and lab director Tracey Fullerton describes these experimental games as positing, “the possibility of a game mechanic that expressed peacefulness, wonder and awe” as well as enlightenment (wonder being one of my favourite words in fact; here’s a link to my TEDx 2010 talk on AR & Wonderment). The core mechanic in Bill Viola’s PlayStation 3 game is described in his artist statement as “the act of traveling and reflecting rather than reaching certain destinations – the trip along a path of enlightenment.” I view this description as parallel to the mechanics and design intent of Transcendenz.

It is interesting to consider what kind of aesthetic and direction AR games may take on if such a mechanic is followed and applied as opposed to the current direction, which are predominately AR first-person shooter games.

The second reference I think of is Ian Bogost’s latest book, “How to Do Things with Video Games” and his chapter on “Relaxation”, particularly as he refers to lean back and lean forward media in relation to video games. Bogost distinguishes how leaning forward “requires continuous attention, thought and movement” and leaning back is associated with relaxation and passivity (he even mentions gluttony). He writes, “To relax through a game requires abandoning the value of leaning forward and focusing on how games can also allow players to achieve satisfaction by leaning back.” Transcendenz leans back in a state of relaxation and enlightenment, yet simultaneously asks the user to lean forward and not abandon that state, to actively engage with their environment and be present, immersed and interactive. The core mechanics of Transcendenz, comparable to Viola’s The Night Journey, are exploration, reflection and action through emotional experience to transform the user / player — and, after all, isn’t this what designing AR experiences should be about: engaging the viewer in meaningful, contextual, interactive learning experiences which are rooted in the real world.

Let’s continue the conversation on Twitter: I’m @ARstories.


Helen Papagiannis