Perception Design
Designing Spatial Experiences that people would naturally inhabit
Moving from user experience (UX) to participant experience (PX)
Throughout media history, we have always interacted with information on some form of 2D surfaces as observers from the outside. Think books, billboards, TVs, phone screens, car dashboards. People’s ideas have always been presented as flat depictions in one way or another. The only exception (I can think of) applies to theatre and sculpture where we enjoy art in its 3D form, and experiential spaces like modern museums and secret cinemas where visitors are expected to engage with the surroundings to live the magic of the story from the inside.
The way we humans learn about and interact with the world around us is more developed than mere clicks and swipes on flat, rectangular squares. Our sensory and perceptual systems funnel in floods of external signals like light, sound, smell, motion, orientation, depth, language, culture, etc. The brain receives those stimuli, processes them at a lightning speed, and builds a perceptual model of the situation. This in return dictates specific chemical and neural reactions as a response in a physical, cognitive, psychological, or even emotional form.
Being on a rollercoaster is one example of such experience where many human senses are evoked by external cues. According to Professor Roy Kalawsky, director of The Centre for Virtual Engineering Center and the University of Loughborough, we feel the air rushing over our faces. We smell the fumes emitted from the carriage breaks. People scream around us from all directions. Vibrations and inertial forces squeeze all our body parts. It is an intense experience where our complete nervous system is switched on, guaranteeing to leave us with mixed emotions between fear and excitement.
There is a lot of activity happening in our brain, body, and heart during such sensory-rich experience. It is the asynchronous combination of influences on those biological systems that creates a psychological perception to understand the physical context and “[fit] new information into [our] own frame of reference” for future use, as Erik Bar and Stan Boshouwers describe in their book World of Wonders.
Could this exact natural human phenomenon of understanding the physical world through perception be used to perceive experiences in the virtual ones? If so, how can designers of these experiences leverage it?
1. Bring Sensorial Experiences to Virtual Worlds
Experiences in computer-generated 3D worlds try to target as many sensory aspects as those in the rollercoaster experience to deeply engage participants and encourage longer dwell time. Spatial websites like Valentino Insights tap onto our 3D spatial awareness instinct, which we once relied on as a species to hunt and avoid being hunted. It leverages familiar visual cues such as light contracts, depth, portals, animations, etc. to attract and retain our attention. Here, we are virtually invited into the mansion of the Italian designer to discover, connect with the brand, get inspired, and eventually shop collections magically levitating in the air.
Multi-player 3D video games and their branded virtual worlds like GUCCI Garden Archetypes in Roblox, extend that instinctive spatial awareness by catering for other human needs like free self-expression and belonging to a herd, i.e. connect with others. Together we play, learn, shop, and even work better, triggering thus our social behaviors, cognitive capabilities, and sensational emotions to build an all-inclusive perception of the experience we go through as a group.
In augmented reality (AR) and mixed reality (MR), where digital layers enter the real world, our brain is occupied analyzing presented digital illusions through inputs from the eyes and ears, and perceptually accepting them as extensions to the physical world. Virtual reality (VR) on the other hand, disconnects us from the real world to immerse our senses in a digital one. In both AR and VR, our vestibular system subconsciously coordinates with our perception to manage how we move safely and connect with the place based on what it already knows from real life.
All those spatial experiences lead to the eminent Metaverse, that futuristic 3D version of the Internet which we will inhabit — not just use. It will utilize many of our sensorial and physical capacities to survive digitally and fulfil our virtual psychological desires, like dressing up one’s avatar in Roblox with that limited virtual GUCCI dress. Suddenly, our perception of living life as we know is taking an interesting turn, at least for many. And it is this perception that plays a crucial part in determining the quality of the final experience we would virtually undergo there. I call these dimensional digital experiences where human perception is key to inhibit and participate in them Spatial Experience (SX).
Spatial Experience (SX) is a three dimensional experience that people inhabit through different digital forms, leveraging their human perception to effectively participate in and understand it.
2. Spatial Experiences and their Participants
A Spatial Experience exits flatland and extends beyond traditional rectangular frames, to enter 3D realms where many human senses are engaged simultaneously in processing the presented digital illusions. They engage the full biological and psychological spectrums, and thus require broader ways of thinking of the context-of-use and deeper empathy with people.
Designing spatial experiences is far more intricate than that of mobile, car dashboards, AI voice assistants, or other common computing platforms.
Look and feel have always been the coins of transaction in conventional User Experience (UX) design and Interaction Design (IxD) for software systems. In spatial however, people are not just passive observers or users of an application, entertained by intuitive interfaces that allow them to complete a task quickly or make their lives easier. They live the application and participate in co-authoring its story narrative, which is very often a non-linear one: can start from any point, and can move in any direction. Storytelling suddenly becomes storyliving.
In spatial experiences, people live the application and participate in co-authoring its narrative, making it a storyliving experience not just storytelling, and converting people to become participants instead of users.
People also apply natural interactions as those performed in the real world, less mediated through standard UI elements. They do so with others whilst physically moving in dimensional environments, be it real or synthetic.
This intimacy and control gained through SX call for more coins to be introduced to the existing design currency. Connect is one as explained in the previous section. It reflects how participants simultaneously transact and connect with a brand, one another, and even the world around them (physical or virtual) when inside an experience. Balancing the three could quickly become an overwhelming cognitive load to them and thus should be considered thoroughly during the spatial design process.
People also apply different levels of body movements to achieve intended outcomes. That however may be hindered by their physical abilities and the available space they are in (too small causes claustrophobia, too big can get them lost). Additionally, any misalignment between what the body perceives as motion and eyes see could lead to serious accidents. That said, move would become another coin to carefully design for in SX.
Spatial Experience (SX) = look + feel + connect + move
Many psychological and physical factors are in play here to define the look, feel, connect, and move of SX. The ultimate goal is to design something that immerses people’s full perception of the world and places them as active participants in an experience, not just users of it. Perception is as important in computer-generated spatial environments as it has been in the physical spatial world since cavemen.
The question that swiftly emerges here is: how can we approach this noble goal most effectively for spatial experiences? The answer: a different design mindset is needed.
3. A Lateral Design Mindset for SX is Needed — Enter Perception Design (PD)
I have been studying this concept of sensory-driven, participatory-oriented spatial experiences for some time as part of my responsibilities in delivering XR work to our clients at Accenture. It is a known fact that the field of UX in a software context is very broad, intersecting many art and science disciplines. According to Dan Saffer, a pioneer Interaction designer and author, it is “a stew of disciplines”.
Moreover, it is understandable that UX is a framework made of tools, practices, and techniques that can solve different problems.
When it comes to design challenges afforded by the different forms of spatial computing though, I feel that many of those tools and techniques are not a perfect fit. This is not a surprise given that they have been developed throughout the industry for 2D interactions. The spatial human perceptual angle is very often a second or third thought by practitioners.
How can we bring physical and human perception to center stage when designing SX?
How can we manipulate existing mental models to help suspend the disbelief of digital illusions?
What factors do designers need to consider when crafting an experience to avoid making participants sick or alienated?
How can we delight them to reach the utmost levels of believability possible in a virtual or augmented world within the technical constraints imposed by the systems they run on?
Shifting from focusing on the use in UX to focusing on inhabitancy is the first steppingstone to answer such questions in my opinion. Rather than studying how prospect users would be using a spatial application — be it 3D websites, 3D games, AR/VR, or the Metaverse, we should switch our design attention to how participants would be inhabiting them. I call this Participant Experience (PX).
The same natural human perception that helps understand the real world and define the quality of engagements would be used here to understand and define those in the virtual one too. Perception leads to a positive or negative conclusive impression of spatial experiences in any given situation by considering the following parameters:
- Multi-sensory signals, e.g. “this virtual doorknob looks and behaves as I am expecting”
- Innate emotions, e.g. “the meeting room is familiar and makes me feel comfortable”
- Intuition and past cognitive memories, e.g. “from what I learnt, this screw should fit here”
- Personal and social behaviors, e.g. “I have experienced this cyber bullying before. I know what to do”
- Acquired motor skills, e.g. “my movement to virtually place the new food mixer in AR is not restricted physically or offensive to others around”
Designers of spatial experiences need to shift their design mindset from asking “how participants would use” an application to “how they would inhabit it” . It become Participant Experience (PX).
By adding inspiration as a wrapper around the above, a participant’s imagination is evoked to see new connections and relationships between people, places, objects, and ideas from the outside. From the inside, inspiration creates a personal understanding and consensus of the situation, and a new perception to rely on.
When we carefully consider the parameters that cause our perception to plausibly understand and accept any situation with least confusion, hassle, or contradiction to intents, we arrive at the holy grail of human-centered design. This is more paramount for spatial than any other computing medium for the following reasons:
- It is so personal — literally, it is on the participant or they are inside it
- It adds an instinctive third dimension to interactions, and even a fourth if we consider time
- It is much more amplified by the exhaustive streams of input coming from multiple senses at the same time.
Ultimately, designers cannot explicitly design an experience. They can however consider and manipulate factors to design for an experience. ““Designers create experience, but only indirectly.” states Katie Salen and Eric Zimmerman in their book Rules of Play*. This said, perception and the above parameters become the precursor for any spatial experience.
Designers cannot design experiences. They design for experiences, by considering the adjacent factors and parameters.
Spatial designers, those who design inhabitable spatial experiences in virtual 3D spaces, should aim to design for perception to land on better user experience, or more eloquently said: to land on better participant experience. The former is the cause, the latter is the ultimate result.
I refer to this shift in design mindset for spatial experiences as Perception Design (PD). It is a set of design principles that target the physical, behavioral, emotional, and cognitive aspects of human perception to ultimately create an intuitive spatial experience for participants in virtual worlds. It focuses on moldings behaviors, emotions, senses, community, and movement together through the design process to form a more comprehensive mapping between psychological drivers and people’s intentions. Luckily, we do not have to start from a blank canvas. There are many examples and industry practices at our disposal to use as frame of reference to learn from and build on top as shall be explained in later articles.
Perception Design (PD) is a set of principles that target the physical, behavioral, emotional, and cognitive aspects of the human perception to create intuitive spatial experience for participants in synthetic worlds.
4. What’s next?
I have tried to explain how we humans understand the world around us through the diverse stimuli our sensory and perceptual systems receive. I have elaborated on it being influenced by our personal experiences from the physical world to drive certain actions in the virtual one, be it as an individual or part of a group. The thesis of my argument has been that emotions, inspirations, and physical motion all play a great deal in shaping our perception and acceptance of surreal notions in SX to elevate intuitiveness for participants. And designers should aim to target that perception in order to design better spatial experiences.
This is part of an in-depth series of articles I am working on to introduce the concepts behind Perception Design and to lay the foundation for a new design mindset aimed at Spatial Experience (SX). I have explained the fundamental logic behind it and its rationale in this first article. In the next one, I will cover how our human perception works and how it stretches to understand and interact with virtual worlds through preconceived mental models. After that, I will elaborate the different levels that could be achieved for spatial experiences with the SX Ladder concept i will present. In the ones that follow, I will detail the building blocks that make up a powerful SX. Following that, I will defend my abandonment of the User when talking about spatial experiences and why I call for the industry to adopt the term Participant instead.
Crucially, the concepts of Perception Design (PD) and Spatial Experience (SX) are still evolving. It’s the start of an interesting exploratory journey, which I hope you can join me in to shape more concretely.
*[edit 1] Thanks to Kathryn Yu for sharing this quote from Rules of Play through a tweet.
References
Bar, E. & Boshouwers, S. (2018). WOW: Worlds of Wonder (1st ed.). BIS Publishers.
Kalawsky, R. S. (2000). The Validity of Presence as a Reliable Human Performance Metric in Immersive Environments. 3rd Annual International Workshop on Presence, (pp. 1–16). Delft.
Papagiannis, H. (2017). Augmented Human (1st ed.). O’Reilly.
Saffer, D. (2010). Designing for Interaction: Creating Innovative Applications and Devices (2nd ed.). New Riders.
Feel free to reach out to Fadi on LinkedIn or Twitter @FadiChehimi
Copyright © 2021 Accenture. All rights reserved.
