How to get started with VR design in 2023
3 tips from 3 years of VR interaction design experience
With the recent push for the metaverse by Meta(Facebook) and emerging tech companies, students and designers alike are now intrigued, more than ever, by the world of Virtual Reality.
As the lead designer for Sword Reverie, an open-world VR JRPG game, I have had around three years of experience designing with Steam VR & Oculus.
Recently I had the chance to take an IxD & MHCID course, both as a student and a guest lecturer on Spatialized Everything, led by the University of Washingtons’ Axel Rosler where topics of VR are being explored in both an academic and industry setting.
Many students from the class found this immersive “new technology” (It's been around for a very very long time)both captivating and daunting.
Common questions I get are, “Hey Leon, how does VR design work exactly? Is this a developer-only thing? How can I, as a new designer get my hands on a project with no prior experience ?”
Writing this article would be a great way to serve as both self-reflection, and help answer some of those most commonly asked questions students and industry professionals alike have reached out to me with.
When designing Sword Reverie, or any other polished VR interactive experience, there is a certain “design talent” needed no doubt.
However, through this article, I will break down exactly what this “talent” is into three concrete areas of skill that you can learn and grow in.
- Mastery of the design process.
- Spatial design knowledge & experience.
- Technical Knowledge and willingness to learn.
1. Mastery of the design process
“Leon, how does VR design even work really? Where does a designer start?”
VR, AR, MR, Metaverse, whatever term you want to call it, one thing is clear, the design process is universal.
The UX process of designing for Virtual Reality is just the same as designing for mobile, desktop, or any other device/platform you are familiar with.
“But, Leon! In all hose videos I've watched, like in Ready Player One where reality is blended with digital technology, I just don't see how the design process comes into play? How can a designer technically mock-up or prototype these things?!”
Hmmm, I know, some of you might feel unconvinced.
I'm glad for those who remain skeptical, so let me introduce to you what I call the Iceberg effect.
Looking at a final polished Virtual reality product can be misleading to the untrained eye.
Whether it's something speculative like the world in Ready Player One or our VR world of Sword reverie, one might easily get caught up in the 3D models, the hardware, the interactions, or all of the above, which I refer to as the technical knowledge or components of VR design.
However, like an iceberg would suggest, the majority of the design/product work, is below the water. That tip you see, the technicalities of the final product, all of that is only a small tip of the iceberg.
Using the iceberg model, you can see this oversimplified explanation of how VR design really works.
You can see under the iceberg is that design process we leverage daily. Whether it's identifying product requirements, conducting design research, or early ideations, no Unity or technical tools are needed at all.
Traditional design methods such as storyboarding(shown below), design sketching, whiteboarding are all where the majority of the UX heavy lifting is done.
Only when we get near the surface of the iceberg(creating prototypes such as the one below) do skills like Unity start being important.
What this should mean to you is, VR design is exciting. Because too many of us, it is the wild west of UX /interaction design. Best practices are still being explored.
That means the problems you are trying to solve in VR will often be something no one has ever dealt with. If you are not an expert on the design process yet, you might struggle with how to make a good data-driven or user-centered design decision in VR.
So being able to understand the design process holistically, and not caught up on the hardware and technical tools just yet, is where you should start.
2. Spatial design knowledge & experience
“What is spatial design? Do I need to know 3D modeling with C4D or Maya?”
Once again, similar to my first advice with the design process, try to remove the technical tools and software away from the equation first.
VR deals with 3D, tangible, and physical experiences. Therefore either from architecture, 3D modeling, industrial design, or a Minecraft creative expert like myself, having great spatial awareness will be very helpful when designing.
However, I encourage everyone reading to try out VR design first before saying “Dealing with 3D, it's just not for me.”
During our UW masters, HCI&D lectures on spatial design, one of my favorite quotes from our professor Axel is, “We as humans, we evolved moving from space. Nothing is flat naturally.” (Sorry to break it to you flat earth theorists)
I especially like this quote, beucase I want you to remember, flat spaces, screens especially, are something artificially created and developed for interactions by humans, it does not exist in the real world.
Spatial interaction design, in the 3d world, is how the world really functions.
So even if you have no prior experience interacting with 3D environments, like Axel says, “Learning spatial or VR design is really like seeing how the world really is, for the second time around.”
A good UX designer working in the 3D space understands this. They know a lot of the work is done for them already.
For example, when someone sees a tree stump, they naturally know how to sit on it, you don't need to reteach them how to sit on it just because it's not a chair. The same applies to many VR interactions. You don't need to invent how to “grab” in VR, trying to grab something is intuitive, users naturally will reach for it.
Of course, it's not all that simple. When we move on to real products there are a lot of details and intricacies between the real world and a digital VR interactive experience.
Here is a great example below, from my time prototyping & testing VR interaction in Sword Reverie with hundreds of users.
At the time, our sword-swinging interactions just didn't feel that satisfying, even though we mimicked sword weight with our controllers. (This is achieved by slowing the perceived sword swing by adding a slight delay between your controller movement and the sword movement in-game.)
What we failed to realize at the time was our design of being more accurate to real physics, was not our goal. Our goal should be to “make the physics feel good, rather than feel real.” Therefore we proposed the redesign below.
Instead of adding the delay, we wanted the swords to feel heavier before the user swings, but lighter/faster when you swing them. This speeding up motion makes the player feel more satisfied as if they landed a huge blow to the enemy. (Even though it's the opposite of physics in the real world.)
There are countless examples like this to consider when designing in VR.
How do we design an experience for players playing standing vs sitting? How do we make sure the player can find a menu that is positioned behind them? When do we show the players hands and when do we not? (showing hands will naturally lead people to want to grab things, once they realize something isn't grabbable, immersion breaks.)
A lot of this knowledge will grow as you worked in the VR space longer, but hopefully, through reading this article the designs you create in a 3D space will be a lot more thoughtful than before.
3. Technical Knowledge and willingness to learn.
“Leon what kind of classes, bootcamps, or masters design programs do you recommend if I want to learn Unity and the technical skills for VR design? It seems VR design needs “coding” with Unity, Is this true?”
Finally, we get to the long-awaited question, Unity. What is it, where do I learn it? Here is my one-sentence answer, “Not from class.”
What I do want you to know is, it is totally okay if you are brand new to these tools but you have to be a good self learner.
Classes are and should be for theories. They teach you how to think, design, and analyze a problem critically. Never does an “established academic course” aim to teach you technical skills.
That is why I don't recommend you to try to take a VR design class or Bootcamp. If you want to learn it, GitHub, Youtube, and Google Search are what I suggest.
“So Leon, How hard is it to pick up Unity?”
Depending on If you are a solo designer or not, how much you need to master technical tools such as Unity 3D or Unreal engine will vary. (If you work in a team, the designer won't and should not touch any coding.)
These tools are free to try and, as I said above, there are tons of community resources out there. If you work in a team, you can always try to have someone show you the ropes.
The point is, if learning something new scares you, then VR design probably will be challenging to you. However, to me, that is where the fun is.
Anyone can make a few quick VR/AR sketches or mocks in Figma, Photoshop, or even After effects, but you can’t really call yourself a VR designer if nothing you ever create gets prototyped and tried by real users can you?
Final thoughts.
So to wrap it up, VR design is very exciting, if you are ready and willing to learn on the job/project through the 3 main areas listed above. If you already have some existing knowledge on Unity or Unreal, great, diving into VR design will be easier for you than others.
Some last advice, if you are currently a design student, no matter if you are doing a master's or undergraduate study, this is one of the best opportunities for you to learn and explore VR. (Even if your curriculum does not have a VR-related course.)
For example, take a look below at a VR project I did as an example for UW & RISD students as part of a VR/AR course I taught.
Our final project was on data humanism, ie “exploring how data feels”. This does not need a full-fledged VR experience, but I encouraged students to design one anyways.
Point of the example? Trying to learn through a semi-directed project or a pure passion project are all some of the best ways to get started.
This up-and-coming VR world needs to be designed. If you do not design it somebody else will!
