Virtual reality (VR) can seem like being transported to a different world. It’s an exciting technology but, after we put the headset on, we rarely stop to ask: how and why does it all seem so realistic? In this article, we are going to explore some of the concepts and techniques that make VR so convincing. We will discover that VR is effective because it mimics the experiences we encounter and the qualities of the real world around us. Our eyes and ears work the same whether we are in a real world or a virtual one. When we simulate the way we experience the real world—for instance, by simulating three- dimensional scenes using stereoscopic vision—VR can make us feel like as if we are in a different world altogether, but a very realistic-feeling one. After reading this article, you will have a basic understanding of what happens in VR. In short, you will have a sense of how VR technology is designed to tap into the way our brains work each day. You will see how VR tricks our brains to believe that we are somewhere else entirely, in interesting and sometimes unexpected ways.
SENSING VR AS REAL
Virtual reality (VR) environments can be as small as the cockpit of an airplane or as large as an entire virtual world. These environments are designed to be as realistic as possible. Immersion refers to how well technology can simulate the ways we sense and perceive the world in our everyday life. VR is considered immersive when our experience in a virtual world is similar to our experience in the real world. In the real world, for example, you can walk or run at different speeds. If you were in a virtual world and you could only move at one speed, then the virtual world would not be immersive because your experience in VR would not match up with your experience of the real world, where you can walk or run at varying speeds. The technology behind VR is designed to make us feel like as if we have left the place we are standing and have been transported somewhere completely different. The more convincing (or immersive) the virtual world is, the more we start believing—or at least feeling as if—we are in the virtual environment.
So how does this happen? Let us imagine that we are going to build an alternate universe for a friend to experience; it needs to be very convincing for us to succeed. If we get it right, then our friend’s brain will be “tricked” into sensing that this universe we have designed feels real, even though she knows, of course, that it is an illusion. If we get it wrong, our friend’s experiences in the alternate universe will fall short of how her brain would perceive (or interpret) things in the real world. She might think the experience is enjoyable, but her brain will not be “tricked” effectively. A third possibility is that we get it really wrong. This last scenario could mean that our friend experiences cybersickness, which is when VR tricks your brain into feelings of motion-sickness (that queasy feeling some people get in a car, plane, or on a boat). In other words, VR is ineffective when the virtual world “behaves” differently than the real one.
HOW VR “TRICKS” OUR BRAIN
To understand how effective VR works, we first need to understand a little about how the brain makes sense of the world around us. Let us stop and think about the senses that allow us to experience the world: vision, hearing, and touch, to name a few. To make sense of the world, the brain needs to first bring in information from sensory organs, such as the eyes, ears, and skin. But bringing in the information only describes sensation —the act of transmitting information from the sense organs to the brain. What happens next is that the brain interprets this information, allowing us to understand what is happening in the environment. The interpretation of the senses which create our understanding is called perception.. For instance, we can see a dog running across the room, hear her bark, and feel her fur brush against our skin—these are sensations that we come to understand and perceive as experiences. The sensations all come together through perception to give us the experience of the dog. It is this interplay of sensation (using vision, hearing, etc.) and perception (our brain’s interpretation of this information) that creates our experience of reality.
HOW INGREDIENTS IN VR CREATE A DIGITAL REALITY
There is a lot to keep track of if we are to construct an alternate universe! Because our senses are numerous and complex, we will discuss a simpler example—an alternate universe that we can experience through vision alone. We can still make this place convincing because our brains tend to rely more on vision than on any other sense. Let us imagine that what we create is something like a Martian dune in Figure 1. How do we start with this simple image and get our friend to perceive it as an immersive environment? For starters, our friend needs to be able to experience this two-dimensional image of a Martian dune as though it were as real as the three-dimensional space you are in right now. To make this happen, we will need to begin by building something into our environment called stereoscopic vision.
A typical photograph is a motionless picture viewed from a single perspective. In the real world, however, when we view a scene we can move around and look at things from different angles. In Fig2, we have Leika the dog, sitting by a chair. We can sometimes see more of the chair than we can see of Leika, depending on our particular vantage point. How much our view of Leika is blocked by the chair depends on where we are standing. It is also important to note that we get a sense of depth in these pictures. In the photos, we can tell that the chair is (usually) closer to us than Leika, because it partially blocks our view of her. Knowing how close or far away things are partly depends upon our having. Binocular vision means that our left and right eyes see things from slightly different viewpoints, because they are located on different sides of the face. This means that our brain has to merge together information from these two perspectives. This process is called Steroscopics where our brains to let us know whether an object is close up or farther away. For our friend, stereopsis is what will create the illusion of being in the picture instead of just looking at it.
HOW TO CREATE A VIRTUAL REALITY
So how do we create stereoscopic vision for our friend’s virtual experience? First, we need to get her a VR headset. VR headsets simulate binocular vision by presenting slightly different images to each eye, giving the illusion that a two-dimensional picture is a three-dimensional environment. Advanced VR headsets can be expensive, but you can make an inexpensive one at home with mobile VR and then use a mobile as the presentation screen (using apps that split the image into separate views for each eye) but this often leads to a poor experience, and one that damages the appeal true VR pertains.
Seeing in stereo is not the only thing we will need, because our friend will want to look around at her surroundings the way we do when we are exploring a new place. So, we need to introduce the idea of head–tracking. Imagine walking into a beautiful cathedral. Chances are you will look up and down, left and right, and even behind you. In the real world, this happens so naturally that we do not even notice. However, in discovering how to create a VR environment, we often need to stop and question things that we usually take for granted.
What would happen if our VR headset always showed the same piece of the picture no matter how far up or down we looked? It would not be terribly convincing! An unconvincing virtual environment is just what we would get if we forgot about the head-tracking component of VR. When our friend looks up or down, the angle of the Martian dune should match the angle her head is pointed. When she turns around, the immersive environment needs to show her the visual information that was previously “behind” her. Head-tracking simply monitors the direction that your head is pointing by using something called an accelerometer which can sense whether (and in what direction) something is moving. Smartphones have accelerometers built into them by which you are able to enjoy certain types of games that involve tilting your phone. The phone’s accelerometer detects how you are tilting it and then adjusts your movement in the game accordingly.
VIRTUAL REALITY GONE RAPIDLY WRONG
Accelerometers in VR imitate the “accelerometers” we carry around with us in our heads. Any time we move our heads, sense organs in our inner ears—the vestibular system—provide the brain with information about how our head is oriented in space. The vestibular system is crucial for helping us maintain balance, making sure we do not fall down, and letting us know if we are lying down or standing up. Building head-tracking into our friend’s VR headset will make the virtual environment convincing if we get it right. But if it goes wrong, then our friend could get something called cybersickness. Cybersickness happens in VR when a person feels disoriented and nauseous. It is like the kind of motion sickness you might get while trying to read a book in the backseat of a car. Interestingly, however, motion sickness and cybersickness are not quite the same thing. A closer look at the difference will help us understand how VR can be so successful at “tricking” the brain, sometimes in the wrong kinds of ways.
What happens when we get motion sickness? It is the result of the brain receiving two conflicting signals. One signal comes from the eyes and the other from the inner ears (specifically, the parts that sense how your head is tilted in space, not the parts that detect sounds!). If you are trying to read a book in a moving car, your eyes convey to the brain that you are not moving, but your vestibular system registers the movement of the car as it leans into the turns, brakes, and accelerates. Your vestibular system sends your brain information that you are moving. Your brain cannot always quite reconcile these two contrary messages, and sometimes the result is motion sickness. This is partly why staring out the window can help relieving motion sickness. By having your eyes send “we are moving” signals to the brain, there is no longer disconnect between the messages from your eyes and ears, and you start to feel better.
So, how is it possible to feel motion sick in VR when you are not moving at all? Your vestibular system generally is not highly activated in VR, because you are often either standing still or sitting. But, stereoscopic vision and head-tracking give you the illusion of moving. Cybersickness is thought to result from this illusion, also known as vection. You might have experienced a vection illusion if you were ever on a train or bus and thought you were moving forward or backward when it was in fact the vehicle next to you that was moving. This illusion can arise in VR, because when moving through virtual space (using a game controller, for instance), your eyes say, “I am moving,” whereas your vestibular system says, “I am staying still.” Cybersickness can also occur when there is a time delay between when your head moves and when the screen adjusts the perspective of the environment. Some people are particularly prone to motion- and cybersickness, though the exact reason for this difference between individuals is not yet known.
JUST HOW REAL DOES VR FEEL?
Let us assume we have got the components of our VR headset correct and that our friend is not prone to cybersickness; we have pulled off a successful virtual environment, but how do we know how real it feels? Presence is an important concept in VR. Presence is used to measure how much a person feels like they are now in the virtual environment, instead of in the physical one. One way to measure presence is by recording a person’s heart rate and other signs of stress. If you get too close to a cliff edge in real life, you will likely experience certain sensations: a faster heartbeat, sweaty palms, and more rapid breathing. Measuring these same symptoms of stress can be also done with people on a virtual cliff edge in a simulated environment. One of the many ways VR is used outside of gaming is actually for the treatment of specific phobias, such as acrophobia (fear of heights). With the careful use of VR by mental health professionals, people who have an intense fear of heights (or other types of phobias) can be treated by a process called systematic desensitization, in which they are able to slowly master their fear in a safe environment.
Virtual reality has the potential to allow us to experience things we would likely never encounter in real life. The virtual environment we created for our friend only included the sense of vision. Advanced VR technology, however, incorporates other senses as well. The more of our senses that are correctly incorporated into a VR environment, the more immersive, or true-to-life, it is. The more immersive the VR world is, the more present we feel and the more we lose track of the place where we actually reside.
There is a big difference between learning about something through reading or watching documentaries and actually getting to experience it. Often, we learn about subjects, such as astronomy through textbooks and videos. In the future, however, science class might just include field trips to VR environments, where we get to explore and feel what it might be like to walk around on a Martian dune. Ultimately, this technology “tricks” our brain, making us feel like we are somewhere else by mimicking the perceptual experiences we have in the real world, and convincing us that we are inside our games, or on the surface of a different planet. How will you use this exciting technology?
Portalco delivers an alternate method of delivery for any VR or non-VR application through advanced interactive (up-to 360-degree) projection displays. Our innovative Portal range include immersive development environments ready to integrate with any organisational, experiential or experimental requirement. The Portal Play platform is the first ready-to-go projection platform of it’s type and is designed specifically to enable mass adoption for users to access, benefit and evolve from immersive projection technologies & shared immersive rooms.