Have you ever worn a virtual reality headset excited for the immersive experience, only to have it marred by a disruptive black grid blanketing your vision? That isn’t your eyes playing tricks on you: it’s actually a real tech phenomenon, known as the screen door effect.
Of course, the key to a successful immersive application is to captivate the user and involve themin the virtual world, not just offer them a view from the outside (we already have standard screens and monitors for that!). And if the entire world is blanketed by a strange meshing like the one you get on a screen door, it may feel as painful as being slammed in the face by said door…okay, maybe not that painful, but the full power of immersion can’t be unlocked.
Why exactly does the screen door effect happen?
Nowadays, most electronic screens are either LCD or OLED displays. They are made up of loads of pixels, each consisting of different-coloured subpixels (red, green and blue).
If you have a TV or a computer monitor, chances are it’s an LCD. All of the pixels are lit up by a singular backlight, which illuminates the entire screen.
OLEDs are most commonly found in smartphones and virtual reality headsets (though OLED TVs and monitors exist now too!). OLED displays tend to be crisper; the structure of the pixels is often quite different, and each individual pixel emits light by itself.
As illustrated in the above images, between each of the pixels (and subpixels) is an area of unlit space – and it’s this space that is the culprit of the intrusive screen meshing.
The screen door effect isn’t unique to VR displays; if you sit too close to the television or put a magnifying glass over your smartphone’s screen, you may be able to see the miniscule mosaic of red, green and blue shapes. But when you casually watch TV or text a friend, your eyes will be at a comfortable distance from the screen where you won’t even be able to make out the distance between each pixel.
However, with head-mounted devices, it’s a different story. When you wear a virtual reality headset, your eyes are much closer to the display than they would be looking at a phone, computer monitor or television. The lens magnification and all-encompassing field of view of the headset visuals only further exaggerate the gap between the pixels.
How can we prevent it?
Due to the way LCD and OLED screens work, the screen door effect is impossible to shut out completely (at least at the moment). That doesn’t mean there aren’t any ways we can make the effect less prevalent, but they’re not without their own drawbacks.
Probably the most obvious option. Increasing the amount of pixels per inch (ppi) decreases the amount of black space in the display, and more and more tech companies are recognising the need to improve the quality and crispness of their product’s visual screens. Nowadays, a commercial VR headset display can have upwards of 500 pixels per inch
However, a higher pixel density may require a larger display to host them all on, not to mention a larger amount of system memory to handle each pixel’s information without sacrificing motion smoothness.
Diffusion filters are a common and inexpensive visual trick for making images look softer and smoother. The material of the diffuser scatters the emitted light, creating the illusion of the pixels blending into one another and eliminating a lot of the explicit gaps between each one.
It’s important to note, however, that this technique also carries some burdens of its own. If not done carefully, a diffuser may make visuals look too blurry to be convincing, and even cause more disruptive artifacts such as sparkling and moiré-patterned (ripple-like) interference. Many users may also feel themselves having to make more effort to focus on the visuals, resulting in both hindered immersion and bad eye strain!
Low pass filters
A low-pass filter positioned over the screen is a straightforward way of filtering out the most high-frequency black space of the display.
Not only does implementing such a filter bump up the costs, but also power consumption (which is also expensive, and in today’s ecologically-minded climate, not ideal!)
Researchers at Facebook Reality Labs, the developers of the Oculus Rift display, have proposed a way of mitigating the screen door effect via piezo actuators, devices which convert electrical energy into a high-speed force of displacement without any moving parts. Two of these devices would shift the pixels along the screen so that they occupy unlit gaps, minimising the perceived distance between them. While this sounds like it would make everything shaky, this would be done at such a rapid rate that the picture would still appear stable to the viewer.
Though studies into this method are currently limited, the researchers highlight that this method is most likely not ‘one-size-fits-all’, and the screen door effect in a display would need to be inspected and characterised before deciding how the pixels should be moved.
In 2018, Valve Corp (creators of the Valve Index VR headset and the Steam game marketplace) filed a patent for a display they believe can greatly mitigate the screen door effect.
Sandwiched between this display’s lens and the eye’s view is a phase optic, which consists of an array of microlenses (tiny lenses less than a millimetre in diameter, often in micrometres!). These would magnify the pixels and scatter light, making the meshing of gaps much more subtle. While this is probably the most sophisticated method mentioned, it’s also very expensive and fiddly to implement, so it may be a long time before we see this technique in use.