With the rapid evolution and diffusion of virtual reality (VR) and augmented reality (AR) devices, the digital and real worlds are being further integrated and expanded. Human perception will become synthesized within new 3D spaces, allowing the sharing of knowledge and distributed experiences.
Head-mounted displays, which provide immersion experiences in virtual space, finally reached full-scale adoption in 2016. The technology began when different companies sought to replace human visual perception with that of virtual reality popularizing the VR-HMD (head-mounted display) devices in approximately 2010. These efforts coincided with the time when VR-HMD technologies became available at affordable prices. Typical units consist of a computer that generates virtual 3D-space images in real time, a position-tracking technology that tracks human movements and a small display that enables high-definition. With prices 1/10th of those in the past, these products are rapidly becoming popular in many households.
The experience of replacing human eyesight with a virtual space is a sensation that humans have never before experienced. It is completely different from that of 3D TV, where the audience passively watch images popping out of the screen. With a VR device, eyesight is completely covered by a virtual space. A movement of the user’s neck triggers the rendering of additional virtual space, which rapidly heightens the sense of immersion. In addition, moving the face closer makes an image look larger.
Successful VR-HMDs have inspired great anticipation for the development of a device that replaces tactile and other human senses with virtual reality. However, a method to re-create the sensation of touching an object, lifting it to feel its weight and feeling its movement or temperature as well as other sensations is still in development. An enormous technological barrier needs to be overcome before minute finger movements can be sensed together with appropriate tactile sensations provided as a response. To accomplish this, the development of an intuitive controller that minimizes the user’s recognition of isolation from the real world, and a combination of a visual effects device and a suit that enhances sensory experiences, will be required.
An HMD interface that connects the user to a virtual space still has significant room for improvement. For example, an HMD needs to be connected to the base unit of a computer with an extremely high-speed calculation ability using a thick cable. For this reason, the user with an HMD cannot move about at will. An HMD should optimally be wireless. The next best thing would be for one without a base unit, which is being proposed. This HMD copies an idea from a smartphone processor that has both an advanced calculation ability and power saving. As the built-in lenses become thinner, the current situation, where a large, heavy, black box is worn in front of the face, will gradually improve. This technology will also be combined with the reproduction of images with a definition of over 8K, which exceeds the level recognizable by the retina, and with a response delay of 1 millisecond or less. In addition, with the release of a more accurate, innovative and outdoor-ready position-tracking technology, the so-called ”drunken” issue, caused by the gap between the position information detected by humans and the virtual space, the difficulty in outdoor use, and other limitations, will probably be mitigated. There is also a proposal to develop a system that enables collaboration without HMDs by pasting a virtual space in the physical space, with the ability to interact with it using projection mapping technology.
The development of technologies called augmented reality (AR) and mixed reality (MR) also made significant progress during 2016. While VR means a complete replacement of the real-world view with a virtual reality, in AR the real world is blended together with virtual reality. Unlike VR, the goggles used in an AR HMD are transparent, letting the user see the real world. Virtual reality is pasted on real world space to create an intermingled world. Expected applications for this technology include learning, training and design. For example, the user can make a gigantic aircraft engine appear virtually in the room, observing its details at will by getting close to and into the object.
Many developers and forward-thinking companies have been focusing their investments on AR to speed its evolution. However, AR-HMDs currently have several technological obstacles and devices on the market are only for developers. One significant limitation is that the area viewable in virtual reality is still narrow, allowing virtual reality to blend with the real world only in a specific area. In spite of its present limitation, the superimposition of virtual reality to a human’s natural eyesight may someday lead to a revolutionary UI to connect humans to information systems.
AR is also expected to generate new applications designed for the flat screen of a smartphone. An example of the many commercial uses of such applications include a situation where the user takes photos of his/her room, and superimposes the image with that of a new piece of furniture prior to purchase. Although VR is used for product introductions and advertisements, more widespread adoption has been hindered by a psychological and physical resistance to wearing goggles. Smartphone AR can overcome this obstacle. Furthermore, AR may well be effective even under the current circumstances, where the development of a device that re-creates human senses, except for eyesight, is still in its infant stage. For example, delicate assembly work and training in medical practice require the use of actual tools or a full-sized model to repeat a task or operation while feeling a tactile sensation. AR can paste a patient body or a material that changes while being processed using a virtual reality. This enables effective and repeatable training that blends VR with a reality where an object such as a model and a tool can be physically touched.
As HMDs advance and improved interfaces are provided, people are expected to use virtual reality on a daily basis. An interface called VROS, which provides an integrated UI by combining different virtual spaces created by the computer, will emerge as well. This type of UI will probably blend the human-to-computer interactions with the real world, making us forget that it is a system UI.
However, the essence of the current fast-moving technical evolution may be a new exploration into the unknown world of human senses. Thanks to the diffusion of HMDs and the accompanying increase in the number of developers, as well as huge amounts of user feedback, in-depth knowledge about different human senses is being accumulated. For example, the size of an object identified by human eyesight has been assumed to change depending on the person’s attentiveness. This is a challenge for 3D content producers as they struggle to determine the display size of an object that makes humans feel “appropriate.” Some have reported that the actual-world action of catching a ball flying toward you is done with different movements or speed if the same action is presented through virtual reality. Knowledge such as this will not only be used in the recreation of human senses as a form of entertainment, but also in the creation of more natural interfaces, contributing to the improvement of the system-human relationship.