Smart speakers and AR/VR headsets brought innovative experiences and enabled users to access computers intuitively. More devices will enable machines to discern human behavior and replicate human senses more precisely, minimizing human-IT frictions. The interaction will be naturally integrated with the surrounding environment providing intuitively augmented experiences to users.
A revolution is underway in interface technology. Typical examples include user interface (UI) devices collectively known as XR1 that connect humans to virtual reality and cloud-based voice UI services that permit voice-based interaction with IT devices.
The VR head-mount display (HMD), a UI device that offers a strong immersive experience in the virtual world, is one of the XR devices becoming popular particularly in entertainment. Partly because the specifications of some VR HMD devices have been standardized, personal computer companies are releasing new products at a rapid pace.
Augmented reality (AR) is also becoming popular with smartphones. Generally, the AR presented on smartphones is lower in accuracy between the virtual world and the real world. This is because a smartphone acquires the positional information required for AR based solely on information from the camera installed in the unit and the gyro-sensor. No special hardware is attached to the smartphone. However, the accuracy provided by this setup is enough for many applications. The hundreds of millions of smartphones sold each year represent a gigantic group of potential users and presents a major opportunity for application developers. Many new apps are being added to the market, expanding and creating a virtuous diffusion cycle.
Voice UI services have also been used in a wide variety of products, and are provided by large vendors as a cloud-based AI via the web. This service recognizes the human voice and returns a response in a natural-sounding voice. The voice UI service works in multiple languages and in cooperation with internet search engines and more than 10,000 other services soon to be available worldwide including ecommerce, travel and car sharing. As a result, voice UI services are establishing a global dominant position as a gateway to voice-related services.
1 XR is a collective term for virtual reality, augmented reality and merged reality
The functions, weight and size of VR HMD have also evolved, with many high-end VR HMDs about to be launched. These new displays have vastly improved resolution quality, lower weight and enhanced adaptability. The traditional VR HMD, which is a heavy device that hangs over the head and uses a wide cable to connect to a computer, was a market barrier for its expansion to casual users. To address this, two new types of VR HMDs are expected to be introduced to the market for an improved user experience, including a standalone type that uses a VR HMD and requires no computer connection and a smaller, lightweight, thin model that aims at being close to eyeglasses in size.
Business applications for XR devices are also expanding. For example, the 3D-coordinated design system that uses VR HMDs designs and simulates industrial products in 3D instead of on flat design drawings. This system also enables cooperative work among multiple people in different locations. Although these ideas existed before, the evolution of highly accurate VR HMDs and powerful computers have brought the concept to a practical level. It is breakthrough technology that, for example, lets a user break down a vehicle made with a huge number of parts to the last screw, and expand, reduce and rotate images at will to view them in a 3D immersive space. These types of systems have the potential of innovating CAD-based, 3D design that requires experience and craftsmanship, leading to further efficiency.
The number of different types of products with a voice UI service is also increasing. At its onset, voice UI service became popular as a smart speaker associated with a music service. Currently, this service is installed in many household appliances as well. For cloud vendors that offer sophisticated voice recognition services, competition is heating up for technology development, market share and technical dominance. For instance, home appliance manufacturers have made multiple cloud AIs available for selecting product features, allowing users to make their own choices directly. It is probably safe to say that the focus of competition for home appliance manufacturers has shifted to the attractiveness of the product itself and the effectiveness of its voice UI. In addition, such interfaces are being introduced to office machines, where competition will likely intensify.
The ultimate challenge of UI development is the transparency of UI. If the interaction between humans and IT devices can be achieved using natural behavior and perception inherent in humans, humans should then be able to use the devices as if they were interacting with other humans. It would not be necessary to learn how to use the device, or for users to be fully aware that the other party is not human. UI innovation continues to evolve from punch cards first used to convey numerical values to the computer, to keyboard and a display and then a GUI and mouse. More recently, the touch interface appeared as a more natural interface for people. In either approach, interfaces are moving closer to a more natural interaction between people and computers. However, none of these are truly an organic form of interactivity since users remain aware that they are communicating with computers and must acquire proficiency in using these devices. The industry goal is a transparent interface that can be used without a learning curve.
The solution to make human-to-device communication transparent is within sight. For example, the ability to detect calls from human voice, gestures and communication with expression is being developed through high-accuracy sensors, AI-supported voice recognition and image recognition technologies. Further increases in accuracy is anticipated.
On the other hand, the development of techniques to make humans naturally detect stimulus from an IT device is underway. Humans receive stimuli from the external world in many ways such as the five senses of touch, taste, hearing, sight and smell. While UI devices designed for the senses of eyesight and hearing are well developed, the reproduction of technology for taste and smell, and particularly the sense of touch, which plays a major role in perception, are still immature. Even the latest research is far from reaching the level where the stimuli to the human hand can be reproduced.
This has led to an alternative approach for a transparent UI without the need of a device to reproduce stimuli to humans. Instead of directly reproducing the sense of touch or taste, this approach uses existing devices to simulate the sense of touch. An example of this approach is a study that uses VR HMD to elicit the sense of touch from the sense of eyesight. By holding a controller and moving one’s hand, it makes the hand in the virtual space move. Picking up an object lets the user experience how the object changes. The user feels a simulated touch — even the object’s weight. Information from the sense of eyesight can also change the taste of food. Even walking along a curved wall the user gets the feeling of walking straight forward based on the visual information available. Thus, a transparent UI might become available soon if studies accelerate the simulation of senses using existing interfaces.
After the aggregation of completed transparent UIs, super-intelligent interfaces might enable a simpler and more serene world. When a person’s natural communication method is detected and the response is sent back to a person in a similar way, users of transparent UIs will be able to leverage IT with an unprecedented level of productivity. In fact, people will behave naturally and may not even be aware that they are using IT.
Super-intelligence interfaces will result in an optimized user experience (UX) for each individual. In addition, it will be important for UI-based services in the future to use software-based human senses to offer personalized ease of use in performing tasks the best possible way. For example, when reproducing a relaxing environment in which the user can focus on a task, the reproduction of a particular realistic environment is not important. What is required is an optimal environment. As a result, the system might completely filter out unnecessary stimuli or distracting factors for the user. In a super-intelligence interface world, people would probably experience a relaxed feeling in their own space, while using the robust abilities of IT without being aware of it, thus achieving high productivity.