Technology to capture real world environments has become commonplace, allowing the replication of objects and space in precise size and positional relationship. Generated environments with augmented information will become an entirely new medium transforming business and individual perception.
Spatial computing is a technology that allows people to extend the advantages of a computer beyond the scope of a screen by using surrounding spatial environments. With this technology, computer graphics (CGs) are projected in real space. These are not images to see, but rather a space into which one can enter and use information that the computer provides. Use of this technology is growing at an unprecedented rate, yet its advantages are difficult for people to comprehend fully without actually experiencing it.
Although spatial computing is still in its infancy, anyone can experience it using an augmented reality head-mounted display (AR-HMD) or a smartphone. It consists of simple computer graphics of real objects at actual size that get overlaid on the real world as if they were actually present. As a result, one can walk around and see these computer-generated objects, and even touch them.
A typical application for this technology is spatial planning. The conventional way to conduct ex-ante confirmation of a delivery route, installation location and workspace for large equipment uses a 2D diagram or a small-scale model. A superior alternative uses spatial computing to display a full-size CG of the equipment in the actual factory space. Workers can view inside the factory to visually check the equipment from all angles. This not only reduces time and effort but prevents errors due to unforeseen circumstances.
Another common example of spatial computing use is to provide work assistance by including more observational information. For example, to inspect a finished piece of an industrial product on the factory floor an inspector visually compares the product with the procedure book. Spatial computing, on the other hand, can clearly show the key points of inspection and how to use pertinent inspection tools by superimposing CGs directly on the actual object. Spatial computing used in medical surgery, where the image gained from a test can be superimposed accurately on a patient’s body, is another beneficial use of this technology.
A critical factor when using spatial computing is the accurate size and positioning of the object shown via CG in real space. For the purposes of this white paper, “3D data” refers to the real-scale data that indicates the dimensions of an object, while a “3D map” indicates the real-scale space and position in which the object is placed.
Large-scale, dedicated equipment is needed to digitize objects and space. For example, detailed 3D data of a person is produced by placing a person wearing markers in a dome-like facility with multiple, carefully positioned cameras. Huge numbers of photographs are then synthesized through data processing to generate 3D data. Although smaller devices have been developed, part of the task still requires the knowledge of experts.
However, AI has made creating 3D data far easier. Object estimation technology that leverages machine learning generates 3D data based on several photographs taken of an object. Better still, the entire undertaking can be completed on a smartphone. The development of technology that constructs high-quality 3D data using fewer photographs and cameras is only now showing promise.
The emergence of technology that lets individual users use their own devices to conveniently generate 3D data and 3D maps is fueling the popularization of spatial computing. Low-cost, high-accuracy sensors, which have been rapidly developed over the past ten years, are making the generation of 3D maps more convenient. Some head-mounted displays can now convert a space to a 3D map immediately in front of a user. Currently, resultant 3D maps are rather crude, consisting of simple lines. Yet they are sufficient to understand the size and position of an object placed in a room.
One can only imagine the impact of integrating 3D maps generated by individuals with those in the public domain. For instance, companies involved in self-driving cars have launched a platform that integrates detailed road information collected by running vehicles. 3D maps that contain the size and position information of every building, sidewalk, tree and street sign in cities where people move about will be pasted onto high-definition satellite images. Based on these, the direction in which to go is shown in the streetscape in front of the user, while the signs of restaurants and bars display their customers’ evaluations and word-of-mouth messages. As real-scale 3D maps continue to become available, the value of spatial computing will increase dramatically.
The convenience of spatial computing will progress further when CGs reach photo quality. This value will be particularly evident in the marketing of products and experiences. Imagine having the ability to try on clothes, visualizing yourself dressed in different locations and backgrounds, such as a business meeting in an office or at a cocktail party at a social venue.
The current state of spatial computing, however, is far from photorealistic. This is because today’s limited computer performance gives priority to real-time characteristics over photo-real attributes. For instance, when spatial computing displays movement of a person’s head and eyes, a delay time of over 0.02 seconds gives users a sense of discomfort. This precedence given to real-time attributes results in a compromise in image quality.
To overcome this issue, technologies have been proposed to lighten the processing loads of head-mounted displays. One promising technology called “foveated rendering” has proved highly effective. It displays in high definition only the parts of an image on which the human eye is focused. In addition, other trials are using photorealistic images generated on a cloud via a minimum-delay 5G network. The perfection of such technologies that provide photo quality in real time will catapult spatial computing to the next level.
Spatial computing is garnering increased attention and is generating great enthusiasm within the development community. Companies that are aggressively promoting digital business are paying enormous attention to this technology in order to transform customer experiences. One firm spent several years successfully providing spatial computing technology on smartphones as a standard feature. Others are releasing innovative head-mounted displays and creating spatial computing in the cloud. In addition, technology such as spacial anchors and rendering are making possible multi-user, spatially aware mixed realty experiences, which in turn can harness the power of collective wisdom. The competition is increasing, particularly in the area of customer contact.
Behaviors of people themselves will also likely change due to future advances in spatial computing. Consider a day when instead of manually entering words to search for a topic on a smartphone, a person will simply look at a thing or location. For a generation that takes instant search results for granted, the way they view the world each day will change with spatial computing.