Augmented reality is a technology that combines virtual scenery or information with real physical environment and presents them interactively in front of users, so as to create a space shared by virtual and reality.In essence, augmented reality is a new interface technology that integrates software and hardware technologies such as positioning, presentation and interaction. Its purpose is to make users feel the virtual and real spaceSpatiotemporal correlationAnd integration to enhance users' perception and cognition of the real environment.
Augmented reality technology has three basic elements, namely, the fusion of virtual and real space, real-time online interaction, and three-dimensional registration of virtual and real space.The fusion of virtual and real space, emphasizing the coexistence of virtual elements and real elements, is the key to enhance users' perception of the real environment;Real time online interaction, emphasizing the real-time nature of interaction response calculation between users and virtual and real objects to meet the response requirements of users' senses to the time dimension;The three-dimensional registration of virtual and real space emphasizes the accuracy and intelligence of users' perception of space, which reflects the space-time consistency of the fusion of virtual and real space.These three elements are the key to realize the enhanced perception of the real environmentSpatial orientationDependent, therefore, AR systems usually rely onHelmetAnd other special equipment to present the effect of virtual reality integration.[2]
Historical development
Announce
edit
The development of augmented reality technology can not be separated from the research and development of virtual reality fusion presentation device, whose history can be traced back to the 1960s.In 1968, the father of computer graphicsIvan SutherlandThe idea of the Ultimate Display was put forward, and the first headworn augmented reality display was designed and implementedthe sword of Damocles , which is realized by the facilitytransmissionDisplays a simple wireframe model that can only be superimposed in front of the field of view.[2]
the sword of Damocles
In 1974, Myrib Krueger invented the Videoplace system, which can interact with the projection picture through the user's own silhouette.
In 1990, Boeing engineers Tom Caudell and David Mizell developed a penetrating head mounted display device to allow aircraft assembly workers to view the assembly drawings of cables in real time, and formally proposed the term "augmented reality".
In 1993, Louis Rosenberg of the US Air Force Research Laboratory developed the first immersive remote augmented reality system Virtual Fixture.Feiner proposed a famous knowledge driven augmented reality systemKARMA。
In 1994, Julie Martin realized the simultaneous performance of dancers and virtual scenes in the stage play Dancing in Cyberspace.
In 1998, Sportsvision used video enhancement technology in the live broadcast of the NFL League in the United States.
In 2000, Hirokazu Kato developed the ARToolkit, which is called the first internationally released computer augmented reality library.The emergence of ARToolkit has enabled many programmers to have simple and easy to use augmented reality development tools, which has effectively promoted the application and popularization of augmented reality technology.
In 2017, Apple and Google successively embedded the augmented reality system into their mobile terminals, and launched the augmented reality software development platforms ARKit and ARCore respectively.China's Shangtang Technology, NetEase and other companies also subsequently launched the independent augmented reality software development platform SenseAR and Insight AR.Augmented reality has gradually become a new technology to serve the society and the public.[2-4]
Fundamentals
Announce
edit
Augmented reality requiresthree-dimensional spaceTo complete the fusion of virtual and real vision, special display devices are needed to meet the needs of visual fusion perception.At present, the display devices of augmented reality can be divided into three categories according to the use method: wearable devices, mobile handheld display devices and space enhancement devices.
Wearable device
Wearable devices generally include helmet mounted displays, projectors, AR glasses, contact lenses, etc., which are worn on the head. The most widely used is helmet mounted displays.According to the display model of the real environment, the augmented reality HMD can be divided into optical penetration HMD and video penetration HMD.Through the transparent lens at the front of the user's glasses, the optical penetrating helmet reflects the light while passing through the light from the real world, reflecting the virtual scene into the human eye, thus forming a scene of virtual reality fusion.The video penetrating helmet captures the scene influence in real time through the binocular camera at the front of the eye, and superimposes the virtual scene on the video screen, which is presented in the binocular display configured at the front of the user's eye.
Mobile handheld device
Mobile handheld display devices, that is, smart phones, tablets and other mobile terminals with cameras and certain computing and drawing capabilities, use the built-in cameras to capture the real world, and then merge with the virtual world drawn by themselves to present it to users.
The space augmented reality technology directly projects the virtual world into the real world by using optical principles or special equipment. It generally projects the virtual image into a fixed space, such as projecting the image onto an object or into a virtual image in three-dimensional space.[2]
In addition to some visual display devices, auditory devices have also gradually become one of the research contents of augmented reality.Through audio signal processing technology, the Virtual Auditory Space, which is researched by Chen Fei of South University of Science and Technology, will reconstruct the sound source at any position in 3D space to reproduce the sound source orientation and generate realistic virtual auditory perception based on some special acoustic effects of the human ear and acoustic correlation algorithms.[5]
At the same time, some augmented reality devices are equipped with tactile devices, such as the vibration feeling generated by the mechanical actuator of the eccentric rotating mass (ERM) vibration motor.Eccentric rotary mass motors have different dimensions and vibration force specifications, resulting in different tactility.At the same time, the human sensory receptors can transmit small electrical pulses along the afferent nerve to the central nervous system through mechanical stimulation.For example, Tesla Suit DK1 system developed by Tesla Studios (St. Andrew, Scotland).
Technical characteristics
Announce
edit
(1) 3D registration technology
3D registration is to solve the problem of how toVirtual sceneKey technologies related to real scenes.3D registration technology binds the virtual scene to the coordinate system of the real scene, calculates the projection information of the virtual scene under the perspective with the user's movement and the change of the perspective, and fuses it to the image of the real scene, ensuring that the virtual scene and the real scene share the same space.When bothRelative staticThe position relationship and scale relationship between the virtual scene and the real scene are consistent with the camera's position and pose changes;When the two are moving relative to each other, it is necessary to use 3D registration technology to accurately solve the geometric information of the 3D moving scene and the motion trajectory of the camera.Because the human eye is very sensitive to the perception of the picture, if the 3D registration result is not accurate enough, the picture presented to the user will shake and the scene will drift, which will seriously affect the user's immersion experience.
(2) Realistic rendering
On the basis of 3D registration technology, virtual scenes or information need to be presented and fused visually.In augmented reality systems, there are generally two types of visual fusion presentation methods. One is to superimpose words and numbers or other information on the real scene in an appropriate visual way to realize the active push and efficient presentation of knowledge;The other emphasizes the real sense of existence of virtual scenes in the real environment, and draws realistic virtual scenes under unified lighting conditions.Generally, global light is usedLighting modelIt makes the high fidelity analog light transfer in the virtual environment, which is relatively time-consuming.With the continuous improvement of CPU and GPU performance, real-time global illumination rendering of virtual scenes or scenes is becoming possible with some sampling approximation and calculation methods.
(3) Human computer interaction
The goal of human-computer interaction is to take users as the center, use various devices or interfaces, interact with virtual or real goals, and complete the determined interactive tasks through information exchange.Augmented reality system allows users to interact in the three-dimensional space of virtual reality fusion, and complete the determined interaction task through information exchange.The augmented reality system allows users to interact in the three-dimensional space of virtual reality fusion, and operate virtual objects by connecting other devices. Therefore, the interaction system should avoid confusion in the process of information registration or presentation, so as to improve users' interactive experience.
In the interaction of augmented reality, there is no single, fixed input and output mode, nor the output of two-dimensional images on the display, so it is difficult to have a fixed, universal interaction design.Augmented reality mainly uses 3D interaction, which has higher degrees of freedom, more interaction methods, larger interaction tasks, and more complex 3D user interface definitions. The current interaction mainly includes (1) touch interaction (2) gesture interaction (3) voice interaction (4) physical interaction
application
Announce
edit
With the continuous development of augmented reality technology, it shows a broad application prospect.At present, it has been applied in digital entertainment, cultural education, medical diagnosis and treatment, industrial production, urban operation and maintenance, e-commerce and other fields.
(1) Digital entertainment
At present, digital entertainment is the most widely used field of augmented reality technology, which produces a large number of augmented reality related movies, television and games.For example, under the control of technology and operation teams, virtual idols active in various stages can perform various talent performances like real artists in online and offline scenes, grow and live in real society, and interact with fans.At the same time, a large number of mobile augmented reality games, such as the popular Pokeman Go, have attracted a large number of game players and greatly promoted the popularization of augmented reality technology.[6]
(2) Cultural education
Augmented reality technology has also been widely used in education.On the one hand, augmented reality can enhance the interaction between students and teachers and the interest of the classroom.Teachers can use physical props and augmented reality technology to dynamically and vividly present the nature and phenomenon associated with each knowledge point, so as to enhance students' understanding and perception of the knowledge they have learned.On the other hand, augmented reality technology can associate various visual media with the content of books, and automatically present the corresponding fusion media information through books.At the same time, let students experience the real related scenes in the classroom.[2][7]
(3) Medical diagnosis and treatment
Medical diagnosis and treatment is one of the earliest and most challenging fields of augmented reality technology.Augmented reality technology can show users medical related knowledge or examination results, and assist doctors to complete high-precision treatment operations.The biggest challenge of the application of augmented reality technology in medical diagnosis and treatment is how to improve the accuracy of identification and on-site positioning, so that medical personnel can more accurately perceive complex disease parts and avoid misoperation.
(4) Industrial production
In the highly digital and information-based industry, augmented reality technology can play a role in many stages, such as design and development, product production and maintenance services, with the help of the interactive integration principle of virtual environment and real environment, showing a very broad application prospect.The aviation industry was the first industry to introduce augmented reality technology.Because the digitized West Sydney is too large, it is difficult to grasp the digital information by the memory of the human brain alone, and the separation of the digital information from the real scene is also not conducive to the understanding, judgment and implementation of the operators on the scene, which reduces the efficiency of the digital information.The installation of complex pipelines and hundreds of kilometers of cables and connectors in the aircraft is the main battlefield for the application of augmented reality technology in the aviation industry at present. The augmented reality system guides the construction personnel step by step to accurately perform complex operations and achieve high-quality assembly through a powerful interactive virtual reality fusion display interface.[2]
(5) E-commerce
For e-commerce, augmented reality can create a new way of shopping and enhance consumers' shopping experience.Users can experience goods from a distance and make more satisfactory choices.In recent years, Tmall Mall has launched the "AR-GO". Consumers can not only place their favorite products in their homes with 1:1 effect in Tmall APP, but also walk around the room and observe from different angles and distances.
