Augmented Reality

From SI410
Revision as of 17:04, 28 April 2019 by Gillfa (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Augmented reality used in Google Translate to superimpose translated text onto the original image[1]
Niantic Lab's PokémonGO AR view[2]

Augmented Reality (AR) is the integration of artificial elements onto a real-world environment by an electronic device, in order to supplement the user's perception in some way. The goals of this technology are ultimately to transform the real world into an interactive environment, particularly through the computer vision-based overlay of digitally created visual artifacts, though auditory and tactile elements can be implemented as well. Unlike virtual reality, augmented reality is meant to support reality instead of recreating a new one for the agent. AR can be implemented in wearable technology, similar to the approach of the suspended Google Glass[3], but is now used most prominently in applications for smartphones, one example being PokémonGO. Uses across a number of industries, including aviation, design, medicine, education, gaming, and the military, have revolutionized the workplaces and human lifestyle.

Collectively, AR systems aim to integrate external information with that of reality to enhance the user's day-to-day operations. Augmented reality can be analogized to a hologram, as it is presenting artifacts not physically there, but with an image presented on the surface of the device, rather than appearing in the physical location or spot. The first augmented reality device was credited by Ivan Sutherland in 1968[4], with the namesake term being developed by Boeing in the 1990s[5]. In 2016, the AR industry reached $1.1 billion in investments[6], and Microsoft Inc. released the HoloLens, a headset created to enhance daily life with the ability for developers to design their own apps specifically for the system.

Research and Development

Computer Vision and Software

The most essential components to functioning augmented reality are 1) the ability to recognize real images in order to superimpose artificial ones on top of them, and 2) computer-generated graphics that coincide with reality in some way. Computer vision is thus an essential element to integrating reality and virtuality, as is graphics processing[7].


Augmented reality hardware largely consists of wearable technology. Often a headset with goggles is used to overlay the virtual elements onto the view of reality. In some cases, like Google Glass[8], the hardware is minimal, but in others, like Microsoft HoloLens[9], the hardware is more substantial. More recently, smartphones, such as Apple's iPhone line or the various Android-supporting phones[10], can download AR-based applications. Developers can make use of the GPS and gyroscopic elements common in smartphone, as well as the wide variety of higher-quality cameras built into phones.



The history of augmented reality is rather diverse, and its past shows that it can be implemented for many different applications. Starting in 1968 with Ivan Sutherland, augmented reality began as a headset that projected wire-frame drawings, and in 1974, Myron Krueger built a lab of projectors that emitted shapes onto a screen, allowing people to interact with the new artificial "environment". Later, in 1990, Tom Caudell first termed "augmented reality" at Boeing.[11].

In 1998, the first commercial use of AR began in the NFL, to project first down markers onto the television viewing screens. By 2000, the ARToolKit by Hirokazu Kato was released in order to add computer graphics to video. This technology is still used today.[12]. And now, NASA employs the Hybrid Synthetic Vision system to overlay maps during flight.

In 2013, Volkswagen utilized AR glasses to display car repair information for mechanics via a system named MARTA. Shortly after in 2013, Google announced their Glass project, a set of glasses with augmented reality technology with the ability to take pictures and display information from the weather to your email.[13]

In April of 2017, Facebook CEO Mark Zuckerberg announced plans for Facebook to begin heavily investing in augmented reality technology. The announcement came in the midst of the 2017 F8 Conference where the company announced its intent to make AR Facebook's next "major platform." Facebook sees AR technology as an opportunity to enrich our lives and provide more profound ways to interact with friends and family over internet connections. At the F8 Conference, the company listed leaving virtual notes in real world places or playing AR games that "recognize and draw from real-world spaces" as potential applications for the technology.[14]

Aviation and Military

AeroGlass view during landing of small aircraft[15]

One of the largest challenges for aviation professionals is accurately and safely navigating an aircraft. Today, many aerospace companies utilize augmented reality simulations in training their pilots. Additionally, the company AeroGlass [16] has developed an augmented reality system for commercial and private aviation pilots. By superimposing relevant details using the goggle-like hardware over the view from the cockpit, they aim to make aviation safer and more stable. In 2016, AeroGlass was picked up by AirBus's BizLab for research and development[17]. Similarly, the United States military has rolled out AR reality helmets that supports their pilot's views during flight in high speed aircraft[18]. NASA has also used AR in their flight technology during the tenure of the Space Shuttle[19].

ARC4 by ARA. Hardware (left) and superimposed images (right)[20]

The United States ground forces also employ augmented reality helmets called ARC4 that allow them to see maps and more easily communicate with their counterparts on the ground. Developed by ARA, the goal was to allow for better coordination of ground forces during active duty[21]. By integrating day-time and night-time vision technology, the display provides real time information about the mission, the location of forces, and potential threats nearby. Also, for training purposes the military employs Augmented Immersive Team Training, a program designed to get soldiers out of simulators and into the field[22]. This type of technology can better prepare troops for combat missions and ultimately protect their lives in the face of danger.


AR's role in education is still being explored, but currently there are many applications that integrate augmented reality into education and the classroom. Research into how AR can be used to treat attention disorders such as ADHD and how it can be used to cater to differing learning styles in classrooms has been promising.

Google Tango is being integrated into museums, such as the Detroit Institute of Art, where visitors can rent out Tango supported phones and explore the exhibits through a new lens [23]. For instance, viewing a mummy from above using the Tango application can show an X-ray image of the skeleton below the surface of the sarcophagus. There are also applications available to create augmented reality-based works of art, which can be utilized in art classes.


Anatomy of the heart seen using Microsoft HoloLens[24]

AR can revolutionize medicine and healthcare. Ensuring treatment is quick and accurate can be life or death for some patients. As of 2016, but beginning much earlier, augmented reality applications have been used in the clinic. For instance, the application AccuVein helps clinicians find a patient's veins, and the application Anatomy 4D is used for studying anatomy. Surgeons are able to better understand their surgeries before and during the procedure, providing greater opportunity for success. Augmented reality can also be implemented for people with impaired vision[25].

But AR does not solely apply to the profession of medicine. Like in lower education, medical schools can benefit greatly from augmented reality so as to replicate real world situations and give experience to students of medicine.

IKEA augmented reality app to visualize furniture in your home


Augmented reality applications have increased in popularity in the design industry. There are numerous applications in both the Apple App Store and Google Play, the Android Application Shop, for design-centered augmented reality. in 2013, IKEA released an application allowing users to place virtual furniture in their homes using their smartphone's camera[26]. This gives customers the opportunity to visualize how their spaces will look before buying goods from IKEA in real time. Google Tango expands on this idea, giving users the chance to virtually transform their spaces with furniture and appliances. With Tango, developers can create their own applications for design-related purposes for personal and business use [27].

Foreign Language

One of the most useful ways that AR is being used is in the translation of foreign languages. The concept behind AR translations is that users could be using a device (AR glasses, phone, tablet) and when foreign language text is presented in front of the device, the smart AR program would augment the text into the translation of the user's desired language. This technology has not been fully developed, but it can be seen in several large companies. Google Translate, Google's online foreign language translation and dictionary program, has in particular put a substantial effort into developing AR translations, and have an AR feature on their current version of the app.[28]

Video Games

Smartphones now make augmented reality gaming far more accessible and can be tailored to individual needs.

Of the most recent AR games and applications, PokémonGO, developed by Niantic Labs, is of the most popular[29]. As in Nintendo's Pokémon series, the game focuses on catching virtual creatures named Pokémon, which then are able to train and battle those of others'. Released on July 6, 2016, the game skyrocketed to the top of the Apple App Store and Google Play charts and sparked a social interaction across the world around this common interest. Among achieving status as one of the most successful mobile games of all time, and breaking records like fastest to earn $100 million, and most-downloaded in first month of release, by 2018, Pokemon GO had grossed almost $2 billion in revenue, and had been downloaded 800 million times. While the game is not strictly based in AR, when catching different Pokémon, the user has the option to overlay the creature onto the image created using the smartphone's camera to make it seem like the Pokémon is actually right in front of you when you go to catch it using the game's PokéBalls. The game has been praised for influencing users to be outside, engage in physical activity, and exploring nature, in order to properly engage with the in-game events, features, and virtual Pokemon creatures. [30]


Cadillac's heads-up display[31]

Use in automobiles for augmented dashboards has become more popular for their "heads-up" displays. To help drivers keep their eyes on the road, many car companies have been installing augmented reality-based systems that project driving information onto the windshield in front of the driver. In contrast to most other AR technologies, the driver does not need a headset to view the augmentation[32].

Past and Present AR Devices

Below are two of the most well known AR platforms:

Google Glass[33]

Google Glass hardware
  • Created by Google
  • Released in 2014 to the public
  • Thin-framed glasses with camera and small screen in one lense
  • Can be integrated with other Google technologies, including Mail, Weather, Hangout, and Maps, just to name a few
  • Developers can create their own applications
  • Sales price of $1500.00
  • Discontinued after one version (though still in development by Google)

Microsoft HoloLens[34]

Microsoft HoloLens hardware[35]
  • Created by Microsoft
  • Released in 2016
  • Framed glasses with a screen over the lenses
  • Able to support Skype, a Microsoft product
  • Developers can produce their own apps
  • Sales price of $3000.00

AR vs VR

Augmented reality and virtual reality are related, but different from each other. Both technologies allow users to immerse themselves in an artificial reality, separated from real life. Virtual reality transfers users from their current state into an uninterrupted reality. To provide an uninterrupted reality, VR companies create aids to block the perception of users natural reality. These aids include things like closed visors and goggles. They allow users to be blocked off from their natural reality, while transposing them into an uninterrupted artificial reality. Augmented reality utilizes a different technique. Instead of blinding users of their natural perception to transfer users into a new reality, AR uses an aid to take users' current reality and provide something new to it. Rather than closed visors as an aid, AR uses items like sunglasses which could connect to other devices nearby such as PCs and smartphones.

Ethical Issues

Very little policy has been written to protect users from the dangerous AR presents, and virtually no existing policy has been updated to include AR technologies. As the product is mostly up to the developer, many concerns need to be addressed.


Many AR technologies require access to many different forms of sensory data, which could include video feeds, audio feeds, GPS location, and more, to function well. The issue arises with if and where this data is stored, along with who has access to this data. Additionally, companies need to keep this information secure from attackers who will try and gain access and or leak the data. [36] The rising popularity of Augmented Reality has made the topic of privacy in information technology controversial. Many people are concerned that augmented reality has weakened how people protect their personal information and has made it more vulnerable to be hacked. [37]/. Additionally, AR technology integrates many features that allow for exploitation of information and privacy. For instance, facial recognition technology integrated into wearable AR can identify people in real time, which can be used as surveillance [38].


Another growing concern with Augmented Reality is safety for those people who use AR applications. Especially true for AR games and navigation applications, such as Pokemon GO, which require the complete attention and focus of the user. [39] Research indicates that the human brain has limited capacity to execute multiple tasks at a time meaning that it is difficult to multitask[40] People who are playing this Augmented Reality games tend to focus on their mobile devices and not be attentive to the world around them which can put them in harms way such as being hit by a car and falling down. Some augmented reality applications have introduced measures to minimize potential harm like checking that the user is not driving or pausing during certain activities.


As with privacy issues, companies can potentially exploit the wearable technology to target individuals for their products or services. Given that AR technology could revolve around image recognition or geo-location, when one is in the vicinity of a checkpoint, advertisements can be displayed accordingly. While laws exist to protect consumers in established technologies, AR presents a gray area in the context of the law that can exploited, and because wearable AR like Microsoft HoloLens or Google Glass is highly personal, it can compromise the experience and autonomy of such a device [41].


  1. Google Adds 20 More Languages to Its Translate AR App Augmented Reality Trends, Retrieved 20 February 2017
  2. Iran is first country to ban augmented reality app Pokemon Go, Retrieved 19 February 2017
  3. Disconnected: Sales of Google Glass Suspended Pending Makeover NBC NEWS, Retrieved on 19 February 2017
  4. Ivan Sutherland A.M. Turing Award, Retrieved on 19 February 2017
  5. What is Augmented Reality (AR): Augmented Reality Defined, iPhone Augmented Reality Apps and Games and More Digital Trends, Retrieved 19 February 2017
  6. Infographic: The History of Augmented Reality Augment, Retrieved 18 February 2017
  7. Confluence of Computer Vision and Interactive Graphics for Augmented Reality CalTech, Retrieved 19 February 2017
  8. Google Glass Wikipedia, Retrieved 19 February 2017
  9. Microsoft HoloLens Microsoft, Retrieved 19 February 2017
  10. Android Android, Retrieved 18 February 2017
  11. Infographic: The History of Augmented Reality Augment, Retrieved 18 February 2017
  12. Infographic: The History of Augmented Reality Augment, Retrieved 18 February 2017
  13. 10 Augmented Reality Apps for Healthcare Medpage Today, Retrieved 20 February
  14. Chiu, Eric [International Business Times., 04. Apr. 2017].
  15. AeroGlass AeroGlass, Retrieved 17 February 2017]
  16. AeroGlass AeroGlass, Retrieved 17 February 2017
  17. Airbus picks Aero Glass for 6-month innovation accelerator AeroGlass, Retrieved 18 February 2017
  18. The F-35's $400,000 Third-Generation "Magic" Helmet Is Here Popular Mechanics, Retrieved 19 February 2017
  19. Augmented reality to help astronauts make sense of space, Retrieved 18 2017
  20. Google glass for war: The US military funded smart helmet that can beam information to soldiers on the battlefield DailyMail, Retrieved 19 2017
  21. Google glass for war: The US military funded smart helmet that can beam information to soldiers on the battlefield DailyMail, Retrieved 19 2017
  22. Augmented Reality Training Systems Destined for Marine Schoolhouses, Retrieved 19 February
  23. Google Tango Google Tango, Retrieved 18 February 2017
  24. 10 Augmented Reality Apps for Healthcare Medpage Today, Retrieved 20 February
  25. 10 Augmented Reality Apps for Healthcare Medpage Today, Retrieved 20 February
  26. IKEA catalog uses augmented reality to give a virtual preview of furniture in a room New Atlas, Retrieved 19 February 2017
  27. Google Tango Google Tango, Retrieved 18 February 2017
  29. Iran is first country to ban augmented reality app Pokemon Go, Retrieved 19 February 2017
  30. Chamary, 2018, Why 'Pokemon GO' Is the World's Most Important Game,
  31. Heads up display Wikipedia, Retrieved 23 February 2017
  32. What Is The Cadillac ATS Heads Up Display? autobytel, Retrieved 23 February 2017
  33. Google Glass Wikipedia, Retrieved 19 February 2017
  34. Microsoft HoloLens. Microsoft HoloLens Microsoft, Retrieved 19 February 2017
  35. Microsoft HoloLens. Microsoft HoloLens Microsoft, Retrieved 19 February 2017
  36. Roesner, Franziska, Kohno, Tadayoshi, Molnar, David Security and Privacy for Augmented Reality Systems April 2013. Retrived April 23, 2017
  38. Wassom, Brian D. Augmented reality law, privacy, and ethics: law, society, and emerging AR technologies. Waltham, MA, USA: Elsevier Syngress, 2015. Print.
  40. Mike Neal, Jon Cabiria, Jerri Lynn Hogg and Shane Pase. "Psychological keys to success in MAR systems," Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH), 2011 IEEE International Symposium On , vol., no., pp.1, 26-29, Oct. 2011.
  41. Wassom, Brian D. Augmented reality law, privacy, and ethics: law, society, and emerging AR technologies. Waltham, MA, USA: Elsevier Syngress, 2015. Print.