Augmented reality (AR) is an experience where designers enhance parts of users’ physical world with computer-generated input. Designers create inputs—ranging from sound to video, to graphics to GPS overlays and more—in digital content, which responds in real-time to changes in the user’s environment, typically movement.
See the differences between Augmented, Virtual and Mixed Reality here.
Augmented reality has science-fiction roots dating to 1901. However, Thomas Caudell described the term as a technology only in 1990 while designing to help Boeing workers visualize intricate aircraft systems. A major advance came in 1992 with Louis Rosenberg’s complex Virtual Fixtures AR system for the US Air Force. AR releases followed in the consumer world, notably the ARQuake game (2000) and the design tool ARToolkit (2009). The 2010s witnessed a technological explosion—for example, with Microsoft’s HoloLens in 2015—that stretched beyond AR in the classical sense, while AR software became increasingly sophisticated, popular and affordable.
The term XR includes AR, MR, VR, and any technology that blends the physical and the digital world.
© Interaction Design Foundation, CC BY-SA 4.0
Under the umbrella term extended reality (XR), AR differs from virtual reality (VR) and mixed reality (MR). Some confusion exists, notably between AR and MR. Especially amid the 2020s’ technology boom, considerable debate continues about what each term covers. In user experience (UX) design, you have:
AR: You design for digital elements to appear over real-world views, sometimes with limited interactivity between them, often via smartphones. Examples include Apple’s ARKit and Android’s ARCore (developer kits), the Pokémon Go game.
VR: You design immersive experiences that isolate users from the real world, typically via headset devices. Examples include PSVR for gaming, Oculus and Google Cardboard, where users can explore, e.g., Stonehenge using headset-mounted smartphones.
MR: You design to combine AR and VR elements so digital objects can interact with the real world; therefore, you design elements anchored to a real environment. Examples include Magic Leap and HoloLens, which users can use, e.g., to learn more directly how to fix items.
Because of the slight overlap regarding interactivity, brands sometimes use AR interchangeably with MR. “Augmented reality” remains popular—despite the point that the original sense of AR design is overlaying digital elements upon real-world views, e.g., GPS filters/overlays on smartphone screens so users can find directions from street views. So, digital elements are merely superimposed on real-world views, not anchored directly to them: The computer-generated content can’t interact with the real-world elements users see—unlike in MR. The HoloLens is MR, for instance, because it interprets the space in a room and combines digital objects with the user’s physical environment.
Augmented reality combines real-world sensory input with computer-generated real-time content using SLAM (Simultaneous Localization and Mapping).
The three stages a device goes through to display AR content.
© Interaction Design Foundation, CC BY-SA 4.0
This process involves three main steps:
Sensing and Tracking: The AR device senses the environment with cameras, accelerometers, gyroscopes, GPS, and even lasers to track the position and orientation of the user and their device.
Image Processing and Recognition: The system analyzes the sensor data and identifies objects or features in the environment that can be augmented. The device uses image processing and recognition algorithms to identify and track objects in real time.
Rendering and Display: The final step is to generate and display the computer-generated content on top of the real-world environment. This step renders and displays virtual objects in the correct perspective and position relative to the user's viewpoint. To the user it should seem as if the objects are really there, like a hologram.
AR designers made considerable strides in the 2010s—a decade full of invaluable AR lessons and examples while the required sensors became cheaper.
Pokémon GO is noteworthy, a GPS-oriented, social AR app that “inserts” Pokémon characters into users’ environments so users can find and capture them on device screens.
Google’s AR stickers are another prime example; users drop realistic images into their camera shots. Users find AR particularly appealing for its entertainment value. Still, AR’s mainstream future appears assured across a wide range of applications, including education inside museums. With AR applications, you can bring experiences closer to users in their environments through designs that are more directly engaging, personalized and—indeed—fun. Storytelling is a great way to achieve this.
“Augmented reality is going to change everything.”
— Tim Cook, Apple’s CEO
A UX designer for AR needs to understand context of use, which refers to the specific situation or environment in which the users will use AR technology.
In this video, Frank Spillers, founder of UX consultancy Experience Dynamics, covers the key characteristics of context of use that you should consider to create successful AR experiences.
Copyright holder: T0KEEYO Appearance time: 0:45 - 0:49 Copyright license and terms: CC BY Link: https://www.youtube.com/watch?v=cUpCBHmPCPo&ab_channel=T0KEEYO
Copyright holder: Lone Fox Appearance time: 0:49 - 0:53 Copyright license and terms: CC BY Link: https://www.youtube.com/watch?v=FshzYgDXS44&ab_channel=LoneFox
To design successful experiences for AR consider:
Safety: Remember users’ real-world contexts; don’t distract/mislead them into danger.
Overkill: Beware of drowning users’ senses with meaningless data; keep experiences contextualized.
Environment: Unlike desktop experiences, AR happens anywhere. So, the aim is primarily for users’ contexts regarding whether they’re outdoors/indoors and moving/static. Whatever their setting, users expect pleasurable, user-friendly experiences. AR UX’s Rob Manson stipulates user scenarios:
Public: Interacting with software, using the entire body
Personal: Using smartphones in public spaces
Intimate: Sitting, using a desktop
Private: Using a wearable
Comfort: Make comfortable designs to prevent physical strains and reduce cognitive load.
Security: AR data is rich, so design to ensure users’ data is secure.
Familiarize yourself with AR terminology, such as spatial cognition, bodystorming, Field of View and task load.
Constantly ask “Where are users?” and how they’ll apply and adopt your design.
Remember physical limitations—users hold devices longer while seated, etc.
Make interfaces automatic so users needn’t be prompted with commands. Consider voice controls.
Use AR-software-creating resources optimally (e.g., Apple’s ARKit).
Offer easy onboarding.
Provide clues and maximum predictability.
Prioritize screen real estate.
Design for accessibility.
Design animations where you consider how frame rates and processing power impact device compatibility.
Ensure your design interprets and responds to users’ head and body movements so users can act intuitively and freely without giving commands.
Ultimately, understand what users—in various contexts—expect before you try to meet the experience demands. Do user testing that covers all feasible conditions (lighting, weather, etc.).
Learn how to design your AR experiences with our UX Design for Augmented Reality course.
Find some vital AR considerations here in Designing for Augmented Reality.
Read a specialist’s detailed take on AR in The Principles of Good UX for Augmented Reality | by Tyler Wilson.
You can see Apple’s guidelines for designing for AR in Augmented reality | Apple Developer Documentation.
Read more about AR-MR-VR differences in the article Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR?
Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) each offer unique digital experiences, but they differ in how they treat the real and virtual worlds.
AR overlays digital elements onto the real environment through devices like smartphones or AR glasses—think of Pokémon Go or IKEA’s Place app.
VR, in contrast, creates a fully immersive digital environment that replaces the real world entirely, usually through headsets like the early Oculus Quest. Users in VR interact only with the simulated world.
MR combines the two, as the name suggests. It anchors digital objects into the real world so users can interact with both simultaneously, using devices like Microsoft’s HoloLens, Meta (previously Oculus) Quest 2 and later.
Each technology serves different needs—while AR enhances real-world experiences, VR transports users elsewhere, and MR blends digital and physical spaces for highly interactive experiences.
Watch as CEO of Experience Dynamics, Frank Spillers discusses AR:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
Augmented reality (AR) applications enhance how users experience, interact with, and customize products or spaces. For example, in interior design, apps like IKEA Place allow users to visualize furniture at home by placing true-to-scale 3D models into real rooms. In fashion design, AR mirrors help customers try on clothes virtually, improving online shopping experiences. Automotive designers use AR to project concept models onto real environments, speeding up prototyping and user testing. TeamViewer Frontline is an enterprise productivity platform with fully integrated AR solutions to enhance productivity and efficiency in various industries. Examples also extend to the medical world, where surgeons use AR overlays to guide the cuts they need to make. Of course, AR features in game design, too, with Pokémon GO featuring as a leading example.
Watch as CEO of Experience Dynamics, Frank Spillers discusses AR:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
To design UX for augmented reality (AR) applications, focus on blending digital elements naturally into the real world and according to what users of your app expect to find and do. Always prioritize context—ensure virtual objects align realistically with users’ environments. Design intuitive interactions that mirror real-world behaviors, like tapping, grabbing, or swiping. Keep interfaces minimal and don’t clutter the user’s view with too much information. Use spatial audio cues and visual anchors to guide attention subtly. Always maintain good contrast and readability, even in varied lighting conditions.
Also, remember to design for the human body and consider ergonomics: minimize arm fatigue by designing interactions that require comfortable, natural gestures. In any case, user testing is critical—so, test with real-world conditions to catch usability issues early and bring AR design closer to your users.
Watch as Frank Spillers discusses AR:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
In AR environments, the best UI elements are those that feel natural, simple, and context-aware. Floating buttons, minimal menus, and gesture-based controls work especially well because they do not overwhelm the user's view. Anchored UI elements—such as tooltips or navigation markers—help users stay oriented without breaking immersion. Spatial audio cues act as powerful UI signals, too, by directing attention or confirming actions. Use clear, high-contrast visuals to maintain readability in changing lighting conditions. Interactive elements must be large enough for easy selection without requiring precise movements. Avoid clutter; prioritize essential information to reduce cognitive load. The best chosen elements and thoughtful UI design make AR experiences feel seamless, boosting both usability and user satisfaction.
Watch as CEO of Experience Dynamics, Frank Spillers discusses AR:
Enjoy our Master Class How To Craft Immersive Experiences in XR with Mia Guo, Senior Product Designer, Magic Leap.
To balance real-world and digital elements in AR interfaces, prioritize enhancing the environment instead of overwhelming it. Place digital content in ways that respect the user’s physical surroundings—avoid blocking key real-world features like pathways or objects. Design UI elements that blend naturally into the user’s space, using semi-transparent visuals and subtle animations. Always anchor important information to relevant real-world objects or areas to maintain context. Keep digital elements lightweight and minimal to prevent clutter and visual fatigue.
Test your designs in diverse real-world conditions, such as different lighting and movement scenarios, to ensure consistency. Smart balancing makes the technology feel invisible, a vital part of a seamless experience. It helps users stay immersed in their real environment while benefiting from digital enhancements.
Watch as Frank Spillers discusses AR:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
Spatial design principles in augmented reality (AR) focus on how digital elements interact with physical space to create natural and intuitive experiences for users. One key principle is anchoring—to tie digital objects to real-world surfaces so they stay stable and believable. Depth perception is crucial; so, use shadows, scale, and occlusion to make digital elements feel grounded in the environment. Maintain clear spatial hierarchy by putting important elements within the user’s immediate view and secondary content further away. User movement must feel effortless, so avoid forcing users to make extreme gestures or move too much to interact. Always account for environmental variability like lighting, textures, and obstacles. Thoughtful spatial design turns digital overlays into seamless parts of the real world—and helps pave the way to seamless AR experiences.
Watch as Frank Spillers discusses AR:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
When designing for mobile AR, consider hardware limitations first—smartphones have smaller screens, lower processing power, and limited battery life compared to dedicated AR devices. Keep experiences lightweight to prevent overheating and lag. Mobile cameras and sensors can struggle with tracking in low light, reflective surfaces, or cluttered environments, so design interactions that allow for tracking errors. User ergonomics are another major factor; prolonged device holding causes fatigue, so limit session lengths and encourage breaks. Also, mobile AR often requires standing or moving around, so always prioritize user safety and provide clear spatial boundaries. Finally, ensure accessibility by designing intuitive interactions that work for users of all abilities. Research shows that 39% of users abandon mobile AR apps due to performance or usability issues—designing around limitations helps deliver smoother, more satisfying experiences.
Watch as Frank Spillers discusses AR:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
The biggest challenges in designing for AR revolve around blending digital content seamlessly with the real world. One major hurdle is environmental variability—AR experiences must work across different lighting conditions, textures, and spaces. Also, user comfort poses a challenge; a user holding a device for long periods or using awkward gestures can become fatigued. So, prioritize ergonomic interactions. Spatial accuracy remains a challenge, as imperfect tracking can lead to floating or jittery digital objects that break immersion. Designers must also carefully balance visual complexity, ensuring digital elements don’t clutter or obscure the real world. Last, but not least, designing intuitive onboarding is critical since many users are new to AR experiences.
Watch as CEO of Experience Dynamics, Frank Spillers discusses AR:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
When designing AR content, cultural considerations are crucial for creating respectful, relatable, and inclusive experiences. Always tailor visual elements, gestures, and symbols to fit the cultural context of your target audience. For instance, colors carry different meanings across cultures—while white symbolizes purity in Western countries, it often represents mourning in parts of Asia. Avoid culturally insensitive imagery, language, or humor that could alienate users. Consider local norms regarding privacy and public interaction, especially for AR experiences that involve location data or shared spaces.
Social AR and respect for personal space are important considerations in any case, but they can be even more vital to get right when designing for other cultures. Adapt voiceovers, text, and even body language in AR avatars to respect cultural communication styles. Thoughtful cultural design not only prevents misunderstandings but also builds stronger emotional connections with diverse audiences.
Watch as Author and Expert in Human-Computer Interaction (HCI), Professor Alan Dix explains important points about designing with culture in mind:
Copyright holder: Tommi Vainikainen _ Appearance time: 2:56 - 3:03 Copyright license and terms: Public domain, via Wikimedia Commons
Copyright holder: Maik Meid _ Appearance time: 2:56 - 3:03 Copyright license and terms: CC BY 2.0, via Wikimedia Commons _ Link: https://commons.wikimedia.org/wiki/File:Norge_93.jpg
Copyright holder: Paju _ Appearance time: 2:56 - 3:03 Copyright license and terms: CC BY-SA 3.0, via Wikimedia Commons _ Link: https://commons.wikimedia.org/wiki/File:Kaivokselan_kaivokset_kyltti.jpg
Copyright holder: Tiia Monto _ Appearance time: 2:56 - 3:03 Copyright license and terms: CC BY-SA 3.0, via Wikimedia Commons _ Link: https://commons.wikimedia.org/wiki/File:Turku_-_harbour_sign.jpg
Learn how to design AR experiences with our UX Design for Augmented Reality course.
To make AR experiences accessible to people with disabilities, design flexible, multi-sensory interactions that accommodate different needs. Provide audio descriptions for visual elements to support users with visual impairments, and ensure that gestures have voice-controlled or touch alternatives. Include captions for any audio content to assist users who are deaf or hard of hearing. Avoid small, precise interactions—use large, easily reachable elements for users with motor impairments. Offer customizable settings like adjustable text size, contrast, and control sensitivity.
Always test your AR experiences with assistive technologies such as screen readers and voice recognition software. Users come in many shapes and sizes and have many levels of ability, so stay mindful as you design in AR. Thoughtful accessibility in AR not only meets ethical and legal standards; it also enhances usability and engagement for all users.
Watch our video about accessibility and its importance in design:
Learn how to design AR experiences with our UX Design for Augmented Reality course.
Schmalstieg, D., & Höllerer, T. (2016). Augmented Reality: Principles and Practice. Addison-Wesley Professional.
Augmented Reality: Principles and Practice by Dieter Schmalstieg and Tobias Höllerer offers a comprehensive exploration of augmented reality (AR), integrating insights from computer vision, graphics, and human-computer interaction. The book delves into technical foundations such as display systems, tracking, calibration, and registration, while also addressing visualization, interaction, and software architecture. Its balanced approach between theoretical concepts and practical applications makes it an essential resource for UX designers, developers, and researchers aiming to create effective and engaging AR experiences. By bridging engineering principles with user experience considerations, this work stands as a critical reference in the field of AR.
Papagiannis, H. (2017). Augmented Human: How Technology is Shaping the New Reality. O'Reilly Media. https://www.oreilly.com/library/view/augmented-human/9781491928325/
Augmented Human by Helen Papagiannis explores how augmented reality (AR) is transforming our experiences of the world by enhancing human capabilities across domains such as art, health, communication, and education. Written for a broad audience—including UX designers, technologists, and cultural analysts—it examines both the current state and the future trajectory of AR. The book emphasizes storytelling, design thinking, and the human-centric philosophy that should guide emerging technologies. By grounding its insights in real-world examples and visionary use cases, it equips UX professionals with a deep understanding of how to design engaging and meaningful AR experiences that truly empower users.
Hillmann, C. (2021). UX for XR: User Experience Design and Strategies for Immersive Technologies. Apress.
UX for XR by Cornel Hillmann is a vital resource for UX designers working with immersive technologies like augmented reality (AR), virtual reality (VR), and mixed reality (MR). The book explores the unique challenges of designing user experiences in spatial and multi-sensory environments. Hillmann covers key topics such as spatial interfaces, user testing in XR, and multi-modal feedback systems. What sets this book apart is its practical guidance rooted in real project experience, making it both theoretically insightful and immediately applicable. It empowers designers to build more intuitive, inclusive, and effective XR applications, making it a key text for advancing user-centered immersive tech design.
Dirin, A., & Laine, T. H. (2018). User experience in mobile augmented reality: Emotions, challenges, opportunities and best practices. Computers, 7(2), 33.
This article examines user experience (UX) in mobile augmented reality (MAR) by focusing specifically on users’ emotional responses during interaction. Dirin and Laine define UX as the emotional journey experienced with an application, and through two case studies—including a Virtual Campus Tour MAR app—they analyze common challenges, opportunities, and best practices. Their findings highlight emotional engagement, technical limitations, and contextual design as critical factors for successful MAR applications. Importantly, they synthesize research and empirical results to recommend practices for improving emotional resonance and usability in future MAR designs. This paper is influential for emphasizing emotion-centered UX design in the rapidly evolving MAR landscape.
Stumpp, S., Kretschmer, A., & Latoschik, M. E. (2019). User experience design with augmented reality (AR). In Mensch und Computer 2019 (pp. 809–812). ACM.
This paper presents a framework for developing user experiences tailored to augmented reality (AR) systems. Stumpp, Kretschmer, and Latoschik argue that traditional UX methods need adaptation to address AR's unique challenges, including spatial interactions, mixed reality blending, and real-world context sensitivity. They introduce specific design dimensions and provide conceptual guidance for enhancing AR usability and engagement. Importantly, this work highlights the cognitive and ergonomic factors critical for AR application success. It has been influential because it bridges gaps between general UX design theory and the practical demands of immersive AR environments, offering a targeted model for future AR user experience research and practice.
Remember, the more you learn about design, the more you make yourself valuable.
Improve your UX / UI Design skills and grow your career! Join IxDF now!
You earned your gift with a perfect score! Let us send it to you.
We've emailed your gift to name@email.com.
Improve your UX / UI Design skills and grow your career! Join IxDF now!
Here's the entire UX literature on Augmented Reality (AR) by the Interaction Design Foundation, collated in one place:
We believe in Open Access and the democratization of knowledge. Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.
If you want this to change, , link to us, or join us to help us democratize design knowledge!
