AWE XR USA 2022: Key Insights on Technology Innovations — XR and Metaverse

The Annual XR International Conference

Andi Sama — CIO, Sinergi Wahana Gemilang

In Summary
- AWE XR USA 2022, both in-person and virtual (Santa Clara, CA, USA).
- Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), Extended Reality (XR), Artificial Intelligence (AI), Bio-Interfaces, Haptics. 
- The transition from Web 2.0 to Web 3.0.

AWE is back!. The annual AR World Expo was held in mid-2022 in Santa Clara Convention Center, California, USA. This world’s 2022 most significant International AR (Augmented Reality), VR (Virtual Reality), and XR (Mixed Reality) hybrid event was both in-person and virtual on June 1–3, 2022.

Thanks to the advancements in technologies enabling the availability of COVID-19 vaccines within less than a year. Governments could start deploying vaccines to their people worldwide in early 2021. The expectation has been to achieve herd immunity in the shortest possible time, thus significantly slowing down the spreading of the virus.

While most Governments started to ease the regulations for following strict health protocols, some are still advising people to follow mask-wearing (especially indoors), including practicing social distancing (getting less and less as more business activities are getting back to normal) and regularly washing hands.

Ever since WHO declared Covid-19 a pandemic in March 2020, it has impacted many aspects of most people and industries worldwide, including regular International events such as AWE (AR World Expo).

AWE XR 2022 accepted only WHO-approved Covid-19 vaccinated participants (either fully approved or approved for emergency use authorization by World Health Organization).

The opening speech by Ori Inbar, the Co-Founder of AWE (June 1, 2022). Ori Inbar shared how XR is making dreams a reality and announced a USD 100K XR Prize that challenges the industry to fight climate change with XR.

AWE has been the medium for the XR ecosystem to connect since 2010. AWE.Live has also been available online 24x7 in the last few years. In June 2022, AWE was back in Santa Clara, California, for an in-person event.

Defining Metaverse

As of June 7, 2022, there is no clear definition of Metaverse in the Merriam-Webster dictionary, America’s most trusted online dictionary for English word definitions, meanings, and pronunciation. The search for Metaverse returned, “The word you’ve entered isn’t in the dictionary.” There may still be no clear definition in the near future as the field is still a work in progress.

However, Oxford Languages defines the Metaverse as “a virtual-reality space where users can interact with a computer-generated environment and other users.”

Oxford Languages definition for Metaverse.

Interaction activities can be either working or playing. A computer-generated environment can be anything that we can imagine. From a simple shopping complex where we can do shopping activities and perform transactions, a musical event in which we can participate as participants, an outdoor environment with multiple virtual objects, or even an outer space environment with unimaginable creatively created objects that can be presented in either AR, VR, or MR for instance.

The interested reader may refer to further readings on Metaverse “Mimpi menuju Open Metaverse — The Next Internet (R)Evolution?” (Andi Sama et al., 2022) and on NFT “Mari Mengenal NFT” (Andi Sama, 2022b). Articles are in the Indonesian language.

Highlights of AWE 2022

• Unity’s Chief Executive Officer, John Riccitiello offered his own definition of the ‘Metaverse’. According to John: “50% of World Games and 60% of AR/VR experiences are built on the Unity platform”. He continued by defining Metaverse as the “Next generation of the Internet that is (1) always real-time, and (2) mostly 3D, (3) mostly interactive, (4) mostly social, and (5) mostly persistent.

Unity’s CEO offered his own definition of the ‘Metaverse’.

• Qualcomm’s Hugo Swart announced the global release of the company’s Snapdragon Spaces XR Developer Platform.

Qualcomm enables a seamless integration from the smart glasses to various “hosts”, e.g. mobile phones, cars, routers, or the cloud. Presented during his main stage presentation “Your Ticket to the Metaverse: Spatial Computing Experiences with Snapdragon Spaces and What’s Next in XR”.

• Haptx allowed attendees to experience haptic by inviting them to be within the experience itself through sensing (experiencing real realism in interacting with 3D virtual objects in the virtual world — as if we are interacting physically with the actual objects themselves). Haptic gloves have 133 points of tactile feedback per hand. The gloves provide industry-grade performance targeted for professional applications.

A conference participant is experiencing Haptx at AWE USA 2022.

• Hololight — The platform to host and stream XR applications to XR devices, bringing the power of the cloud to mobile devices. This is a kind of Netflix for AR and VR applications.

Hololight — enables streaming of AR and VR applications direct to the AR/VR headsets.

• “MetaTr@versal: A Day in the Life”: A Play About Humanity in the Metaverse!. The session illustrates the new reality in real-time by embracing immersive technologies, and telling the story of a day in the life of Sophia as she traverses the Metaverse to create meaningful memories with her friends and family and creates hyper-connective pathways to excel in her career from anywhere in the world.

The Metaverse connects us virtually. It bridges the physical space between us so we can be together, wherever we are.

• Magic Leap demonstrated Magic Leap 2, the latest wearable targeted for Enterprise. It is not yet available for purchase. Magic Leap received a Best in Show AR award. Another exhibitor that received this award was OVR for Best Healthcare & Wellness Solution (Osso VR: Virtual Reality Surgical Training Platform). Other awards are discussed later in this article.

The Author is experiencing Magic Leap 2 smart glass by coming in person at AWE USA 2022.

• Snap announced its first AR Lensathon with a Global Overall First Place prize of USD 50K.
• Signify, Phillip’s company explained Lifi technology (it uses light to transfer data), an alternative to Wifi for line sight of communication enabling very high speed e.g. 250–1000 MBps (Yes, you are reading this correctly. It is the big “B”, meaning MegaBytes per second, not Megabits per seconds).

Trulifi, a Lifi Technology by Signify.

• Smart Glasses — It All Starts With the Display. Near Eye display for AR Eyewear by Dispelix is the enabling technology for many applications such as Microsoft Hololens, and Magic Leap wearable.

• Tiktok shared Effect House, enabling designers, developers, and creators to build AR effects and experiences on TikTok. Several creators were invited early to share their exciting initial experiences with the Effect House in the last few weeks. In addition to AR, Effect house is powered heavily by AI — especially in computer vision.

Tiktok Effect House.

• Meta’s AR Eyewear demonstrated advancements in optics (e.g. realistic 3D display on mobile phones, television, game, etc) that can “promote” 2D content to 3D content, creating a rich perception for the human eye. At the moment, the display size can be as large as 27 inches.
• NASA’s Marco Tempest demonstrated the magic of AR in real-time (Metaverse in a Box) —Marco received an Awesome award for the awesome and interactive demonstration.

Part of the demonstration, doing a virtual tour by “teleporting” to Mars in virtual reality.

Expo Hall & Playground were showing multiple advancements of technologies (2nd and 3rd day). There was also Startup Pitch Competition.

Panel discussions covered various topics such as “The Road from 3D Graphics to Metaverse”, “The Volumetric Future” — Building an effective & safe community in XR, “Investor Q&A”, “Designing & Developing for DEI + Accessibility”, “The Metaverse Protagonist: Storytelling Techniques for AR Entertainment”, and many more.

Mixed reality wearables are coming and with it an opportunity to turn users into protagonists in their own interactive story. Panel discussion on “The Metaverse Protagonist: Storytelling Techniques for AR Entertainment” discussed this, through the creation of an iOS AR-based experience app called “Forensic Detective”. Casting the user as the role of the lead character poses many narratives, technical, and usability challenges that have never before needed to be solved. This new form of entertainment is prolific and provides an opportunity for a new generation of storytellers.

Through invention, prototyping, and iteration, the Forensic Detective is created, an immersive augmented reality experience that rose to number 2 in entertainment in the App store, after Netflix.

Panel discussion on “The Metaverse Protagonist: Storytelling Techniques for AR Entertainment”.

Avataar and Qualcomm were present as the Titanium Sponsors. Gold Sponsors included several players in these emerging technologies: ArborXR, Holo-Light, HP, Meta, Meta Material, Metadome, Niantic, and SOAR. Avataar received the Best Enterprise Solution award.

Sponsors at AWE USA 2022.

More awards during AWE XR USA 2022:

• Best Developer Tool — ZapWorks: Universal AR SDKs.
• Best in Location-Based Entertainment — Battery Powered VR Bumper Cars.
• Best Art or Film — UNITE Space Explorers: The ISS Experience — Ep. 3.
• Best Campaign — ACROSS TECHNOLOGY TO ART: A virtual journey to Arles with LG SIGNATURE.
• Best Headworn Device — Varjo Aero.
• Best Consumer App — Measure Your Space.
• Best Collaboration Tool — Hubs Cloud.
• Best Game or Toy — Tilt Five Holographic Gaming System.
• Best Creator & Authoring Tool — ZapWorks: The all-in-one WebAR Platform.
• Best Interaction Product — Galea.
• Best Indie Creator(s) — Audio Trip.
• Best Use of AI — Volu: a camera for the Metaverse.
• Best Societal Impact — Breonna’s Garden Virtual Reality Experience.

Looking Ahead

Technologies have been steadily moving forward with a lot of successful innovations along the way. The advancements are getting faster and faster. Many of the readers may not remember or even be aware that there was a mainframe era that was very popular as the main computation engine in the 1960s-1970s, with its well-known green-screen applications. Well, mainframes are still operating now managing several of the most complex banking or credit card systems in the world, for example.

Later in the early 1980s came client-server with a graphical user interface followed by the next wave of the web (early 1990s) and mobile-based applications (iOS and Android-based applications in 2008).

Web 2.0 to Web 3.0

Nowadays, the interactive web (web 2.0) has been part of our lives, accessible mostly from mobile smartphones. In one of the panel discussions in AWE, it was said that user access through mobile has been the majority, about 80% — the important insight when designing content for web and mobile.

Web 2.0 is based on 2D. The transition to web 3.0 and the future will be 3D powered by spatial computing. As technologies are getting ready, existing mostly-adopted mobile smartphones will transition gradually to wearables such as AR/VR smart glasses, smart wristbands, haptics, and various XR-enabled devices. This will change how people work, play, and collaborate in the near future. It is seamlessly integrating the physical and virtual world (hybrid world) in various industries, enabled by key supporting technologies such as AR, VR, MR, XR, AI, Blockchain, and NFT.

The transition from Web 2.0 (current Internet) to Web 3.0 (Immervise Experience, e.g. Metaverse).

The Challenge of Delivering 3D Content by Streaming

Most of us have been enjoying Netflix for watching the on-demand movie without being aware of the complex infrastructure that Netflix has built in order to provide convenient streaming services (2D content). On the other hand, streaming GigaBytes and TerraBytes bandwidth-hungry 3D content have been a real challenge for network infrastructure. Take Meta Oculus Quest 2 for example, some of the best high-quality 3D content (including Games) are best experienced locally by first downloading the content to the device and then watching or playing, instead of streaming the 3D content directly from the cloud to the VR device.

Key delivery bottlenecks (e.g. the huge bandwidth requirement) and what technological advancements are being made and some case examples of recent metaverse implementations.

Wireless Technologies Wifi6 and 5G have been the key technology enablers for the Web 3.0 interactive experience, especially with the need for high-speed content streaming from the Cloud to XR devices. Lifi technology provided by Signify (a Philips company) will also be an excellent alternative in the near future for a line of sight super high-speed light-based communication.

The technologies for supporting the deployment of interactive 3D content at scale still need to be explored, developed, and deployed. Starting by 2030 maybe?

Intellectual Property (IP)

Businesses that have been heavily investing in Intellectual Property e.g. patents may be in a better position to be successful in the later future if the patents are adopted and widely used in multiple products and implementations. For XR, investment in patents can potentially be in the hardware, enabling firmware or software, or even in business software applications for example.

VR180 vs VR360

One of the panel discussions in AWE also kind of debated whether it would be better to create more VR180 or VR360 content for the future, considering the potential adoption of the user. I would be more toward VR180 as human eyes are designed to view the environment at 180 degrees rather than 360 degrees. The content (e.g. for the cinematic experience, a film) with mostly VR180 (say 70–80%) and occasionally with VR360 (20–30%) may be the best combination.

Interoperability — OpenXR and WebXR

OpenXR defines the standard and API for all XR-based devices, so developers won’t have to customize their products for each new device in the future. WebXR does the same, by integrating devices into the browsers. Interoperability will be the key to the advancements of XR in the future to come.

Building The Real-world Applications

In the majority, the gaming & entertainment industry has been adopting XR-based applications, so far. Other industries like enterprise, retail, e-commerce, advertising, healthcare & wellness, training & education, and banking are catching up with various business implementable business use cases by leveraging the advancements in XR technologies.

With the need for tight integration with other advancements in technology such as 5G/6G, Artificial Intelligence (AI), Bigdata, Internet of Things (IoT), and Blockchain, the adoption of XR technologies is getting good traction among Telecommunication and Information Technology professionals. This is expected to speed up the availability of diverse resources to develop more applications in the near future.

So, are we ready to be part of the exciting future of the XR-enabled world? We can start by doing something now. Exploring what’s available for us to experience now, for instance.

Notes on AR, VR, MR, and XR

To align ourselves with the terms, let’s have a quick look at the definition for each of them.

Augmented Reality (AR) enables us to experience the virtual world overlaying the physical world. Typically, we use an app on a Smartphone, overlaying virtual objects on the physical world.

Virtual Reality (VR) enables us to experience the real virtual world through devices and a set of sensors that stimulate a part of or all of our senses — typically, we use a VR headset equipped with a set of sensors.

Mix Reality (MR) is a combination of VR and AR. Through MR, we can interact with the virtual world.

Extended Reality (XR) is the umbrella covering all of them: AR, VR, and MR.

References

• Alon Grinshpoon, 2021, “Build a Cloud-Connected AR/VR App in 15 Minutes or Less”, AWE USA 2021.
• Andi Sama, 2022a, “My First 2022 Business Travel in Covid-19 Pandemic — Is it worth it?.” — To be published in late June 2022.
• Andi Sama, 2022b, “Mari Mengenal NFT.”
• Andi Sama et al., 2022, “Mimpi menuju Open Metaverse — The Next Internet (R)Evolution?.”
• AWE USA, 2022, “AWE USA 2022: Where XR Business Thrives” June, Santa Clara, CA, USA.
• Envision, 2020, “Extended Reality (XR).”