Accelerating the future of perception features research

The Spatial Meshing feature enriches the way users interact with the 3D environment and provides better augmented reality experiences. As example, Spatial Meshing allows users to place virtual board games on top of a table and to drive virtual avatars to navigate through the environment. One of the vital ingredients for a good user experience when using Spatial Meshing is accurate and efficient 3D reconstruction. In this post, we briefly cover two of our recent public contributions to improve depth perception and 3D reconstructions, accepted for presentation in the International Conference on Computer Vision (ICCV).

September 29, 2023

Depth-Guided Neural 3D Scene Reconstruction

Without depth sensors, existing volumetric neural scene reconstruction methods (Atlasⁱ, NeuralReconⁱⁱ, etc.) suffer from depth ambiguity when back projecting 2D features to 3D space. In this work, we propose to utilize depth priors, obtained from efficient monocular depth estimation, to guide the feature back-projection process. Figure 1 illustrates how depth guidance could reduce erroneous features and improve separation of objects in the reconstruction.

Figure 1. Feature back-projection without (Average-based) and with depth guidance.

Another common pitfall of volumetric-based scene reconstruction methods is the use of average for multi-view feature fusion. The average operation discards the cross-view consensus information which is critical to distinguish voxels on and off surfaces. We propose two alternative fusion mechanisms: the variance-based (var) fusion and the cross-attention-based (c-att) fusion. The non-learnable variance operator is shown to be as efficient as the average operator and delivers significantly better reconstruction. The learnable cross-attention module further improves the reconstructed geometry while being more efficient than the existing self-attention-based fusion modules.

Figure 2. The accuracy-efficiency tradeoff for 3D reconstruction. Closed circles: offline methods; Open circles and stars: online methods.

Figure 2 compares the two variants of our method (DG-Recon) against the SOTA 3D reconstruction methods, including VoRTXⁱⁱⁱ, SimpleRecon^iv , TransformerFusion^v and 3DVNet^vi, in terms of the reconstruction F-score and frames per second. Our method achieves the best performance-efficiency trade-off among online methods and reaches F-score close to the SOTA offline method, VoRTX. The video below demonstrates a live reconstruction session with our method. The geometry inside the office room is incrementally updated and refined every 9 keyframes (approximately every 1.5 seconds).

Figure 3. Live reconstruction with DG-Recon.

For more technical details on this work, please refer to our ICCV23 paper: Jihong Ju, Ching Wei Tseng, Oleksandr Bailo, Georgi Dikov, and Mohsen Ghafoorian, “DG-Recon: Depth-Guided Neural 3D Scene Reconstruction”, International Conference on Computer Vision 2023.

Improved Self-supervised Depth Estimation on Reflective Surfaces

Requiring per-pixel ground-truth at large volume for getting a reliable supervised training is a laborious task that also depends on specialized hardware, e.g., LIDAR that limits the size and diversity of the data. Therefore, self-supervised training schemes have gained extensive attention, resulting in competitive self-supervised methods that mainly rely on photometric image reconstruction loss^viii.
However, the discrepancy between the optimization objective (photometric image reconstruction) and the actual test-time use (dense depth prediction) can sometimes result in degenerate solutions that satisfy the training objective but are not desirable from the the point of view of actual usage.
One such example in this scenario is the behavior of self-supervised depth models on specular/reflective surfaces, where the models are observed to predict much larger depth values than the real surface distance on the specular reflections. Figure 4 depicts the reason why this happens, an example demonstrating such mis-predictions and how 3DDistillation (3DD), our proposed method published at ICCV23, resolves this issue.

Figure 4. An illustration of the reason for mis-prediction of self-supervised depth models on specular surfaces.

The main idea behind our proposed method is to bring further 3D prior in the training process through the 3D fusion of the predicted depth, projecting them back to the corresponding frames and using them as pseudo-labels to train the model; thus the name “3DDistillation”! As shown in quantitative and qualitative analysis (see Figures 5 and 6), this significantly rectifies the model’s behavior on the specular surfaces, while keeping the model still fully self-supervised with no requirement to sensory ground-truth. By experimenting with different model architectures (Monodepth2, HR-Depth^ix, MonoVIT^x ), we showed that our contributions are generic and agnostic to the underlying architecture.

Figure 5. Quantitative comparisons on Scannet Reflection Set with various backbone models.

Figure 6. Sample qualitative depth predictions and 3D reconstruction from 3D Distillation.

For more technical details on this work, please refer to our ICCV23 paper: Xuepeng Shi, Georgi Dikov, Gerhard Reitmayr, Tae-Kyun Kim, and Mohsen Ghafoorian, “3D Distillation: Improving Self-Supervised Monocular Depth Estimation on Reflective Surfaces”, International Conference on Computer Vision 2023.

Find out more about Spatial Meshing and its use cases.

The aforementioned contributions are just two examples of the many cutting-edge research projects we are conducting at XR Labs Europe. Our goal is to constantly push the boundaries of the perception feature’s quality that are offered through the Snapdragon Spaces XR Developer Platform. The team actively explores novel ideas and methodologies to further improve our methods and their accuracy and efficiency, and to enable a better experience for our XR technology end-users and developers.

---

ⁱ Murez, Zak, et al. “Atlas: End-to-end 3d scene reconstruction from posed images.” European Conference on Computer Vision 2020.
ⁱⁱSun, Jiaming, et al. “NeuralRecon: Real-time coherent 3D reconstruction from monocular video.” Conference on Computer Vision and Pattern Recognition. 2021.
ⁱⁱⁱStier, Noah, et al. “Vortx: Volumetric 3d reconstruction with transformers for voxelwise view selection and fusion.” International Conference on 3D Vision. 2021.
^ivSayed, Mohamed, et al. “SimpleRecon: 3D reconstruction without 3D convolutions.” European Conference on Computer Vision. 2022.
^vBozic, Aljaz, et al. “Transformerfusion: Monocular rgb scene reconstruction using transformers.” Advances in Neural Information Processing Systems. 2021
^viRich, Alexander, et al. “3dvnet: Multi-view depth prediction and volumetric refinement.” International Conference on 3D Vision. 2021.
^viiGodard, Clément et al., “Unsupervised monocular depth estimation with left-right consistency.” Conference on Computer Vision and Pattern Recognition. 2017.
^viiiGodard, Clément, et al. “Digging into self-supervised monocular depth estimation.” International Conference on Computer Vision. 2019.
^ixDai, Angela, et al. “ScanNet: Richly-annotated 3d reconstructions of indoor scenes.” Conference on Computer Vision and Pattern Recognition. 2017.
^xLyu, Xiaoyang, et al. “Hr-depth: High resolution self-supervised monocular depth estimation.” AAAI Conference on Artificial Intelligence. 2021.
^xiZhao, Chaoqiang, et al. “Monovit: Self-supervised monocular depth estimation with a vision transformer.” International Conference on 3D Vision. 2022.

Snapdragon branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

Meet some of the 80+ Companies in the Snapdragon Spaces Ecosystem

At AWE 2023 we announced that our Snapdragon Spaces Pathfinder Program has grown to over 80 member companies. This program helps qualifying developers succeed through early access to platform technology, project funding, co-marketing and promotion, and hardware development kits.

September 18, 2022

But it’s not just program members who make up the ecosystem behind our Snapdragon Spaces™ XR Developer Platform. Qualcomm Technologies is collaborating with an ecosystem of leading global operators and smartphone OEMs to bring headworn XR experiences to market across devices and regions around the world. In conjunction with 3D engine developers and content creators building immersive XR content, this ecosystem of companies is ushering in the next generation of XR headsets and experiences to the market.

Several of these companies have publicly announced how they’re collaborating with the platform and use Snapdragon Spaces to build apps for headworn XR. With so many projects spanning a wide range of verticals and use cases, you can’t help but become inspired. So, we decided to share the links to these announcements, organized by vertical, hoping they’ll inspire your next XR project.

Enterprise: Collaboration

• Arthur Digital : Immersive collaborative products, including virtual whiteboards, screen sharing, and real-time spatial communication for distributed teams. Use cases include learning and development, and workshops.

• Arvizio : Their AR Instructor product offers live see-what-I-see videos and interactive mark-ups in AR, enabling instructions to be overlaid in the user’s field of view.

• Hololight : Company’s Stream SDK offloads real-time rendering of compute-intensive images in AR/VR apps to powerful cloud infrastructure or local servers.

• Lenovo : Lenovo is working with several XR developers to make cutting-edge applications powered by Snapdragon Spaces technology available on the ThinkReality VRX. Use cases today include immersive training as well as collaboration in 3D environments.

• Pretia : The company is porting its MetaAssist mobile app to AR glasses.

• ScopeAR : Frontline workers use the company’s WorkLink AR app for employee training, product and equipment assembly, maintenance and repair, field and customer support, and more.

• Sphere : The company creates AR environment where remote teams feel like they are working in the same room through highly-realistic avatars, real-time spatial-audio language translations, and integrations with conventional conferencing

• Taqtile : Their Manifest application enables deskless workers to access AR-enabled instructions and remote experts to perform tasks more efficiently and safely. It supports an expanding number of heads-up displays (HUDs), so enterprise customers can select hardware platforms based on their requirements

Training

• DataMesh : Digital twin content creation and collaboration platform makes digital twins more accessible to frontline workers and creators while addressing workflow challenges in training, planning, and operations.

• DigiLens : Waveguide display technologies and headworn smartglasses for enterprise and industrial use cases, powered by the Snapdragon XR2 5G Platform .

• Pixo VR : Simplifies access to and management of XR content. The platform can host any XR content, works on all devices, and offers a vast library of off-the-shelf VR training content.

• Roundtable Learning : VR headsets for immersive enterprise training solutions.

• Uptale : Immersive learning ranging from standard operating procedures and security rules to quality control and managerial situations, and onboarding.

Productivity

• Nomtek : Their StickiesXR project is a Figma prototype that transforms traditional 2D workflows into XR experiences.

Education

• AWE 2023 award winner PhiBonacci : Provides hands-on training that is safe, efficient, and impactful for learning and working in medical, engineering, and industry 4.0 fields.

WebXR

• Wonderland: a development platform for VR, AR and 3D on the web, now built on Snapdragon Spaces to expand cooperation with platform and hardware providers.

• Wolvic : Offers a multi-device, open-source web browser for XR.

Gaming and Entertainment

• Mirroscape : MR technology and AR glasses to enable tabletop games in XR. Maps and miniatures are anchored to players’ tables just like real objects, providing them with unique and immersive views from virtually anywhere.

• Skonec Entertainment : Develops XR content that enables efficient and sophisticated spatial experiences, particularly in XR game development.

Health, Wellness, and Fitness

• Kittch: Provides interactive cooking instructions and other actions (e.g., set timers) using AR glasses, hand tracking, and eye tracking, thus freeing users’ hands to perform the necessary cooking steps.

• XR Health : Immersive VR, AR, or MR healthcare experiences, including training and collaboration in 3D, on devices powered by Snapdragon.

Others

• ArborXR: Provides XR management software to Qualcomm Technologies’ XR customers and OEMs. Organizations can manage device and app deployment and lock down the user experience with a kiosk mode and a customizable launcher.

• Echo3D : Cloud-based platform that streamlines the distribution of XR software and services.

• LAMINA1 : Blockchain optimized for the open metaverse, including novel approaches that leverage blockchain, NFTs, and smart contracts to bridge virtual and real-world environments (e.g., next-generation ticketing and loyalty programs).

• OPPO MR Glass : OPPO MR Glass Developer Edition will become the official Snapdragon Spaces hardware developer kit in China.

• TCL Ray Neo : The company will upgrade their RayNeo’s AR wearables with spatial awareness, object recognition and tracking, gesture recognition, and more, creating new possibilities in smart home automation, indoor navigation, and gaming.

Get Started

Initially released in June, 2022, and regularly updated (see our changelog) , Snapdragon Spaces provides developers with the versatility to build AR, MR, and VR apps across enterprise and consumer verticals. Developers, operators, and OEMs are free to monetize globally through existing Android-based app store infrastructure on smartphones. With proven technology and a cross-device open ecosystem, Snapdragon Spaces gives developers an XR toolkit to unlock pathways to consumer adoption and monetization.

You can get started on your headworn XR app today with the following resources:

• Free downloadable SDKs for Unity and Unreal
• Head over to the Snapdragon Spaces Documentation
• Get equipped with Hardware Development kits .
• Join our Developer Community Forum , Discord channel , and FAQs .

Introducing Spatial Mapping and Meshing

Our 0.11.1 SDK release introduced the Spatial Mapping and Meshing feature to the Snapdragon Spaces™ developer community. Read on to learn more about our engineering team approach, use cases and important factors to consider when getting started.

April 7, 2023

What is Spatial Mapping and Meshing?

Spatial Mapping and Meshing is a feature of Snapdragon Spaces SDK that provides users with an approximate 3D environment model. This feature is critical in helping smart glasses understand and reconstruct the geometry of the environment. Spatial Mapping and Meshing offers both a detailed 3D representation of the environment surrounding the user and a simplified 2D plane representation. Meshing is needed to calculate occlusion masks or running physics interactions between virtual objects with the real world. Snapdragon Spaces developers can access to every element of the mesh (vertex) and request updates to it. Planes are used whenever just a rough understanding of the environment is enough for the application.
For example, when deciding to place an object on top of a table, or a wall. Only the most important planes are returned by the API, since there is a trade-off between how many planes can be extracted and how fast they can be updated with new information from the environment.
In addition to meshes and planes, developers can use collision check to see whether a virtual ray intersects the real world. This is useful for warning the user when they approach a physical boundary – either for safety or for triggering an action from the app.

Why use Spatial Mapping in your AR experiences?

A big part of how well users perceive your app lies in understanding the environment and adapting your application to it. If you strive to achieve higher realism for your digital experiences, Spatial Mapping and Meshing is the right choice. Why? As humans, we can understand the depth of space using visual cues that are spread around us in the real world. Occlusion is one of the most apparent depth cues – it happens when parts or entire objects hide from view, covered by other objects. Light also provides plenty of other natural depth clues – like shadows and glare on objects. Spatial Mapping and Meshing allows to emulate human vision and provide vital information to your augmented reality app to intake depth cues from the real world.

“The Sphere team has been incredibly excited for the Snapdragon Space’s release with Spatial Mapping and Meshing” says Colin Yao, CTO at Sphere. “Sphere is an immersive collaboration solution that offers connected workforce use-cases, training optimization, remote expert assistance, and holographic build planning in one turnkey application. As you can imagine, spatial mesh is a fundamental component of the many ways we use location-based content. By allowing the XR system to understand and interpret the physical environment in which it operates, we can realistically anchor virtual objects and overlays in the real-world” adds Yao. “Lenovo’s ThinkReality A3 smart glasses are amongst the hardware we support, and Sphere’s users that operate on this device will especially benefit from Spatial Mapping and Meshing becoming available.”

How does Snapdragon Spaces engineering team approach Spatial Mapping and Meshing?

Our engineering team’s approach to Spatial Mapping and Meshing is based on two parallel components – frame depth perception and 3D depth fusion. Read on to learn more about each.

• 
• 

Frame Depth Perception We leverage the power of machine learning by training neural networks on a large and diverse set of training data. Additionally, our data set benefits from running efficiently on the powerful Snapdragon® neural processor chips. To scale the diversity and representability of our training datasets, we do not limit the training data to supervised samples (e.g. set of images with measured depth for every pixel). Instead, we leverage unlabelled samples and benefit from self-supervised training schemes. Our machine learning models are extensively optimized to be highly accurate and computationally efficient compared to sensor-based depth observations – all thanks to a hardware-aware model design and implementation.

3D Depth Integration The 3D reconstruction system provides a 3D structure of a scene with a volumetric representation. The volumetric representation divides the scene into a grid of cells (or cubes) of equal size. The cubes in the volumetric representation (we’ll call them samples in this article) stores the signed distance from the sample centre to the closest surface in the scene. This type of 3D structure representation is called the signed distance function (SDF). Free space is represented with positive values that increase with the distance from the nearest surface. Occupied space is represented as samples with a similar but negative value. The actual physical surfaces are represented in the zero-crossings of the sample distance.

The 3D reconstruction system generates the volumetric representation by fusing and integrating depth images into the volumetric reconstruction. For each depth image, the system also requires its pose (camera location and viewing orientation) at the acquisition timestamp, correlated with a global reference coordinate system. The 3D reconstruction system also extracts a 3D mesh representation of the surfaces in the volumetric reconstruction. This is done using a marching cubes algorithm that looks for the zero iso-surface of signed distance values in the grid samples. 3D reconstruction in its entirety is a rather complex and resource-intensive operation. Enabling the mesh of the environment in your XR app brings benefits and feature possibilities that often make it worth the computational cost.

Enabling realism with Spatial Mapping and Meshing

Virtual Object Occlusion Creating augmented reality experiences without occlusion has been common practice for a long time. Having an approximate mesh of the environment can be used to create virtual objects that are occluded by real objects. By leveraging the depth buffer by rendering the environment mesh into the buffer, together with all other virtual objects in your scene, you can effortlessly create occlusion effects. To achieve this effect, you need to create a transparent shader that still writes into the depth buffer.

Virtual lighting and shadows in the real world Similarly, the mesh of the real environment can be used to apply virtual lighting effects to the real world. When creating a virtual light source without a mesh of the environment, only virtual objects will be lit by this virtual light. This can cause some visual discrepancies. When there is a model of the real world that is lit, the discrepancy will be less visible. Shadows again behave quite similarly. Without a model of the environment, shadows coming from virtual objects will not throw a shadow on the real world. This can cause confusion about the objects’ depth and even give users the impression that objects are hovering. To achieve this effect, you need to create a transparent shader that receives lighting and shadows like an opaque shader.

Real lighting and shadows in the virtual world When combining virtual and real-world content it’s often very easy to spot where a virtual object starts and reality ends. If your goal is to have the most realistic augmentations possible you need to combine real and virtual lighting and shadows. You can do this by having light estimation emulate the real-world lighting conditions with a directional light in your scene. Combined with meshing, light estimation allows you to throw shadows from real world objects onto virtual objects.

Limitations Depending on your hardware, there will be certain limitations to the precision of the detected mesh. For example, Lenovo ThinkReality A3 uses a monocular inference-based approach, which can yield certain imprecisions. Transparent objects (such as glass or glossy surfaces) will lead to fragmentations or a hole in the mesh. The further away a real-world object is from the glasses, the less precise the generated mesh for it will be. Quite logical if you think of it – the RGB sensor loses information with distance. The mesh is available for a distance up to 5 meters from the user. Other methods of depth measurement (such as LIDAR) require specialized hardware. There is a cost-effect trade-off. While LIDAR could provide a higher precision for some use cases, RGB cameras are available in most devices already and are more affordable.

Next steps

Refer to Snapdragon Spaces documentation to learn more about Spatial Mapping and Meshing feature and use Unity sample and Unreal sample to leverage Spatial Mapping and Meshing feature.

Snapdragon branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.

Snapdragon Spaces at MIT Reality Hack

Joining the world’s biggest XR hackathon as one of the key sponsors is a big endeavour that comes with a big responsibility. The team brought a dedicated Snapdragon Spaces™ hack track, two hosted workshops and networking events, showcased demos – to help hackathon participants stretch the limits of their imagination.

FEBRUARY 27, 2023

With five days full of tech workshops, talks, discussions and collaborations, the MIT hackathon brought together thought leaders, brand mentors, students, XR creators and technology lovers who flew from all over the world to participate. The overall event had 450 attendees. There were 350+ hackers from 26 countries placed into 70+ teams; and the Snapdragon Spaces track had 10 teams. All skill levels were represented, and participants had the opportunity to explore the latest technology from Qualcomm Technologies and Snapdragon Spaces.

The Snapdragon Spaces team joined other event key sponsors at the opening day Inspiration Expo. Using the opportunity to connect with participants, we answered questions about the XR developer platform, encouraged hackers to join the hack, and showcased our “AR Expo” demo. The demo uses Snapdragon Spaces SDK features, letting hackers experience a variety of AR use cases – from gaming to media streaming and kitchen assistance apps. Later on, hackers had the opportunity to join Steve Lukas, Director of XR Product Management at Qualcomm Technologies, Inc., and Rogue Fong, Senior Engineer at Qualcomm Technologies, Inc., at the workshop dedicated to developing lightweight AR glasses experiences. This augmented reality-focused session educated participants on the features and capabilities of the Snapdragon Spaces XR Developer Platform.

Closer collaboration with hackers followed at the Snapdragon Spaces hack track, where the team hosted 50 participants in 10 teams in a dedicated area for hacking. The range of project ideas was truly impressive, spanning games, social good projects, and education. While some projects leaned into the functionality of Lenovo ThinkReality A3 and Motorola edge+ phone HW devkit , others integrated additional hardware, such as 3D displays and neurofeedback devices.

Our track rewarded the most innovative, compelling and impactful experience using Snapdragon Spaces, with Stone Soup team taking the first prize for using AR technology to help the unhoused conceptualize, customize and co-create their dream homes. The winning project executed their idea with a high level of polish, utilizing Snapdragon Spaces’ Plane Detection and Hit Detection perception technology, and integrated ESRI maps data into their project.

The runner-up, Skully resented a multipurpose education technology solution that brought up learning modules and 3D models when target images are located. The solution offers students a hands-on approach to learning and allows using Hand Tracking and Gaze Tracking to interact with lesson material. Skully is highly inclusive and offers those unfamiliar with interacting with AR learning modules in the form of more traditional learning formats (such as video).

Worthy of honorable mention is Benvision – the winner of “Working together for inclusion and equality” for enabling the visually impaired to experience the world through a combination of the Snapdragon Spaces 6DoF headset plus a bespoke real-time machine learning algorithm which turns landscapes into soundscapes. Also, Up in the Air team, who won in “Spatial Audio” category, introduced the app that turns a daily working space into a pleasing fantasy-style environment, using Hand Tracking for navigation and Spatial Anchors to anchor a workspace in user’s environment.

MIT Reality Hack 2023 left a long-lasting impression on all the participants. The opportunity to engage with key partners, industry leaders, and a broad XR developer community brought a new perspective into the use and future development of spatial computing. Our team left inspired by unforgettable collaboration, ideation and creativity of the hackers and will be working on incorporating their valuable product feedback in the next SDK releases.

Snapdragon Spaces Developer team in collaboration with Qualcomm Developer Network

Snapdragon Spaces is a product of Qualcomm Technologies, Inc. and/or its subsidiaries.

XR and 5G mmWave hackathon challenge in Tampere, Finland

For three days, the Finnish city of Tampere became the hub of the university and start-up developers, who took part in the first-ever Snapdragon Spaces hackathon. Participants had the chance to use state-of-art smart glasses powered by Snapdragon to design new use cases and leverage 5G mmWave connectivity and extended reality (XR).

OCTOBER, 24, 2022

Organized by Qualcomm Europe, the City of Tampere, Elisa Networks, Nokia, CGI, and Ultrahack, the hackathon presented a unique opportunity for local developers to compete and make the best out of the future “Europe’s fastest city” 5G mmW rich data infrastructure, including the iconic Nokia arena. Completed with ten independent teams of developers with various profiles, the event set out multiple goals. First, to enable developers to unlock capacities of fast mmW network. Secondly, to let participants apply XR as a new UI medium and use open data sets the city of Tampere has provided — all while using Snapdragon Spaces™ SDK and DevKit consisting of Lenovo ThinkReality A3 smart glasses and Motorola edge+ smartphone. With just two days and no prior experience working with Snapdragon Spaces SDK, all the teams could develop and demonstrate the potential of XR and 5G mmWave capabilities through innovative project ideas.

The hackathon started on Friday with onboarding, an opening session, and focused mentoring. Participants were briefed on the pitch and expected deliverables on the second day. Afterward, sessions interchanged with mentoring, checkpoints, and visiting the Nokia area for the 5G area for mmWave testing. Sunday, the event’s final day, was spent at the arena, where participants had a chance to do the final testing, get the mentor’s input and prepare for the pitch and presentation round. The jury consisted of Steve Lukas, Director of Product Management for Qualcomm Technologies, Inc., Pekka Sundstorm, Nokia Head of Strategy Execution Europe, Teppo Rantanen, Executive Director at the City of Tampere, and Taavi Mattila, Business Lead at Elisa.

To succeed at the final pitch and demo, participating teams were asked to present concepts that will use the Tampere city data set over fast connectivity including 5G mmW, integrate XR, stand out from already existing solutions and bring additional value to the target audience. The teams were also expected to think through the ease of deployment and utilize Snapdragon Spaces SDK and Hardware Developer Kit, apart from other criteria. Two out of three winning teams leveraged user-generated content in their use cases, while all of them utilized open datasets made available by the city of Tampere. Tampere’s 3D “twin-city” model was particularly popular. Teams thought through how they would utilize 5G mmW both for data retrieval and content upload of user-generated and other content at scale in a multi-user, city-wide deployment.

The concept of Tampere xRT revolved around city location-based art installation and platform. The team delivered an impressive combination of pitch, idea, and implementation. Whispers of Tampere presented the idea of sneaking audio snippets into the city park, allowing people to interact with the city and each other. Audio messages can be placed at any city location and then discovered and retrieved by others. The third winner, AR Digital Spaces presented an idea of dynamic AR showrooms with an immersive ad-based concept. The jury was impressed with an effective combination of various technologies in one app that promised to make ads more engaging and purposeful. Another two participating teams received honorary mentions: ARenify for their promise to transform sports experiences and Public Commuter for the concept aimed to make the city transport network more accessible and inclusive.

All the winning concepts involved integration with XR, cloud-hosted UGC, fast connectivity including 5G mmW for at-scale, city-wide use, as well as audio recording features on Android, Azure Kinect, and web-based portals for managing content. The first of the many Snapdragon Spaces hackathons to come, the event has provided an immense learning experience for all parties in co-designing for the upcoming Metaverse era. The City of Tampere is already exploring the possibility of leveraging some of the ideas and teams as part of many smart city activities. At the same time, the project participants plan to apply for European funding to help expand the 5G mmWave network outside the arena and explore bringing hackathon ideas to the citizens of Tampere.

Snapdragon Spaces Developer team in collaboration with Qualcomm Europe

Snapdragon Spaces is a product of Qualcomm Technologies, Inc. and/or its subsidiaries.