Hand Tracking and Companion Controller prototypes by Designium
Designium is an award-winning technology and design company that merges technology and creativity to create new AR experiences. Selected in the first cohort of the Snapdragon Spaces™ Pathfinder program, Designium created four prototypes to experiment with various features.
The company has worked in AR and XR field since its early days and focuses on digital content applications that combine AR and VPS. As Kuan Ying Wu, Designium’s interactive engineer and designer, explains, developing demo prototypes begins with trying new technology samples to identify the features worth highlighting.
After shortlisting features, he creates quick sketches to generate three to five ideas. Sketches help identify one or two ideas most suitable for development and are then expanded into various scenarios. This approach also helps to think through the functionality and UX details of the experience.
Next up is testing basic functions to understand the target features and whether UX design can be implemented. The prototypes’s goal is to showcase the features, so the design is adjusted around the goal. In the final stage, assets and models are used to create mock-ups, combining functions and materials to make prototypes workable.
Using Snapdragon Spaces Unity SDK, Designium selected Hand Tracking and Companion Controller features. Four demo prototypes put features to test through various scenarios.
AR demos overview:
- Fruit Scale: presents optical measurement combined with a user experience. The prototype educates users about the size of the objects in AR space.
- Sushi Grab: advanced practice for showcasing the Hand Tracking feature.
- Virtual Studio: shows how to lay out functional elements in the space using both hands. The same idea could be applied to interior design, urban planning, and architecture.
- Hologram: shows the Companion Controller combined with the user’s habit of smartphone operation behavior. The feature provides added value when used to drive operational behaviour for AR glasses.
The Fruit Scale prototype shows the basic functionality of the Hand Tracking feature. Leaning into a familiar gesture that people use when showing the size of objects with hands, the experience recreates the same process with AR measuring overlay.
The Sushi Grab demo continues to explore the Hand Tracking functionality, with pinching and releasing gestures that interact with different objects in the environment.
The Virtual Studio presents an idea of interaction with different virtual objects like one can do in the Unity scene window. Using Hand Tracking and Unity MICH-L project, the experience shows how the camera, lighting, and particle system can move and rotate in real-time in AR space.
The Hologram prototype is built upon the basic functionality of the Companion Controller. The prototype explores a swipe behavior (common for smartphones) to control the hologram and make it show, hide and rotate. The standard input of the touchpad (XY axis) is utilized to provide extended control options. Users can swipe up and down to show or hide content and swipe left and right to rotate it.
“Snapdragon Spaces Unity SDK helps you quickly develop interesting and immersive AR experiences for smart glasses. Its Unity SDK is using AR Foundation and OpenXR as the underlying layer, which helps creator with experience in mobile AR development to get started. Although the platform is still evolving, each update has many surprising things, and the improvements are considerable. Many of the questions asked in the forums are resolved in the next update. The replies on the forum are very quick and the answers often help solve the problems immediately. ThinkReality A3 + Motorola Edge+ phone offer great AR experience hardware. Glasses have wide FOV, making them comfortable to wear and promising for creating experiences. We need such AR wearable devices to bring more different experiences to life in the future.”
– Kuan Ying Wu, Interactive Engineer/Designer, Designium
Snapdragon Spaces Pathfinder is a program of Qualcomm Technologies, Inc. and/or its subsidiaries.