publications
2025
-
Violet Yinuo Han, Jesse T. Gonzalez, Christina Yang, Zhiruo Wang, Scott E. Hudson, and Alexandra IonACM UIST’25, Busan, Korea, Sep 2025
Users constantly interact with physical, most often passive, objects. Consider if familiar objects instead proactively assisted users, e.g., a stapler moving across the table to help users organize documents, or a knife moving away to prevent injury as the user is inatten- tively about to lean against the countertop. In this paper, we build on the qualities of tangible interaction and focus on recognizing user needs in everyday tasks to enable ubiquitous yet unobtrusive tangible interaction. To achieve this, we introduce an architecture that leverages large language models (LLMs) to perceive users’ environment and activities, perform spatial-temporal reasoning, and generate object actions aligned with inferred user intentions and object properties. We demonstrate the system’s utility provid- ing proactive assistance with multiple objects and in various daily scenarios. To evaluate our system components, we compare our system-generated output for user goal estimation and object action recommendation with human-annotated baselines, with results indicating good agreement.
-
Violet Yinuo Han, Amber Yinglei Chen, Mason Zadan, Jesse T. Gonzalez, Dinesh K. Patel, Wendy Fangyu Yu, Carmel Majidi, and Alexandra IonACM UIST’25, Busan, Korea, Sep 2025
Dynamic physical interfaces are often dedicated devices designed to adapt their physical properties to user needs. In this paper, we present an actuation system that allows users to transform their existing objects into dynamic physical user interfaces. We design our actuation system to integrate as a self-contained locomotion layer into existing objects that are small-scale, i.e., hand-size rather than furniture-size. We envision that such objects can act as col- laborators: as a studio assistant in a painter’s palette, as tutors in a student’s ruler, or as caretakers for plants evading direct sunlight. The key idea is to decompose the actuation into (1) energy input and (2) steering to achieve a flat form factor. The energy input is provided by simple vibration. We implement steering through differential friction controlled by flat-foldable compliant structures that can be activated electrically. We study the mechanism and its performance, and show its application scenarios enabling dynamic interactions with objects.
-
Violet Yinuo Han, Tianyi Wang, Hyunsung Cho, Kashyap Todi, Ajoy Savio Fernandes, Andre Levi, Zheng Zhang, Tovi Grossman, Alexandra Ion, and Tanya JonkerACM IUI’25, Cagliari, Italy, Mar 2025
Multimodal context-aware interactions integrate multiple sensory inputs, such as gaze, gestures, speech, and environmental signals, to provide adaptive support across diverse user contexts. Building such systems is challenging due to the complexity of sensor fusion, real-time decision-making, and managing uncertainties from noisy inputs. To address these challenges, we propose a hybrid approach combining a dynamic Bayesian network (DBN) with a large language model (LLM). The DBN offers a probabilistic framework for modeling variables, relationships, and temporal dependencies, enabling robust, real-time inference of user intent, while the LLM incorporates world knowledge for contextual reasoning. We demonstrate our approach with a tri-level DBN implementation for tangible interactions, integrating gaze and hand actions to infer user intent in real time. A user evaluation with 10 participants in an everyday office scenario showed that our system can accurately and efficiently infer user intentions, achieving 0.83 per frame accuracy, even in complex environments. These results validate the effectiveness of the DBN+LLM framework for multimodal context-aware interactions.
2024
-
Zhitong Cui, Shuhong Wang, Violet Yinuo Han, Tucker Rae-Grant, Willa Yunqi Yang, Alan Zhu, Scott E. Hudson, and Alexandra IonACM CHI’24, Honolulu, HI, May 2024
We propose augmenting initially passive structures built from simple repeated cells, with novel active units to enable dynamic, shape-changing, and robotic applications. Inspired by metamaterials that can employ mechanisms, we build a framework that allows users to configure cells of this passive structure to allow it to perform complex tasks. A key benefit is that our structures can be repeatedly (re)configured by users inserting our configuration units to turn the passive material into, e.g., locomotion robots, integrated motion platforms, or interactive interfaces, as we demonstrate in this paper.
2023
-
Best Paper AwardViolet Yinuo Han, Hyunsung Cho, Kiyosu Maeda, Alexandra Ion, and David LindlbauerACM ISS’23, Pittsburgh, PA, Nov 2023
Mixed Reality (MR) systems display content freely in space, and present nearly arbitrary amounts of information, enabling ubiquitous access to digital information. This approach, however, introduces clutter and distraction if too much virtual content is shown. We present BlendMR, an optimization-based MR system that blends virtual content onto the physical objects in users’ environments for ambient information display. Our approach takes existing 2D applications and meshes of physical objects as input. It analyses the geometry of the physical objects and identifies regions that are suitable hosts for virtual elements. Using a novel integer programming formulation, our approach then optimally maps selected contents of the 2D applications onto the object, optimizing for factors such as importance and hierarchy of information, viewing angle, and geometric distortion. We evaluate BlendMR by comparing it to a 2D window baseline. Study results show that BlendMR decreases clutter and distraction, and is preferred by users. We demonstrate the applicability of BlendMR in a series of results and usage scenarios.
-
Violet Yinuo Han, Abena Boadi-Agyemang, Yuyu Lin, David Lindlbauer, and Alexandra IonACM UIST’23, San Francisco, CA, Nov 2023
Haptic feedback is important for immersive, assistive, or multimodal interfaces, but engineering devices that generalize across applications is notoriously difficult. To address the issue of versatility, we propose Parametric Haptics, geometry-based tactile feedback devices that are customizable to render a variety of tactile sensations. To achieve this, we integrate the actuation mechanism with the tactor geometry into passive 3D printable patches, which are then connected to a generic wearable actuation interface consisting of micro gear motors. The key benefit of our approach is that the 3D-printed patches are modular, can consist of varying numbers and shapes of tactors, and that the tactors can be grouped and moved by our actuation geometry over large areas of the skin. The patches are soft, thin, conformable, and easy to customize to different use cases, thus potentially enabling a large design space of diverse tactile sensations. In our user study, we investigate the mapping between geometry parameters of our haptic patches and users’ tactile perceptions. Results indicate a good agreement between our parameters and the reported sensations, showing initial evidence that our haptic patches can produce a wide range of sensations for diverse use scenarios. We demonstrate the utility of our approach with wearable prototypes in immersive Virtual Reality (VR) scenarios, embedded into wearable objects such as glasses, and as wearable navigation and notification interfaces. We support designing such patches with a design tool in Rhino.
-
Hugo Souza Oliveira, Niloofar Saeedzadeh Khaanghah, Violet Yinuo Han, Alejandro Carrasco-Pena, Alexandra Ion, Michael Haller, Giuseppe Cantarella, and Niko MünzenriederIEEE Sensors Letters, Nov 2023
Flexible sensors and electronics have gained much attention in recent years. They are especially interesting due to their abilities to conform to static and dynamic surfaces while keeping their functionality. These characteristics make them relevant for a wide range of applications, from health care and fitness monitoring to soft robotics. In this work, we go beyond simple mechanical flexibility and present a lightweight and permeable flexible sensor utilizing melamine foam as a substrate. The foam is coated with metallic copper (Cu) and semiconductive Indium-Gallium-Zinc-Oxide (InGaZnO) to form a thermistor-type temperature sensor. The sensor showed a very stable response when cycling the temperature between 25 °C and 51 °C, exhibiting a maximum sensitivity of −01.6%∘C−1 , a permeability of 366.6gm−2h−1 at 24 °C, and a maximum resistance variation of −2.9%RH−1 when varying the relative humidity from 40% to 70%. The device also remained fully functional even after being bent to a radius of 5 mm.
