Gestural Immersion: Quadraphonic Sound and Visual Interaction
Gestural Immersion is an interactive audiovisual system that explores gesture-based control of spatial sound and generative visuals. Using touchless hand tracking, the system maps embodied movement to quadraphonic audio synthesis and real-time visual structures, transforming the body into an expressive instrument rather than a control interface.
Gestures such as height, speed, tilt, and grasp dynamically influence pitch, spatialization, texture, visual density, and camera motion. The work investigates how intuitive, full-body interaction can blur the boundaries between performer, system, and environment in immersive XR contexts.
This project is developed as a practice-based research system, examining embodiment, spatial perception, and participatory media through real-time audiovisual feedback.
Interaction Model
• Hand position → pitch modulation and quadraphonic spatial routing
• Hand velocity → granular density and visual noise intensity
• Grab / release → amplitude modulation and visual intensity
• Hand tilt → camera orientation and scene dynamics
• Hand velocity → granular density and visual noise intensity
• Grab / release → amplitude modulation and visual intensity
• Hand tilt → camera orientation and scene dynamics
Tech Stack
• Apple Vision Pro — touchless hand tracking
• Max/MSP — quadraphonic sound synthesis and spatialization
• TouchDesigner — generative visual systems
• OSC — real-time audiovisual communication
• Max/MSP — quadraphonic sound synthesis and spatialization
• TouchDesigner — generative visual systems
• OSC — real-time audiovisual communication
Hardware
• Apple Vision Pro
• Quadraphonic speaker array (4-channel spatial audio)
• Audio interface and amplification system
• Quadraphonic speaker array (4-channel spatial audio)
• Audio interface and amplification system
Manuscript under academic review at Leonardo (MIT Press).
Overall system architecture of the interactive audiovisual installation.
Interactive Sphere
An interactive, hand-controlled particle sphere that responds to real-time gesture input, using historically grounded color palettes derived from Van Gogh’s Starry Night, Bauhaus principles, and other modern art movements. Particle behaviors—such as cohesion, dispersion, orbital velocity, and spatial density—are mapped to hand gestures through computer-vision–based recognition.
This project investigates whether embodied, gesture-based interaction can provide more intuitive and expressive control over generative systems compared to traditional GUI elements like sliders and buttons. By combining creative coding, computational design, and art-historical references, the work explores how visual systems can become more responsive, performative, and human-centered.
Tech Stack
MediaPipe • Three.js • WebGL • Computer Vision
Reflection in Verse
AI-powered system that analyzes facial expressions to detect emotions like joy, tension, and vulnerability, then transforms them into unique, personalized poetry using computer vision and natural language processing.
This project explores emotion-as-input as an interaction paradigm, where physiological signals become the primary interface for computational creativity. Building on research in affective computing and computer vision, it investigates how micro-expressions can drive generative systems in preparation for more immersive spatial and AR/VR experiences. This is ongoing research into emotionally responsive interfaces and their applications in extended reality environments.
Tech Stack
MediaPipe • Claude AI • Facial Recognition • Gesture Control
Instructions: Click and drag with your mouse to interact with the object.
Object Interaction
An interactive 3D browser-based prototype exploring foundational object manipulation through mouse input. The project investigates core interaction principles such as input-to-motion mapping, spatial feedback, and real-time visual response within a computational environment.
Designed as an early study in interaction design, this work serves as a precursor to more advanced explorations in embodied, spatial, and XR-based interaction systems.
Tech Stack
Three.js · JavaScript · HTML/CSS · CodePen
Instructions: Click anywhere on the canvas to create points. Each click records your journey through digital space, with lines connecting your path.
Distance Meter
An interactive data visualization exploring spatial cognition and gesture in digital environments. Distance calculated through journey: the measurable space between intention and action in the digital realm. Each click maps the gap between thought and gesture, building a cumulative archive that reveals how we navigate virtual space versus physical reality.
The system visualizes accumulated movement as an architectural pattern, where line thickness corresponds to distance traveled between points, and circle size at midpoints represents journey magnitude. Older marks gradually fade, creating a temporal layer that distinguishes recent movement from historical paths. Connection counts at each node quantify spatial clustering behavior, while real-time distance tracking provides measurable feedback on navigational habits.
This project examines the behavioral patterns that emerge when spatial exploration is reduced to discrete points and connections—a fundamental questioning of how digital interfaces compress and translate physical movement into computational data. The resulting compositions serve as self-portraits of spatial thinking: hesitant versus bold, clustered versus distributed, systematic versus chaotic.
Tech Stack
p5.js · JavaScript · HTML/CSS · Canvas API
This project is an interactive, rule-based visual system where the viewer actively shapes the composition through time-based interventions.
Click to add vertical Buren stripes, drag to paint flow lines, and use keyboard controls to morph, regenerate, or save evolving system states.
Click to add vertical Buren stripes, drag to paint flow lines, and use keyboard controls to morph, regenerate, or save evolving system states.
Buren: Systematic Deconstruction
Buren: Systematic Deconstruction is an interactive, generative system that reinterprets Daniel Buren’s institutional critique through computational design. The work layers animated geometric patterns, flow-field–driven lines, and user-applied vertical stripes to explore the tension between organic expression and systematic constraint.
As users interact, the visual field evolves—revealing how structured systems shape, limit, and overwrite complexity over time.
The project examines how rule-based systems can embody art historical and critical discourse. User actions are preserved rather than erased, emphasizing accumulation, visibility, and control. While interaction offers agency, the system’s constraints remain fixed—reflecting the dynamics of institutional power in digital space.
Tech Stack
p5.js · WebGL · GLSL Shaders · Procedural Pattern Generation · Perlin Noise · Time-Based Animation