Distributed Encountered-Type Haptics in VR
Encountered-type haptics system coordinating distributed mobile robotic units for realistic tactile feedback in VR.
Overview & Research Motivation
Developed for Smart India Hackathon (SIH) 2024, this cyber-physical immersive system addresses the lack of physical resistance in virtual reality environments. Rather than wearable haptic gloves that cannot provide true encounter-type stopping forces, the system coordinates autonomous mobile haptic robots that physically position real surfaces exactly where the user's hand will touch a virtual object.
My contribution focused on VR interaction architecture, real-time spatial mapping, tracked-hand raycasting, haptic contact prediction, UDP-based robotic dispatch, and ArUco marker computer vision localization for low-latency visual-tactile synchronization.
The Core Systems Problem
- Traditional VR haptic gloves simulate vibration but cannot provide ungrounded kinesthetic resistance when a user pushes against a virtual wall or table.
My Specific Technical Contributions
- Built VR raycasting and hand-tracking collision prediction algorithms in Unity / Unreal Engine.
- Implemented low-latency UDP communication protocol coordinating virtual contact events with robotic mobile bases.
- Integrated ArUco marker tracking for sub-centimeter real-world robotic positioning and visual-tactile alignment.
System Architecture & Verification Pipeline
VR Spatial Tracking & Raycasting
Tracks user hand velocity vectors to predict intended contact coordinates 200ms ahead of physical touch.
UDP Dispatch Protocol
Transmits target coordinates to distributed omnidirectional mobile robots over high-speed local wireless.
Computer Vision Calibration
ArUco optical tracking continuously aligns the physical robot surface with the virtual object mesh.
Technical Stack & Tools
Connected Systems & Inquiries
Exploring how evolving cyber-physical state can remain independently verifiable under privacy and adversarial constraints.
ESP32-based multi-sensor condition monitoring, signal filtering, and anomaly detection for pumps and motors.