- Allen Mammen Abraham
- September 2, 2025
Mapping Depth Vision to Vibration Motors for Smarter Obstacle Detection
Introduction
Navigating through unfamiliar or dynamic environments is challenging for visually impaired individuals due to the inherent limitations of traditional assistive tools like white canes and guide dogs. These tools are often constrained in their ability to detect obstacles at various heights or provide a real-time spatial context.
To address these gaps, we explored the idea of an obstacle avoidance aid – a wearable assistive concept that could integrate depth-sensing technology with haptic feedback systems. The vision is to translate spatial data into tactile sensations, enabling users to potentially perceive their surroundings in real time.
The Technology
Depth-Sensing Camera
For depth sensing, many technologies are available like stereo, time of flight and structured light sensors with their advantages and disadvantages. For this application, we can consider the Intel Realsense D435 stereo depth sensing camera as an example. Light emitting depth sensing cameras generally consume more power and hence stereo cameras are preferred. The d435 requires around 3.5W power via a 5V USB type C. This has an ideal range of 0.3m to 3m and an FOV of 87 x 58. The camera supports a minimum of 30fps. But since such high rates of data are not needed for aiding mobility and navigation of visually disabled, lower fps can be taken, like 5 fps. High rates of vibration data can overwhelm users.
Stereo depth cameras rely on ambient light and often struggle in low-light conditions, while infrared (IR) depth cameras perform well in the dark but can be disrupted by sunlight. The Intel RealSense D435 combines stereo vision with active IR projection, enabling reliable depth sensing in most indoor and moderately lit outdoor environments, though it may still face challenges in extreme lighting.
The device employs a depth-sensing camera, mounted on a belt or wearable platform. This camera captures depth data, which is processed into a grayscale grid:
- Black blocks indicate objects in close proximity
- Gray blocks represent mid-range objects
- White blocks signify distant or unobstructed zones
The greyscale image has a range of 256 pixels, with the 0 representing black (closest) and 256 representing white (farthest). This 256 can be split into 3 equal parts and the pixel can be classified based on which range it lies in.
This simplified representation ensures quick processing and effective spatial mapping. To provide intuitive navigation, this depth data is directly mapped to the haptic feedback system, allowing users to “feel” their surroundings through vibrations.
Spatial Data Processing
The core computational process involves:
- Grid Mapping: Depth data is converted into a 5×3 matrix, with each cell representing a distinct spatial region.
- Dynamic Updates: The system continuously refreshes the grid to account for environmental changes, ensuring real-time adaptability.
- Proximity Encoding: Depth data is classified into intensity levels for haptic feedback translation.
The data from the camera can be transferred to the microcontroller on the belt. Processing is managed on lightweight microcontrollers or single-board computers like the Raspberry Pi 4, which optimizes performance while maintaining portability. In the case of a secondary microcontroller needed for wireless data transfer, an Esp32 may also be used.
Haptic Feedback System
The haptic interface consists of 15 vibration coin motors, connected via GPIOs from the microcontroller. These are strategically placed on a wearable glove, with 3 motors on each division of a finger. Each motor corresponds to a cell in the 5×3 spatial grid generated after data processing and varies its intensity based on obstacle proximity:
- High intensity: Closest objects (black)
- Moderate intensity: Mid-range objects (gray)
- No vibration: Safe zones (white)
The GPIO signals can be sent to the motors through wires from the microcontroller present on the belt, or by sending the processed data to a microcontroller present on the glove via Wi-Fi or Bluetooth. The latency will depend on the preferred mode of communication. Though wires limit hand motion for the user, it can reduce the components present in the gloves and also reduce latency.
As the depth-sensing camera detects objects, the corresponding haptic motors activate in real time, translating spatial information into tactile feedback. If an object is lying on the ground, the bottommost motors on the gloves will correspondingly vibrate. This direct link between visual depth data and touch-based signals enables users to navigate safely without requiring visual cues.
This system enables a tactile representation of the surrounding environment, delivering 3D spatial awareness directly to the user’s hand.
Current Consumption of System Components
The system components draw the following current: approximately 700 mA for the Intel RealSense depth camera, up to 3000 mA for the Raspberry Pi 4B, and around 1200 mA for 15 coin vibration motors (assuming 80 mA each), bringing the total current requirement to roughly 4900 mA under peak load conditions.
Component | Current Draw (Approx.) |
Depth Camera (Intel RealSense D435) | 700 mA |
Microcontroller (Raspberry Pi 4B) | 3000 mA |
15 Coin Vibration Motors (80 mA each) | 1200 mA |
Total Peak Current | 4900 mA |
Advantages Over Current Assistive Devices
Most electronic aids currently available in the market are based on canes equipped with ultrasonic sensors. These devices typically provide haptic vibrations or audio alerts to notify the user of nearby obstacles. While they do offer a level of environmental awareness, the information conveyed is quite limited.
Unlike ultrasonic sensors, which measure the distance to a single point directly ahead, providing only one-dimensional depth data, the proposed depth camera-based system captures a full 3D image of the environment. This allows it to identify multiple obstacles, estimate distances, and determine rough 2D locations within the user’s field of view. Ultrasonic sensors can struggle with small, low-lying, or irregularly shaped objects and are limited by their narrow detection cone, especially when dealing with reflective or absorbent surfaces. In contrast, the depth camera offers a broader and more detailed view, overcoming these limitations and enabling better situational awareness and safety through enriched spatial perception.
The depth data is then translated into haptic feedback using 15 coin vibration motors. These motors are spatially mapped to the camera’s output, allowing users to feel where an obstacle is located, how far it is, and get a rough sense of its size. Compared to traditional smart canes that simply vibrate when an obstacle is detected, this system offers higher resolution feedback. Users can better differentiate between multiple objects and interpret the environment more intuitively, building a more accurate mental model of the surrounding—something that current ultrasonic-based systems struggle to provide.
Features and Innovations
- High-Resolution Spatial Mapping: The device functions with a resolution equivalent to 15 ultrasonic sensors, enabling precise detection of objects at varying distances and heights.
- Low-Latency Processing: The device provides real-time feedback with processing delays of less than 50 milliseconds, ensuring responsiveness.
- Customizable Design: The components, such as the depth-sensing camera and haptic motors, are modular, allowing customization for different user needs and cost optimization.
System Architecture
Hardware
- Camera: Depth-sensing unit for environmental mapping
- Microcontroller: Handles real-time data processing and motor control
- Haptic Motors: Provide tactile feedback across the hand
- Power Source: With a total current requirement of around 5000 mA, the selection of a suitable power source depends on the intended runtime
Software
- Depth Data Processing Algorithm: Converts raw depth data into a spatial grid
- Feedback Control: Maps grid intensity to motor vibration levels
- Dynamic Calibration: Adapts sensitivity based on user preferences or environmental conditions
Future Scope
The obstacle avoidance device has significant potential for future enhancements. Key improvements include:
- AI Integration: Machine learning can enable obstacle recognition, distinguishing moving objects (pedestrians, vehicles) and surfaces (stairs, ramps).
- Multimodal Feedback: Incorporating audio cues or bone conduction technology can enhance spatial awareness.
- Personalization: Adaptive learning models can tailor navigation assistance to user movement patterns.
- Connectivity: Bluetooth and Wi-Fi integration would allow real-time synchronization with smartphones for GPS-based guidance.
- Compact Design: Future versions could be lighter and more wearable, such as wristbands, gloves, or smart glasses.
- Smart City Integration: The device could communicate with IoT-enabled infrastructure like pedestrian signals and public transport systems.
- Resolution and Power: Integration of more motors can increase the resolution of the obstacle mapping but might also affect the user’s ability to differentiate and identify vibrations. An additional low-power mode can also be integrated to handle scenarios where a high density of obstacles is detected in front of the user. In such cases, continuous or overlapping vibrations from multiple motors might overwhelm the user and reduce the effectiveness of haptic feedback. The low-power mode can reduce the intensity or frequency of vibrations, or prioritize alerts based on proximity or direction, ensuring that feedback remains meaningful and non-intrusive while conserving power.
- Hand Motion Tracking & Configurable FOV: Depth sensing can be configured for either full 360° surroundings or a specific field of view (FOV). Additionally, integrating hand motion tracking with the depth image would allow the haptic motors to vibrate only when the hand is in the direction of an obstacle. Hand motion tracking would require extra sensors like accelerometers and gyroscopes which could be integrated in the microcontroller present in gloves. With gloves on both hands and overlapping sensing zones, users could move their hands around to actively “feel” their environment.
Beyond visually impaired assistance, this technology could enhance industrial safety in low-visibility environments and VR gaming through immersive haptic feedback.
Conclusion
The device represents a leap forward in assistive technology, combining depth sensing and tactile feedback to empower visually impaired individuals. Its ability to provide real-time 3D spatial awareness enhances user confidence in diverse environments. It promotes independence by helping the visually disabled to navigate unfamiliar indoor spaces, detect obstacles at varying heights, and avoid collisions with pedestrians or objects.
References
- Wearable Obstacle Avoidance Electronic Travel Aids for Blind and Visually Impaired Individuals :- https://ieeexplore.ieee.org/document/10148956
- An introductory guide by Intel RealSense explaining the fundamentals of depth sensing technology :- https://www.intelrealsense.com/beginners-guide-to-depth/
- Haptic Feedback Vibration Motors :- https://www.precisionmicrodrives.com/haptic-feedback-vibration-motors
- A wearable ultrasonic-based obstacle detector to aid visually impaired individuals in avoiding obstacles :- https://dl.acm.org/doi/fullHtml/10.1145/3529190.3529217
