A Fully On-Device Smart Cane for Trip-Hazard Detection: Gyroscope-Gated Vision–Language Inference

Summary: We present a fully on-device smart cane that proactively detects and identifies floor-level trip hazards, then warns the user by voice.

Abstract

Falls from tripping over floor-level objects are a leading cause of injury among older adults, yet existing aids either detect falls after impact or sense obstacles by range and depth, without recognizing the specific hazards that cause them. We present a complete, fully on-device smart cane that proactively detects and identifies floor-level trip hazards and warns the user by voice.

On a Raspberry Pi 5, an inertial gate decides when to look, an on-device vision-language model (Qwen2-VL-2B on a Hailo-10H accelerator) describes the floor scene, and a deterministic parser turns that description into a warn-or-silent decision, all running locally on battery power with about 3.9 s from image capture to spoken alert.

The enabling design choice is to gate capture on the gyroscope stillness, which yields sharp frames despite walking-induced blur while staying within the device’s limited inference budget. On 195 walking sessions across varied rooms, lighting, and floors, a controlled comparison of five frame-selection strategies confirms that gyroscope gating gives the best precision and F1, with the gain in fewer false alarms rather than recall.

The system shows that proactive, language-grounded trip-hazard detection is feasible entirely on-device on a mobility aid.