Current Projects

These projects reflect my current work on deployable sensing and machine learning systems, with an emphasis on on-device inference, multimodal perception, and robust evaluation in real-world environments.

Multi-Stage Thermal-Triggered VLM Framework for On-Device Eating Detection and Caloric Estimation

A resource-aware multimodal sensing system that introduces a thermal-triggered gating mechanism to selectively activate high-cost sensors such as RGB/depth cameras. The pipeline combines temperature-based filtering, connected-component analysis, and spatial constraints for event detection, followed by a NAS-optimized vision-language model and depth-based volumetric reconstruction for caloric estimation. It is designed for real-time, on-device deployment under strict power and compute budgets.

• Thermal-triggered sensing: Designed a low-cost event detection module using temperature thresholding, spatial clustering, and centroid constraints to gate RGB/depth sensing and reduce unnecessary sensor activation.
• Multimodal pipeline: Integrated thermal (MLX90640), RGB/depth, and IMU streams into a unified inference pipeline for eating detection and intake estimation.
• Model optimization: Applied neural architecture search (NAS) to design lightweight models optimized for on-device inference, balancing latency, accuracy, and energy consumption.
• Caloric estimation: Implemented depth-based volumetric reconstruction to estimate food portion size and combine portion estimates with classification outputs for kCal estimation.
• System deployment: Built an end-to-end embedded system supporting real-time inference, adaptive sensing, and efficient data logging under hardware constraints.

Wi-Fi Indoor Positioning via RSSI-FTM Fusion

A probabilistic localization framework that fuses RSSI fingerprinting and FTM ranging through Bayesian filtering, with explicit modeling of LOS/NLOS conditions. The system incorporates a graph neural network-based correction module to improve cross-site generalization and reduce dependence on dense site-specific calibration.

• Sensor fusion: Combined RSSI-based fingerprinting with FTM ranging using Bayesian filtering to improve localization robustness in multipath environments.
• Feature engineering: Designed signal preprocessing and feature extraction pipelines to stabilize RSSI variance and improve spatial consistency.
• GNN correction model: Developed a graph neural network-based refinement model to capture spatial dependencies and enhance cross-site generalization.
• System evaluation: Built automated data collection and benchmarking workflows to ensure reproducible evaluation across indoor environments.