Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring Applications

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Wearable Technology

Background:

• Visual sensors offer accurate health monitoring but raise privacy concerns.
• Silhouette generation from RGB video enhances privacy but hinders individual identification in multi-occupancy homes.
• Distinguishing individuals is crucial for tailored health data analysis in free-living environments.

Purpose of the Study:

• To develop a privacy-preserving method for individual identification in home monitoring systems.
• To overcome the limitations of silhouette-based monitoring in distinguishing subjects.
• To enable subject-tailored data analysis through sensor fusion.

Main Methods:

• A multimodal deep-learning framework was developed to fuse silhouette video and accelerometer data.
• The framework maps video clips and acceleration data into a latent space for comparison.
• Person Re-identification (ReID) was achieved by measuring Euclidean distance between fused data streams.
• A novel triplet loss function was proposed to improve performance and convergence speed.

Main Results:

• The proposed method successfully addresses Person ReID using silhouettes in a multi-sensor context.
• Trained on the SPHERE Calorie Dataset, the system achieved an average Area Under the ROC Curve of 76.3%.
• An assignment accuracy of 77.4% was recorded, demonstrating effective individual identification.
• The novel triplet loss demonstrated improved performance and faster convergence.

Conclusions:

• Sensor fusion of silhouette video and wearable accelerometers offers a viable solution for privacy-preserving individual identification in home monitoring.
• The multimodal deep-learning framework enables accurate, subject-tailored health data analysis in multi-occupancy settings.
• The developed approach enhances the capabilities of home monitoring systems by integrating advanced AI techniques.