Impact of Time Discrepancies on Machine Learning Performance for Multi-Wearable Human Activity Recognition

Abstract

Wearable-based human activity recognition (HAR) has become a relevant tool for identifying everyday activities in various domains like healthcare, sports, and human-computer interaction (HCI). The classification performance can be improved by using multiple complementary sensors, which require accurately matched time bases. Although previous studies on synchronization in HAR suggested that sub-second accuracy is advisable while sub-100 ms accuracy is unnecessary, the specific effect of time discrepancies on machine learning models remained unexplored. We address this with an empirical evaluation of the impact of time discrepancies in multi-wearable HAR. We apply a systematic approach using the example of multi-stage temporal convolutional networks (MS-TCN) for action segmentation, simulating the time discrepancies of time offset and clock skew via rational resampling. Our evaluation spanned 30,025 training and validation runs across different model configurations, totaling over one million core-hours of computation. Our results reveal that time offsets larger than 150 ms should be avoided in training datasets, and offsets beyond 300 ms can already significantly degrade the HAR performance for typical activities of daily living (ADLs). The findings highlight the need for adequate synchronization of training datasets. Our findings have implications for the design and deployment of multi-wearable HAR systems and may extend to other multi-sensor contexts.