Deep learning-based body action classification and ergonomic assessment

Abstract

With the rise of Industry 4.0, manufacturing processes have undergone a significant transformation characterized by increased complexity, high efficiency standards and larger implementation of automation. This leads to an increased demand for greater integration of assistance systems to more effectively support human worker in this dynamic environment. Correctly implementing such systems requires prioritizing the contextualization of physical work above all else. However, not many studies address this aspect. This deficiency is particularly evident in the latest proposed automated risk assessment frameworks, where ergonomic scores are generated without clear indication of which part of specific work task correspond to each score. This study concentrates on implementing a deep learning video classification model that combines two distinct input modalities to detect human body actions in a working environment. Additionally, the proposed systAem incorporates computer vision-based ergonomic risk assessment to accurately identify body movements that pose high ergonomic risks to human workers.This thesis begins with an introduction to the context of future factories and highlights the importance of computer vision-based recognition systems. Next, theoretical fundamentals of closely related topics, including assistance systems, their design, human action recognition, ergonomic risk assessment, and machine learning principles are described. Following this, a detailed literature review of the latest deep learning approaches for action recognition is conducted, covering both single and multimodal methodologies.Next, a deep learning-based action recognition model utilizing two different input data sources is implemented. This is followed by the integration of the implemented deep learning model with a deep learning-based ergonomic risk assessment system. Finally, the proposed framework undergoes testing and evaluation through the simulation of typical manufacturing logistic tasks.The proposed approach has shown potential based on evaluation outcomes. The integrated framework successfully labeled different full-body classes and assigned corresponding ergonomic scores. By accurately identifying risky body poses, it becomes easier to adapt existing workspaces and deploy appropriate context-aware assistance systems.