Automated Graph-level Passes for TinyML Fault Tolerance

Abstract

Deploying Machine Learning (ML) applications on Microcontroller Unit (MCU)-type devices, known as TinyML, poses significant challenges due to constrained resources. Consequently, this restricts the integration of fault tolerance mechanisms. Many redundancy techniques consume substantial amounts of already limited resources. To address this, Algorithm-Based Fault Tolerance (ABFT) methods for ML workloads focus on resource-intensive neural network operators at the kernel-level. However, this abstraction level introduces challenges, as these operators are often encapsulated within vendor-specific proprietary libraries. This work introduces automated and universal graph-level Data Flow Graph (DFG) passes supporting common kernel-level ABFT methods, along with a sophisticated redundancy mechanism called Dual Module Redundancy Island (DMRland). These DFG passes are implemented in the Tensor Virtual Machine (TVM) compiler framework and evaluated on a CPU-only execution platform with the MLPerf™Tiny benchmark. Our experimental results demonstrate that the proposed approach achieves competitive fault resilience in comparison to kernel-level ABFT methods, resulting in a reduction by two orders of magnitude for misclassification with combined ABFT and DMRland methods. Further, the run-time overhead (RTO) remains, on average, 19% lower than for comparable kernel-based implementations.