SBanTEM: A Novel Methodology for Sparse Band Tensors as Soft-Error Mitigation in Sparse Convolutional Neural Networks

Abstract

Over the last two decades, Convolutional Neural Networks (CNNs) have become common in a wide variety of tasks, including safety-critical ones such as autonomous driving, leading to optimizations such as Sparse Convolutional Neural Networks (SparseCNNs). Scaling technologica nodes has led to an exponential increase in transient faults affecting the systems, generating critical soft errors. We introduce SBanTEM a novel methodology for employing sparse band tensors as soft-error mitigation in SparseCNNs. SBanTEM includes a novel mitigation technique, employing band tensors, as they do not require using indices for storing data. We employ progressive reduction of the bandwidth of the selected tensors, allowing the network to train in-between successive prunings, and compensat accuracy loss. Additionally, we implement a Genetic Algorithm (GA) to optimally select the tensors bandwidths in the network. We analyze the resilience of many state-of-the-art CNNs on multiple datasets, showin that resilience is much lower for SparseCNNs, and using SBanTEM makes them as resilient as standard CNNs. SBanTEM's code and result is available at github.com/Alexei95/SBanTEM to boost reproducibility and reusability of the implementation.