Robust and Energy Efficient Deep Learning Systems

Abstract

Due to their unprecedented capability to discover patterns in data, Deep Learning (DL) algorithms have emerged as a powerful and dominant set of techniques for solving challenging problems that fall under the domain of Artificial Intelligence (AI). The models trained using these algorithms, i.e., Deep Neural Networks (DNNs), are nowadays being used in almost every industry for various applications, including safety-critical applications, e.g., autonomous driving, healthcare, and security & surveillance. For safety-critical applications, reliability against hardware-induced faults (e.g., soft errors, device aging, and manufacturing defects) is one of the foremost concerns, as faults at critical locations in a system can significantly degrade its application-level accuracy. The high overheads of conventional redundancy-based fault-mitigation techniques (e.g., dual-/triple-modular redundancy, instruction duplication, and error- correcting codes) coupled with the compute-intensive nature of DNNs limit their applicability for DNN- based applications, especially embedded applications. Therefore, alternative approaches are required that can exploit the intrinsic characteristics of these networks to offer improved resilience at low overhead (in terms of energy, area, and performance). Moreover, the intrinsic resilience characteristics of DNNs can also be exploited for introducing carefully-crafted designer-induced approximations to further improve the energy efficiency of the systems and compensate for the overheads of fault-mitigation techniques. To enable highly robust and energy-efficient DL systems, this PhD work aims at exploiting the unique error-resilience characteristics of DNNs to mitigate the effects of hardware-induced reliability threats at low cost and to further improve the energy efficiency of DNN-based systems through judicious approximations (i.e., carefully-crafted designer-induced approximations in less-sensitive computations/neurons for trading quality for efficiency). To achieve this, this work explores opportunities at both the software and hardware levels. In particular, this work develops novel concepts for substantially reducing the frequency of critical faults by either modifying the system to have a biased fault distribution (biased towards non-critical faults) or by transforming critical faults into the ones that can be tolerated by the system due to the intrinsic resilience of DNNs. Moreover, this work also develops concepts for effectively exploiting the intrinsic resilience of DNNs to improve the energy efficiency by relaxing the accuracy bounds of intermediate computations through designer-induced approximations. In fact, this work shows that when prudently designed, approximations may be deployed without having any application-level accuracy loss, which is crucial for safety-critical systems where energy efficiency and reliability both are important design metrics. Key highlights of the novel contributions of this PhD thesis are: Low-cost fault and aging mitigation for DL systems: This thesis develops the following concepts and techniques for mitigating the effects of hardware-induced reliability threats at low cost by leveraging the intrinsic error-resilience characteristics of DNNs.1. Saliency-driven fault-aware mapping2. A framework for mitigating aging in the on-chip weight memory of DNN acceleratorsApproximations for Energy-Efficient DNN Implementations: To enable highly energy-efficient DNN implementation by leveraging the intrinsic error-resilience of DNNs, the following concepts and techniques are developed in this work.1. Statistical techniques for error estimation of approximate adders2. A cross-layer approximation methodology3. The concept of curable approximations for DNN hardware accelerators4. Non-uniform post-training quantizationIn summary, this thesis presents several techniques for improving the robustness of DNN inference systems against hardware-induced reliability threats at low overhead cost by exploiting the unique error- resilience characteristics of DNNs. The thesis also presents techniques for improving the energy efficiency of DNN inference through carefully-crafted approximations that have minimal impact on the DNN accuracy while offering significant efficiency gains. The techniques are not restricted to a single abstraction layer. Specifically, for improving the energy efficiency, a cross-layer methodology is proposed that systematically combines software-level and hardware-level techniques to achieve higher benefits.