Neural Caching: Improving Longevity of Smart IoT Devices running Artificial Neural Networks

Abstract

Resource constraints and hardware aging make long-term neural in- ference on embedded and IoT devices increasingly challenging. We present Neural Caching, a lightweight inference mechanism that ex- ploits the piecewise-linear structure of Artificial Neural Networks (ANNs) with piecewise linear activation functions to accelerate evaluation without retraining or model compression. Our approach, Neural Caching, associates each locally linear region of an ANN with a cached affine mapping and reuses it for subsequent inputs falling within the same or nearby activation region. This enables full predictions to be computed via a single matrix multiplication instead of potential multiple matrix multiplications. Experiments on four human-activity recognition datasets demonstrate up to an order-of-magnitude reduction in inference time while preserving classification performance metrics within ±0.1 % of baseline ac- curacy. By lowering computational load and power draw, neural caching extends device lifetime and supports sustainable, long-term deployment of machine-learning models in real-world IoT environ- ments.