Modeling Diagonal State Space Models as Electric Circuits for Analog Neural Network Inference

Abstract

Neural networks based on State Space Models (SSMs) have shown good performance on long sequence modeling tasks, such as raw audio classification. So far, their continuous-time parameter representation has not been used for analog neural network computing. We propose AnalogSSM, a diagonal and real-valued SSM architecture that can be converted into a purely analog electric circuit representation consisting of adder/subtraction, low-pass, and rectifier operational amplifier circuits. Targeting hotword detection based on the Google Speech Commands dataset, we evaluate three model configurations ranging from 0.15k – 1.3k parameters. Achieving, on average, over ten individual hotwords, an accuracy range of 84.5% – 90.8% with discrete models in PyTorch. The synthesized electric circuits are simulated and evaluated with LTspice. On average, we observe accuracy drops of 2.9pp with the continuous-time analog circuits only consisting of 70 – 238 operational amplifiers.