A distributed compute fabric for edge intelligence

Abstract

Edge intelligence is a post-cloud computing paradigm, and a consequence of the past decade of developments in Artificial Intelligence (AI), Internet of Things (IoT), and human augmentation. At the intersection of these domains, new applications have emerged that require real-time access to sensor data from the environment, low-latency AI model inferencing, or access to data isolated in edge networks for training AI models, all while operating in highly dynamic and heterogeneous computing environments. These requirements have profound implications on the scale and design of supporting computing platforms that are clearly at odds with the centralized nature of cloud computing. Instead, edge intelligence necessitates a new operational layer that is designed for the characteristics of AI and edge computing systems. This layer weaves both cloud and federated edge resources together using appropriate platform abstractions to form a distributed compute fabric. The main goal of this thesis is to examine the associated challenges, and to provide evidence for the efficacy of the idea. To that end, we develop new system evaluation methodologies and two orthogonal systems: an elastic message-oriented middleware, and a serverless edge computing platform. From a static centralized deployment in the cloud, we bootstrap a network of brokers that diffuse to the edge based on resource availability, and the number of clients and their proximity to edge resources. The system continuously optimizes communication latencies by monitoring client–broker proximity, and reconfiguring connections as necessary. Our serverless platform builds on existing container orchestration systems. The core is a custom scheduler that can make tradeoffs between data and computation movement, and is aware of workload heterogeneity and device capabilities such as GPUs. Furthermore, we demonstrate a method to automatically fine-tune scheduler parameters and optimize high-level operational goals.