Towards Software-Based Vendor-Independent Preemption for Hardware Accelerated Workloads

Abstract

The increasing use of acceleration hardware, such as DSPs, FPGAs and GPUs, has significantly boosted computational performance potential in various domains. However, in safety-critical systems, this hardware can pose risks, especially in mixed-criticality scenarios where non-critical and non-preemptible workloads can delay the execution of critical tasks. To address this challenge, we introduce a vendor-independent Hardware Abstraction Layer (HAL) framework based on Vulkan that enables preemptive execution handling of compute shaders for devices for which a Vulkan driver is available. This mechanism shows potential for timely execution of critical workloads, making it a valuable tool for embedded systems, low-power designs, and machine learning. Our prototype framework utilizes modified GLSL shader source code to support preemption, allowing shaders to exit before completing all work steps if a stop-flag is set by the host system. While Vulkan support is not restricted to GPUs we focus our work on these devices and validate our framework through benchmarks on an NVIDIA GeForce GTX 1080 Ti GPU, comparing performance across various shader types. Results show that our preemptive shaders, though slightly slower due frequent polling of the stop-flag, offer a promising solution for integrating acceleration hardware into safety-critical and real-time heterogeneous compute systems.