Deploying Privacy-Preserving Local LLMs: A Comparative Study Under Realistic Hardware Constraints

Abstract

As organizations increasingly seek to process sensitive information without relying on third-party Application Programming Interfaces, locally hosted large language models (LLMs) have emerged as essential, privacy-preserving alternatives to cloud-based solutions. However, deploying LLMs locally on consumer-grade hardware imposes realistic constraints, such as limited Video RAM, which necessitate aggressive quantization and preclude the immense scaling benefits available in cloud-based models. This paper presents a comparative deployment study of local LLMs under these realistic constraints. To investigate whether result quality and response time are primarily driven by raw parameter count, or if factors such as quantization stability and instruction-tuning play a more decisive role when models must run privately on-device, this paper assesses a diverse selection of open-source models. The models are evaluated on a novel, privacy-critical task: processing and summarizing data sequences from a video-based emotion and engagement detection system, thereby providing a realistic baseline for local human-AI systems.