Zronek, M. (2020). Error Injection in specification-based configurations [Diploma Thesis, Technische Universität Wien]. reposiTUm. https://doi.org/10.34726/hss.2020.59100
Specifications in the context of configurations are ways to describe and constrain configuration values. These specifications can be used to limit the possible errors users and admins can do. Writing these specifications though requires resources such as time. It is also unclear how effective such specifications are and how many misconfigurations they can catch. The quality of the error message is also a point of concern because it should be an improvement to the applications native configuration validation. Our thesis will investigate how effortful it is to write specifications for LCDproc and Cassandra, both medium sized applications with 190-300 configuration settings. We tracked the time needed for writing specifications for both applications and we developed the tool Lyrebird for automatic error injection. With Lyrebird we check how many misconfigurations these applications catch with and without a specification. The error messages are then manually analyzed for 5 key features in terms of quality. Our results show that the additional layer of a configuration specification varies in the detection rate of errors. The more complex an application is, the less likely a specification will catch difficult errors. The quality of the error messages though perform significantly better due to an uniform and standardized error message.