CoMECS: Framework for Comparing and Measuring the Energy Consumption of Software

Abstract

The current human-made climate change is one of the greatest challenges of the last, current and coming generations. Humans produce a huge amount of greenhouse gases, that warm up the atmosphere. The immense thirst for energy that we as a society have is one of the main drivers. Achieving the ambitious goals set in the agreements that most countries of the world signed, will necessarily mean reducing our global energy consumption. The Statistical Review of The World report from 2024 shows that the world only produces around 20% of its energy from renewable sources. The rest emits greenhouse gases during its production. Thus we have to transition to renewable energy to avoid these emissions. This is however not possible to do quickly enough, so besides transforming the energy production, we also have to reduce our energy consumption globally. This includes every single part of the economy, which of course means that IT also needs to evaluate and reduce its impact. Malmodin and Lundén estimate in their paper from 2018 that the energy consumption of the ICT and Entertainment & Media sector will fall overall in the future, but for data centers it will rise. The newest report from the International Energy Agency from 2025 estimates that the energy consumption of data centers will more than double by 2030, especially due to the rise in popularity of AI applications.The IT sector and specifically the part of it that is focused on server infrastructure therefore has to get ready to measure and reduce its energy consumption. Underlying a huge part of modern server infrastructure is open-source software, which is usually maintained by just a small number of volunteers. These people do not have the resources to do extensive testing of the energy consumption of their software. This problem is amplified by the lack of well tested and freely available tooling that can be used to measure the power usage of programs. Additionally the available tools are very different in their input, output and their usage.We therefore propose a framework and methodology which facilitate the measurement of energy consumption of software with a wide variety of available software-based power meters. The three goals that we define for this are Openness, Reproducibility and Standardized Output, which our framework CoMECS has to fulfill. We show how we built a wrapper structure based on Nix, which allows for reproducible experiments. We also present our approach to using SSH for controlling the test machine, on which the energy consumption is measured. These techniques allow us to define and execute reliable and reproducible tests against a desktop machine and server without changing any of the defined wrappers.We then use our framework to compare six software-based power meters against a hardware-based solution. Our results show that using the values from the pkg domain of RAPL sensor or the CodeCarbon tool most closely resembles the real measurements using a hardware-based power meter. This is however only true for CPU and main memory focused work loads. Another experiment we conducted was reproducing previous results of how much energy consumption overhead containerization using Docker causes. Lastly, we evaluated the power consumption of nginx over the course of 16 releases and found that it reduced its average energy consumption between the version 1.11 and 1.26 by around 3%.