Mitigating malware communication infrastructures and exploitation techniques

Abstract

Malware, short for malicious software, is at the root of many security threats on the Internet. Two techniques are essential for advanced malware to achieve its nefarious goals. First, modern malware is controlled remotely: it receives commands over the network that allowthe operator to performcoordinated activities, such as launching a spamcampaign. This thesis presentsmeans to make take-downs that aim at disrupting a malware's communication infrastructuremore effective. Customizable input sample selection for dynamic analysis frameworks allows human analysts to maximize the total value of information obtained from analysis. On a different note, multi-path exploration techniques can help reveal the fallback strategies of malware in case their command and control servers are unreachable. Second, in order to enter a system silently without a user's consent,malware exploits vulnerable software deployed on a system. Although already well-studied, memory corruption vulnerabilities remain to be among the most exploited vulnerabilities to date. This thesis presents a testing approach for binary programs that is aimed at detecting buffer overread vulnerabilities. By combining static and dynamic analysis approaches in a novel way, such vulnerabilities that have the potential to leak sensitive data and render protection mechanisms such as ASLR ineffective, can be revealed. In addition, this thesis introduces a novel protection mechanism against code injection and code reuse attacks for embedded systems. By taking advantage of certain features of RISC architectures, invariants are extracted from a programexecutable image and enforced during runtime, preventing code injection attacks and reducing the attack surface for code reuse attacks.