Protecting cryptographic keys and functions from malware attacks

Modern commodity operating systems, running on commodity hardware, are frequently used to store cryptographic keys and/or to perform cryptographic functions such as digital signatures. The importance of their security can hardly be overestimated because of the following: Digital signatures can not only be used for binding agreements and authenticating Web sites, but are also used for code authentication, including authenticating software updates, such as the widely-used Microsoft Windows Automatic Update. Cryptographic keys are used to encrypt sensitive personal data stored on commodity operating systems.

While security of cryptographic primitives and protocols has been well-understood in abstract models, there is relatively little understanding and study of the security of cryptography on real commodity systems. Furthermore, while one could exploit special hardware to ensure security of cryptographic keys, it is even more difficult to protect cryptographic functions because an attacker can compromise a cryptographic function by compromising any of many different points in the invocation process, including libraries and the operating system. We examine the problem of protecting cryptographic keys and cryptographic functions on commodity hardware and operating systems, with a focus on combating attacks committed by software, primarily malware. Specifically, we make two significant technical contributions: (1) We demonstrate a technique for performing encryption without having the cryptographic key in memory, thereby alleviating RAM disclosure attacks against keys. (2) We create a system for protecting both cryptographic keys and digital signatures from being disclosed or abused (respectively) by malware, while allowing security properties of the signatures to be verified offline by remote parties. As such, this thesis moves a significant step towards bridging the gap between security properties of cryptosystems in abstract models and the needs of security assurance in real-life systems. Our results are also generally applicable to maintaining confidentiality and security of non-cryptographic secrets and functions.