Hardware technologies for securing containers

LWN.net needs you!

Without subscribers, LWN would simply not exist. Please consider signing up for a subscription and helping to keep LWN publishing.

There are plenty of security concerns with running containers and applications that have been containerized—some of those concerns can be reduced or eliminated using hardware techniques. Intel's Arjan van de Ven described some x86 technologies that can help with some of the security problems that containers face at a LinuxCon North America presentation. One of the technologies is brand new, having only been announced a few days before the talk.

Many people are downloading and running containers from the internet without any real checking on their provenance, which "should scare the hell out of you", Van de Ven said. That is a "sharp knife problem" that cannot be solved with hardware technologies, since it all comes down to trust. There are a number of trust issues with that, including whether a container truly comes from where it purports to originate with the binaries that are expected, whether it contains software that has vulnerabilities that have been discovered since it was created, and whether the contents are complying with the licenses that govern the code. Those are all of the same problems that users face when downloading a Linux distribution—the same kinds of solutions will need to applied to containers.

But if you look "beyond the sharp knife", there are security problems where hardware can help. One major concern is that the container is leaky somehow, such that the containerized application can escape its containment. An attacker may use that ability to directly attack the host operating system (OS) or they may attack another container running on the host. In addition, how does a container know that the OS it is running on has not been compromised? These are places where "hardware-assisted security" can help.

Intel's Kernel-Guard Technology (KGT) tries to protect the kernel against certain kinds of malware, Van de Ven said. It places a small monitor between the kernel and the hardware to protect certain kernel data structures or CPU registers from modification. The monitor is not a full hypervisor, but uses similar techniques to protect the system from certain kinds of attacks. Kernel code pages, interrupt descriptor table contents, and page table mappings could be protected using KGT, as could CPU control registers and model-specific registers (MSRs).

Containers, applications, and other components will be able to detect changes in the underlying system and its software using the attestation feature that the Intel Cloud Integrity Technology (CIT) provides. Attestation is a way to prove that the binaries for components like the firmware, bootloader, kernel, and, say, Docker daemon or rkt binary, have not changed. A chain of hashes is calculated for the elements and the Trusted Platform Module (TPM) is used to sign the hash in such a way that others can verify that those elements have not been changed.

The attestation can be extended to prove that a container is running in the right data center or in the right country. That may be important for countries that require their citizens' data to be stored domestically, for example.

It is a "picky and fragile" solution in some ways, since anything that gets changed will change the hash chain. So upgrades need to be handled carefully. In addition, it only proves the state of the software when it was started; if the binary gets changed later by way of a compromise, it won't be detected. There is also a performance cost associated with the feature, so it does not come for free, he said. Attestation is "not for the faint of heart", but can help solve some security problems for containers.

Clear Containers are another technology that can help secure "containers". It provides the isolation of virtualization with the performance of containers by actually running the container in a lightweight virtual machine. He didn't go into much detail about Clear Containers, as he gave another full talk on that subject at the conference. Support for Clear Containers has been added to the rkt container engine as a proof of concept. It works, but there are still plenty of "interesting problems" left to solve, he said.

The supervisor mode access protection (SMAP) and execution prevention (SMEP) features of some x86 processors are changing some of things that we learned in school about CPUs, Van de Ven said. Instead of the traditional ring model, where the most-privileged ring has access to the data in all rings, SMAP and SMEP make the rings almost completely disjoint. If an exploit tricks the kernel into accessing or running user-space code, the CPU will simply fault, stopping the attack in its tracks.

Of course, the kernel needs to access user-space data at times, which is where the overlap between the rings comes into play. The Linux kernel already has special methods to access user-space data; those can lift the SMAP protections for the duration of that access. Any other access will trigger the fault. It doesn't prevent all attacks using bad kernel pointers, but it does make it harder to exploit them. (Support for a feature similar to SMAP for ARM processors has been merged for the 4.3 kernel.)

The final feature he covered had only been announced two days earlier: Intel Software Guard Extensions (SGX). This new feature is "a little weird", Van de Ven said. It allows the system to define a special zone of memory (called an "enclave") that will be used to hold encrypted memory for both code and data. The enclave will also have some defined entry points. Only code that is running inside the enclave can see the unencrypted contents of the memory. Even the kernel cannot access the code and data inside the enclave from the outside.

The typical use case for SGX would be for secure cryptography; the key can be placed in the enclave and cannot be extracted from it. The entry points would provide services using the key, like signing. In addition, the CPU can attest that it is running from within the enclave to a remote server.

It is effectively a "black box with a call table". You may be able to trick the enclave into signing things that it shouldn't have signed, he said, but getting the key out is not possible. If there is a security hole in the code inside the enclave, though, all bets are off. In addition, debugging the code inside the enclave is difficult—you can't simply attach GDB.

The enclave is populated from a driver, Van de Ven said in answer to a question from the audience. Another attendee suggested the "Intel SGX for Dummies" site for more information on the feature.

He circled back around to KGT as he was winding down the talk. That feature will perhaps be the most generally useful for protecting against various kinds of attacks. It can protect all of the read-only memory in the kernel along with all of the MSRs and CPU configuration registers. Many of the data structures in the kernel can be made read-only and be protected using KGT. It can be configured with a set of rules that, for example, would allow only certain functions to change certain parts of memory. So KGT could enforce that only the user-space access methods in the kernel are allowed to change the SMEP and SMAP settings.

KGT is implemented as a mini-hypervisor that requires no kernel changes. The code is available (under the Apache 2.0 license) for those interested.

These hardware technologies are certainly not limited to protecting containers or containerized applications—they are more widely applicable. SMEP and SMAP have been around for a while, but Clear Containers, CIT, KGT, and definitely SGX are all relatively new, so Van de Ven's talk provided a nice quick overview of those ideas. It will be interesting to see how they got used in the future.

[I would like to thank the Linux Foundation for travel assistance to Seattle for LinuxCon North America.]