Docker for Java Developers - Setting Memory and CPU Limits for your Java applications

When deploying java applications in production it has become standard practice to adjust JVM flags to ensure optimal performance and stability.

When deploying Java applications in containers it is important to understand how your runtime will adjust or not adjust based on various container technologies.

Let’s take a look at a real world example to understand how a simple Java application reacts when we adjust container options that control memory and CPU allocation.

This example is applicable to Docker, Kubernetes, Mesos and other orchestrated container environments.

A simple executable

Let’s start off with a simple Java executable that logs to the console the processors detected and the max memory available to the runtime.

public class Docker {

  public static void main(String[] args) {

    Runtime runtime = Runtime.getRuntime();

	int processors = runtime.availableProcessors();
	long maxMemory = runtime.maxMemory();

	System.out.format("Number of processors: %d\n", processors);
	System.out.format("Max memory: %d bytes\n", maxMemory);
  }
}

FROM openjdk:8-jre
COPY /build/libs/java-and-docker-1.0.jar java-and-docker.jar

CMD ["java", "-XX:+UnlockExperimentalVMOptions", "-XX:+UseCGroupMemoryLimitForHeap", "-jar", "java-and-docker.jar"]

In this example, we are going to run JDK 8, which is the lowest container-aware JDK. For more information about the differences in container awareness capabilities between JDK versions, you can check out the previous article here.

Now we need to build the container.

docker build -t zsiegel:java-and-docker .

With the container built, let’s run this on the current machine and see what we get.

docker run zsiegel:java-and-docker
Number of processors: 4
Max memory: 466092032 bytes

You can compare this result to the CPU and memory settings in the Docker app preferences on the Mac or look at how it compares to your machine specs on Linux. The number of cores should match and the ram displayed should be slightly lower.

Memory Limits

docker run -it --memory=512m zsiegel:java-and-docker
Number of processors: 4
Max memory: 119537664 bytes

Now that we have adjusted the amount of memory available to the container itself the heap size is adjusted by the runtime. How did the runtime decide on this value of 119537664 bytes which equates to roughly 120 megabytes? If you dig into the JVM Tuning guide you will see the following.

"Unless the initial and maximum heap sizes are specified on the command line, they're calculated based on the amount of memory on the machine. The default maximum heap size is one-fourth of the physical memory while the initial heap size is 1/64th of physical memory. The maximum amount of space allocated to the young generation is one third of the total heap size."

The runtime by default will use 1/4th of the available memory. In our case this means 512/4 = 128 megabytes which is roughly the number we see.

Processor Limits

Let’s take a look at limiting the cpu that is available to the container and see what happens. As of JDK 8 Update 131, the runtime should be aware of the number of cpus available and will tune thread counts accordingly. In the JVM's case one cpu is equal to one core.

docker run -it --cpus=1 zsiegel:java-and-docker
Number of processors: 4
Max memory: 468189184 bytes

This is not what I expected the first time I ran it so let’s dig in further to understand what is really going on.

The Docker --cpus flag specifies the percentage of available CPU resources a container can use. Specifically it refers to the cpu-shares. It does not adjust the number of physical processors the container has access to, which is what the jdk8 runtime currently uses when setting the number of processors.

There is future work to correct this shortcoming in JDK 10. You can follow the progress and discussion here.

Let’s try using a different flag called --cpuset-cpus. This flag is used to limit the cores a container can use. On a 4 core machine we can specify 0-3. We can specify a single core or even multiple cores by comma separating the index of the cores.

docker run -it --cpuset-cpus=0 zsiegel:java-and-docker
Number of processors: 1
Max memory: 508887040 bytes

docker run -it --cpuset-cpus=0,1 zsiegel:java-and-docker
Number of processors: 2
Max memory: 508887040 bytes

This result is more in line with what we expected. We now have the runtime properly seeing 2 cores available instead of 4. When we do this the runtime will then tune the number of compiler and garbage collection threads accordingly.

It is important to note that many libraries and frameworks that rely on thread pools will also tune the number of threads based on this numbers. You would think that this would address our problems but there is a catch!

Container Orchestration

There is a major problem with the above example. The vast majority of container orchestration tools like Mesos and Kubernetes set the cpu-shares and not the cpuset-cpus. This means that until the work in JDK10 mentioned earlier is completed the runtime and frameworks that rely on runtime.availableProcessors() will be unable to tune their thread count properly. My hope is that this work will be backported to at least JDK9 if possible.

If you want more info on the flags available in Docker for adjusting resource limits you can check out the documentation here.