Optimize Container Images

Fast startup times are crucial for responding promptly to disruptions and periods of high demand, while also enhancing resource efficiency. In this lab, we will begin by discussing a typical implementation that relies on a base image and a full Java Runtime Environment (JRE). To improve our startup times, we will utilize jib to create a custom JRE using jdeps and jlink. As an extra measure, we will employ GraalVM native images to further reduce application startup time and memory usage.

Additionally, we will not focus on optimizing the code at the individual line level. These optimizations are generally specific to a particular workload, and conducting profiling is often necessary to identify costly code sections. In this regard, AWS offers a tool called Amazon CodeGuru Profiler that allows for application profiling. By collecting runtime performance data from your live applications, CodeGuru Profiler generates recommendations to enhance your application's performance. With the assistance of machine learning algorithms, it can identify the most resource-intensive lines of code and provide suggestions to improve efficiency and eliminate CPU bottlenecks.

Another important topic is memory management for Java applications in containers. Since Java 10, it has become much easier to manage the memory of a Java application in containers in a meaningful way. Previously the JVM was not aware of the memory and CPU allocated to the container. Fortunately, the fix has been back-ported to Java 8 (version 8u191). Now the JVM calculates its memory based on the memory for the container and not based on the memory for the underlying host. The best way to identify how much memory is necessary is through load testing in a pre-production environment, such as a staging environment. You can collect these metrics with a service such as CloudWatch Container Insights. Or, do so by using Amazon Managed Service for Prometheus together with Amazon Managed Grafana.

Note that OOM errors are likely to occur during these tests. In order to analyze these errors with tools, such as Eclipse MAT (https://projects.eclipse.org/projects/tools.mat), it is necessary to generate a heap dump. This can be implemented automatically using java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/to/dump, for example. Of course, a host file system must be included in the container so that the heap dump can be analyzed later by other developers. With AWS Fargate, Amazon Elastic File System (EFS) is ideal for this use case. EFS automatically grows and shrinks as you add and remove files with no need for management or provisioning.