AWS CRT Client for Java adds GraalVM Native Image support

We are happy to announce GraalVM Native Image support for the AWS CRT (Common Runtime) package for Java starting with release 0.31.1. The AWS Java SDK picks up the update since version 2.28.7. Using AWS CRT with GraalVM Native Image in a demo app improved request times and both operational overhead and archive size were reduced.

AWS Common Runtime (CRT) is a collection of open-source libraries written in C, while aws-crt-java is the java binding for CRT libraries. With JNI (Java Native Interface), the CRT C-libraries are leveraged within Java.

The AWS CRT HTTP Client can be used by AWS service clients to invoke AWS APIs. Customers can use it as an alternative to the Apache-, Netty- or UrlConnection-based clients. Prior to this release, customers needed to provide manual configuration settings to run the AWS CRT HTTP client in a GraalVM Native Image project. With this release and GraalVM Native Image Support in the AWS SDK for Java 2.x, you can now compile the AWS SDK ahead of time into a GraalVM Native Image with the AWS CRT HTTP client! (See HTTP client recommendations to choose the best HTTP client for your workload.)

The following diagrams show the performance improvement for the demo application which uses the AWS CRT HTTP client on AWS Lambda. Both scenarios write data to an Amazon DynamoDB table. The first writes data from a cold start, and the second writes data from a warm start. We compared the GraalVM Native Image (bundled as an AWS Lambda custom runtime) with the AWS Lambda Java 21 managed runtime.

Cold start request processing time using the GraalVM Native Image experienced a 4X reduction for 90% of the requests. Warm start requests took 18-25% less time to process using the GraalVM Native Image.

Motivation

GraalVM compiles your application ahead of time, so dynamic features such as reflection or JNI method calls must be known at build time. The required reachability metadata configuration can be provided by JSON based configuration files placed under resources/META-INF/native-image. Builders can provide this data by creating their own configuration files, using community provided configuration files from the reachability metadata repository, or relying on library owners to embed the files into their build artifacts.

For the AWS CRT package, we decided to reduce the operational effort for builders by embedding the configuration files within the library.

The following section explains the internal process to add GraalVM Native Image support to the AWS CRT package. To deploy an example application with the new GraalVM Native Image support, see the demo application at the end of this article.

Process for generating JNI configuration files

To add GraalVM Native Image support to the AWS CRT HTTP client, we started by generating the JNI configuration files with the native image tracing agent. The agent tries to track the usage of dynamic features during runtime by attaching itself to a standard JVM. Initially, the agent ran as part of the JUnit test lifecycle via the Maven Surefire Plugin. This step can be added into a Maven profile to enable it on-demand:

<profile>
    <id>generate-graalvm-files</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.5.0</version>
                <configuration>
                    <argLine>-agentlib:native-image-agent=config-merge-dir=graalvm-configuration</argLine>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

The preceding code automatically generate the files with the needed reachability metadata configuration into a predefined folder. The generated configuration might contain classes that are not relevant for the final running artifact (for example, test-related classes).

We further fine-tuned the configuration and removed classes that were not relevant. Finally, we moved the files to the resources/META-INF/native-image folder, which is the default folder where GraalVM looks for configuration files. With this approach, the configuration files are automatically available when the library is used to build a native image.

As a further refinement, we ran our JUnit Test-Suite as a GraalVM native image to verify that the configuration files are covering the most important library paths. The following profile settings configure the test process with GraalVM native build tools:

<profile>
    <id>graalvm-native</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.graalvm.buildtools</groupId>
                <artifactId>native-maven-plugin</artifactId>
                <version>0.10.3</version> <!-- or newer version -->
                <extensions>true</extensions>
                <executions>
                    <execution>
                        <id>test-native</id>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <phase>test</phase>
                    </execution>
                </executions>
                <configuration>
                    <agent>
                        <enabled>true</enabled>
                    </agent>
                    <imageName>aws-crt-java</imageName>
                    <buildArgs>
                        <buildArg>--no-fallback</buildArg>
                        <buildArg>--verbose</buildArg>
                    </buildArgs>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

After verifying the initial configuration succeeds, we modified the build pipelines per platform (macOS, Linux x86-64, ARM64 etc.) to run the JUnit tests as a native image. With this change, we can verify that the configuration files are still valid. When new features are added, the tests will provide feedback when additional GraalVM configuration is necessary.

JNI shared libraries

For JVM-based applications, CRT bundles the pre-built JNI native libraries for all the different platforms (Windows, Linux, macOs) and architectures (ARM64, x86) in the same JAR that is available via the Maven distribution.

The benefit with this approach is that Java developers do not need to compile C code, or any extra dependencies to use the JNI extension from CRT. When CRT is initialized, it looks for the JNI library to load from a common path. If CRT cannot find the library, it will extract the corresponding library from the JAR, based on the current platform and architecture it runs on.

Initially, we chose to include all JNI libraries via the resource configuration file for the GraalVM Native Image build. In that scenario, the process could use the same JVM loading process as described in the proceeding section. In this way, without any additional configuration, the Native Image build for GraalVM worked on all the supported platforms!

Although the images could run on all platforms, GraalVM Native Image builds are platform-dependent. The included shared libraries for additional platforms and architectures increased the size of the artifact. Artifact size is related to startup time and is especially important in certain scenarios, such as AWS Lambda.

To address the binary size issue, we added an extra step during the GraalVM Native Image build to extract the shared library at build time. The following code snippet outlines a GraalVM Feature implementation that extracts the JNI library to the same directory of the native image during the build process:

import org.graalvm.nativeimage.hosted.Feature;

import software.amazon.awssdk.crt.CRT;

public class GraalVMNativeFeature implements Feature {

    @Override
    public void afterImageWrite(AfterImageWriteAccess access) {
        new CRT();
        ExtractLib.extractLibrary(access.getImagePath().getParent().toString());
    }
}

The Feature is enabled by default via the native-image.properties:

--features=software.amazon.awssdk.crt.internal.GraalVMNativeFeature

With the configuration in the preceding section, the JNI library for CRT will be extracted during build process, so that the GraalVM Native Image can avoid packing any of the JNI library with it, and load the shared library directly from the working directory!

The following chart shows the binary size decrease for an example application from 142MB to 101MB. The zipped deployment artifact for a corresponding AWS Lambda function shrinks from 52MB down to 37MB!

Demo application

You can find an example repository with the new AWS CRT HTTP client deployed to AWS lambda in the serverless-graalvm-demo project.

To run the project please follow the following steps.

1. Clone the repository.

   git clone https://github.com/aws-samples/serverless-graalvm-demo.git

2. Deploy the application using the AWS CDK (Cloud Development Kit) to create the necessary infrastructure.

   cdk deploy --all

3. Build the GraalVM native binary image. The GraalVM native image will be built for the designated target environment. The example project uses an Amazon Linux 2023 container image and the GraalVM Native Image tooling to build a native binary. The build process takes a few minutes. After the image is generated, you should see output similar to the following:

   --------------------------------------------------------------------------------------
         24.8s (10.5% of total time) in 275 GCs | Peak RSS: 3.55GB | CPU load: 6.57
   --------------------------------------------------------------------------------------
   Produced artifacts:
    /asset-input/products/target/product-binary (executable)
   ======================================================================================
   Finished generating 'product-binary' in 3m 54s.
   

4. After the native image creation CDK will automatically deploy your application to AWS Lambda:

   ✅  GraalVMPerfTestStack
   
   ✨  Deployment time: 35.28s
   
   Outputs:
   GraalVMPerfTestStack.ApiUrl = https://xxxxxxxxx.eu-north-1.amazonaws.com
    
   ✅  GraalVMDashboard
   
   ✨  Deployment time: 18.95s
   
   Stack ARN:
   arn:aws:cloudformation:eu-north-1:xxxxxxxxxxx:stack/GraalVMDashboard/
   
   ✨  Total time: 312.73s

5. Invoke the image with a load test script. The project contains a load testing script that will invoke several API endpoints automatically.

   cd load-test
   ./run-load-test.sh

6. To cleanup the previously created resources:

   cdk destroy --all

Conclusion

In this blog post, you learned about the AWS CRT package and the newly launched GraalVM Native Image support. We also shared the process we took to add GraalVM Native Image support to the library. By using the GraalVM Native Image, you can reduce startup time and memory footprint for latency sensitive applications. For more information about GraalVM, refer to the official GraalVM Native Image documentation. To get started, see serverless-graalvm-demo in the AWS Samples GitHub repository.

For feedback or reporting issues, contact us through the aws-crt-java repository.

About the author: