Executing GraalVM Tracing Agent with Spring Boot’s Docker Compose plugin in a Lambda function.

11 min readAug 13, 2024

Introduction.

In our previous tutorial, we built a Lambda function using Spring WebFlux. That Lambda function also used Spring Native to build a native Linux executable with the help of GraalVM. I showed you an additional configuration we must implement in the Native Maven plugin only for Lambda functions. I also told you that all our City’s Lambda projects have a parent POM file that extends from Spring Boot’s parent POM. So, let’s focus on this tutorial in our parent POM file, which configures the GraalVM tracing agent for all our projects.

In another previous tutorial, I wrote about Publishing POM files in Maven Central. Our parent POM is published there, and we start from this point. We will also use the new Spring Boot’s Docker Compose plugin to execute our City Data Lambda function in conjunction with the Tracing Agent to capture all function’s interactions, the same one that we’re migrating to Spring Webflux in this tutorial.

As usual in these tutorials, we will use Spring Native with GraalVM (to build a native Linux executable), Spring Cloud AWS, and LocalStack with Testcontainers for integration tests.

Tooling.

To complete this guide, you’ll need the following tools:

Git.
AWS CLI (version 2.17.+)
AWS SAM CLI (version 1.97.+)
OpenJDK (version 21.+). You can use SDKMAN to install it.
Apache Maven (version 3.9.+). You can use SDKMAN to install it.
Docker engine. You can use Docker Desktop or Colima.
Java IDE. You can use IntelliJ.

Note: You can download the project’s source code from my GitHub repository, which contains all the configurations I made in this tutorial.

GraalVM Tracing Agent.

Let’s write some Oracle official documentation about the Tracing Agent to put some context about it:

The tracing agent helps to deal with features as Reflection, Java Native Interface, Class Path Resources, and Dynamic Proxy in the GraalVM environment. It is applicable when the static analysis cannot automatically determine what to put into a native image and undetected usages of these dynamic features need to be provided to the generation process in the form of configuration files. The tracing agent observes the application behaviour and builds configuration files when running on the Java HotSpot VM, thus it can be enabled on the command line with the java command…
During execution, the agent interfaces with the JVM to intercept all calls that look up classes, methods, fields, resources or request proxy accesses. The agent then generates the jni-config.json, reflect-config.json, proxy-config.json and resource-config.json in the specified output directory. The generated files are stand-alone configuration files in JSON format which contain all intercepted dynamic accesses.

Oracle defines the goal of the tracing agent very well. However, as you saw in my tutorial “Storing EDA events in DynamoDB from a Native Lambda Function” from section 4, not all libraries are supported to be included in a native executable due to the exhaustive usage of Java dynamic operations, like in the case of Log4J. That’s why I have used Logback in our last tutorials for logging procedures.

Next, we will examine the other building blocks for native image generation in Spring Boot’s context.

Spring Boot’s parent POM.

If you visit the content of the Spring Boot parent POM (version 3.3.2), you will see a profile called <native> with the following content :

<profile>
  <id>native</id>
  <build>
    <pluginManagement>
      <plugins>
        ...
        <plugin>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-maven-plugin</artifactId>
          <configuration>
            <image>
              <builder>paketobuildpacks/builder-jammy-tiny:latest</builder>
              <env>
                <BP_NATIVE_IMAGE>true</BP_NATIVE_IMAGE>
              </env>
            </image>
          </configuration>
          <executions>
            <execution>
              <id>process-aot</id>
              <goals>
                <goal>process-aot</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
        <plugin>
          <groupId>org.graalvm.buildtools</groupId>
          <artifactId>native-maven-plugin</artifactId>
          <configuration>
            <classesDirectory>${project.build.outputDirectory}</classesDirectory>
            <requiredVersion>22.3</requiredVersion>
          </configuration>
          <executions>
            <execution>
              <id>add-reachability-metadata</id>
              <goals>
                <goal>add-reachability-metadata</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
</profile>

This profile configuration has defined some plugins that help us build the native image. If you go to the Spring Initializr site and generate a project using the “GraalVM Native Support” dependency, the pom file will include the following build configuration:

<build>
  <plugins>
    <plugin>
      <groupId>org.graalvm.buildtools</groupId>
      <artifactId>native-maven-plugin</artifactId>
    </plugin>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

So, your new Spring Boot project will use the <native-maven-plugin> plugin, also defined in the parent POM with some custom configuration, to build the native image.

To generate the native Linux executable, you must execute the following Maven command, indicating also the use of the <native> profile:

mvn native:compile -Pnative

Internally, Spring Boot performs an Ahead-of-Time (AOT) Processing when building the native image. This approach is seen in the <spring-boot-maven-plugin> plugin declared in the Spring Boot parent POM file.

With these configurations in place, let’s prepare our application to be built as a native image.

Using WebFlux in Lambda function.

In our previous tutorial, I demonstrated how to implement WebFlux in a Lambda function that updates data in a DynamoDB table. In this opportunity, let’s implement WebFlux in a Lambda function that reads data from a DynamoDB table.

NOTE: I will not explain all the details in this tutorial because it has another goal. You can visit my previous tutorial to learn more about this topic.

The initial function implementation has the following structure:

@Override
public CityDataResponse apply(Message<CityDataRequest> cityIdRequestMessage) {
    CityDataRequest cityDataRequest = cityIdRequestMessage.getPayload();
    BeanValidations.validateBean(cityDataRequest);
    
    HashMap<String, AttributeValue> keyToGet = new HashMap<>();
    keyToGet.put('id', AttributeValue.builder().s(cityDataRequest.cityId()).build());
    GetItemRequest request = GetItemRequest.builder()
        .key(keyToGet)
        .tableName('Cities')
        .build();
    
    CityDataResponse response;
    Map<String, AttributeValue> returnedItem = this.dynamoDbClient.getItem(request).item();
    try {
        response = this.cityMapper.toCityResponse(city, HttpStatus.OK.value(), null);
    } catch (DynamoDbException exception) {
        response = new CityDataResponse(null, null, null, HttpStatus.INTERNAL_SERVER_ERROR.value(), "Internal server error when trying to find City data.");
    }
    return response;
}

After applying all the steps shown in the previous tutorial to use WebFlux, the method implementation is as follows:

@Override
public Mono<CityDataResponse> apply(Message<byte[]> cityIdRequestMessage) {
    CityDataRequest cityDataRequest = OBJECT_MAPPER.readValue(cityIdRequestMessage.getPayload(), CityDataRequest.class);
    
    return Mono.just(cityDataRequest)
        .doOnNext(BeanValidations::validateBean)
        .map(this.citiesRepository::findCityById)
        .doOnNext(this::validateCityStatus)
        .map(city -> this.cityMapper.toCityResponse(city, HttpStatus.OK.value(), null))
        .onErrorResume(CityDataFunction::handleException);
}

After updating the integration tests following the new function’s method signature, the results are the follows:

Our changes are working as expected. So, let’s build the native image locally using Docker Compose and perform some tests before deploying our Lambda function to AWS.

Native Testing using Docker Compose.

Let’s deploy our native Lambda function locally using Docker Compose and perform some requests to see the results:

docker compose up --build

Docker will build our native image and deploy it in a container using the <public.ecr.aws/lambda/provided:al2023-arm64> image. So we can invoke our Lambda function as it was in AWS. On the other hand, the Docker cluster will deploy a LocalStack container with the required DynamoDB table for the Lambda function interaction.

The compilation and build process will take some time. However, you can see the build process in your terminal window as follows:

After that, you must see the following output indicating that our cluster is ready:

NOTE: I’m also deploying the native image in the Lambda service inside the Localstack container. So, you can perform some additional validations against the deployed function.

In another terminal window/tab, execute the following command to invoke our Lambda function, which has enabled the HTTP port to interact with the function directly, but this is for local testing only:

curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" \
  -d @functions/city-data-function/src/test/resources/requests/valid/lambda-valid-id-request.json

You won’t see a result because of an error in the Lambda function. If you return to the Docker Compose terminal window/tab, you will see an error like this:

Spring couldn’t instantiate a bean required by the DynamoDB client. You can also see a named configuration file that cannot be found.

The definition of dynamic features we saw in the first section of this article could generate these kinds of errors. So, it could be a good idea to implement the GraalVM Tracing Agent to help us identify these kinds of dynamic operations before safely generating the native image.

Tracing Agent in POM files.

Let’s create a new Maven profile in our POM file to run our Spring Boot applications with the tracing agent enabled:

<profile>
    <id>tracing-agent</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>process-classes</phase>
                        <goals>
                            <goal>exec</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <executable>java</executable>
                    <arguments>
                        <argument>
                            -agentlib:native-image-agent=config-output-dir=${project.build.directory}/native-image
                        </argument>
                        <argument>-cp</argument>
                        <classpath/>
                        <argument>${start-class}</argument>
                    </arguments>
                    <environmentVariables>
                        <SPRING_PROFILES_ACTIVE>local</SPRING_PROFILES_ACTIVE>
                    </environmentVariables>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

You must have installed GraalVM in your local environment for this to work. Notice that in the <configuration> block, I’m defining the use of the <java> executable to run our Spring Boot main class defined in the <start-class> variable taken from the parent POM file. Also, I’m passing some arguments to the JVM to enable the Tracing Agent to capture all application operations when running the main class.

The <process-class> phase is because, at this point, Maven has compiled all Java classes, so our Tracing Agent could navigate and identify the dynamics behaviors at runtime. You can use the <package> phase, but this is recommended when you have a JAR file already generated and want to create a native image from it.

So, we can execute the following command to run our Spring Boot application in conjunction with the Tracing Agent:

mvn clean process-classes                  \
  -f functions/city-data-function/pom.xml  \
  -P tracing-agent

You must see the following in your terminal:

As you can see, the Spring Boot application has started, so you can begin sending requests to our Lambda function.

But wait a minute! You must have deployed the LocalStack container with the DynamoDB service enabled to allow the function to interact with the <Cities> table, correct? However, I would like to improve this process by starting the Tracing Agent and the LocalStack container in the same process. Let’s do that in the following section.

Spring Boot’s Docker Compose plugin.

From Spring Boot version 3.1.0, you can deploy your application with a definition of a Docker Compose together. So I created a <compose.yaml> file inside the <functions/city-data-function/tools/spring> directory with the following content:

services:
  localstack:
    image: "localstack/localstack:latest"
    ports:
      - "4566:4566"
    env_file:
      - spring.env
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ../../src/test/resources/localstack/table-data.json:/var/lib/localstack/table-data.json
      - ../../src/test/resources/localstack/table-setup.sh:/etc/localstack/init/ready.d/table-setup.sh

Notice I’m defining the LocalStack service only. I don’t need another container definition to execute my Lambda function. Notice that I’m also adding inside the LocalStack container’s volume the same files I’m using in the <compose.yaml> file located in the project’s root directory. So, we can reuse our configurations here, too.

For this to work, you must add the following Spring Boot dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-docker-compose</artifactId>
    <scope>runtime</scope>
    <optional>true</optional>
</dependency>

As I’m using a <local> Spring profile to configure my application for local deployment, I need to update the <local-application.properties> file with the following configurations:

logging.level.web=debug

spring.main.web-application-type=reactive
spring.cloud.function.web.export.enabled=true

spring.cloud.aws.region.static=us-west-2
spring.cloud.aws.credentials.accessKey=test
spring.cloud.aws.credentials.secretKey=test
spring.cloud.aws.endpoint=http://localhost:4566

spring.docker.compose.enabled=true
spring.docker.compose.lifecycle-management=start_and_stop
spring.docker.compose.file=functions/city-data-function/tools/spring/compose.yaml

The <spring.cloud.aws> properties are used for the Spring Cloud AWS dependency. The <spring.cloud.aws.endpoint> property has the value of <http://localhost:4566>, which is the endpoint of the LocalStack container.

The <spring.docker.compose> properties are used for the Spring Docker Compose dependency. The <spring.docker.compose.file> has the path to our previous <compose.yaml> file.

To start the Spring Boot application, you can execute our Maven command again with the <tracing-agent> profile. But this time, you will see some log messages related to the Docker Compose plugin:

Now, you can invoke the Lambda function directly using the following commands:

curl -H "Content-Type: application/json" "http://localhost:8080/findByIdFunction" \
  -d @functions/city-data-function/src/test/resources/requests/valid/lambda-valid-id-request.json
  
curl -H "Content-Type: application/json" "http://localhost:8080/findByIdFunction" \
  -d @functions/city-data-function/src/test/resources/requests/non-valid/lambda-wrong-uuid-request.json

I chose these 2 requests for the Tracing Agent to capture both the successful case and the one that produces responses with an error code.

In the other terminal window/tab, you will see log messages concerning the request process:

After that, you must execute the following command to copy the generated files from the Tracing Agent with the findings of the function’s operation:

cp -rf functions/city-data-function/target/native-image/* \
  functions/city-data-function/src/main/resources/META-INF/native-image

Notice that the generated files by the Tracing Agent are located in the Maven <target> folder:

These files must now be situated inside the <META-INF/native-image> as follows so the GraalVM build process can take these findings to generate the native image with more contextual information:

Finally, you can execute the main project’s Docker Compose file again:

docker compose --build

Now, you can invoke the Lambda function deployed in Docker Compose to see the results:

curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" \
  -d @functions/city-data-function/src/test/resources/requests/valid/lambda-valid-id-request.json

You must see that this time, the curl command will return a valid response:

Verify that in the Docker Compose terminal, there are no errors in the log messages:

As you can see, there are no errors in the terminal window/tab, and you also have a valid JSON response from the curl command. So, we can conclude that the Tracing Agent helps a lot when generating a native image file with more context about the application execution.

Deploying to AWS using SAM-CLI.

As we did in our previous tutorial, you must execute the following commands inside the <functions> directory, where is the SAM config files:

sam build

sam deploy                                       \
    --parameter-overrides SpringProfile='dev'    \
    --no-confirm-changeset                       \
    --disable-rollback                           \
    --profile 'your-aws-profile'

After a successful deployment, you can run the following command to invoke the Lambda function in AWS from the project’s root directory:

aws lambda invoke                           \
  --function-name 'city-data-function'      \
  --payload file://functions/city-data-function/src/test/resources/requests/valid/lambda-valid-id-request.json \
  --cli-binary-format raw-in-base64-out     \
  --profile 'your-aws-profile'              \
  ~/Downloads/response.json

You can open the content of the <response.json> file to see the result:

Finally, you can open the CloudWatch console to see the log messages:

Excellent!! You deployed a Lambda function in AWS that uses Spring Cloud Function and Spring WebFlux, so you build a reactive Lambda function. But if that is not enough, we can also create a native Linux executable with the help of GraalVM and the Tracing Agent, so our Lambda function is also a native image.

I hope this article has been helpful to you, and I’ll see you in the next one.

Thanks for reading.