JSON Structured Logging for .NET Lambda Functions

We are announcing support for JSON structured logging for the .NET managed runtime. This makes the .NET managed runtime compatible with the previously announced logging controls for AWS Lambda, allowing you to toggle logging format and log levels using the Lambda API.

Formatting log messages as JSON documents makes it easier to search, filter, and analyze your Lambda function’s logs. This can be important for setting up monitoring and alarms with certain fields found in log messages.

Enabling JSON Structured Logging

By default the log format of Lambda functions is Text. The log format can be changed using AWS tools like the AWS Console, the .NET Amazon.Lambda.Tools global tool, AWS Tools for PowerShell, and the AWS CLI. In the Lambda console, the logging configuration can be edited in the “Monitoring and operations tools” tab under “Configuration”.

Starting with version 5.11.0 of Amazon.Lambda.Tools, the command line switches --log-format, --log-application-level, --log-system-level, and --log-group were added for configuring a function’s logging. The following command shows how to deploy a .NET Lambda function with JSON logging format

dotnet tool install -g amazon.lambda.tools
dotnet lambda deploy-function <function-name> --log-format JSON

Formatting log messages as JSON

The JSON log messages are written as a single line with newlines escaped in the message. For clarity, this post will show the JSON log messages in “pretty print” style.

Note: With container based Lambda functions, each newline is treated as a separate CloudWatch Log message. This can make working with stack traces difficult with each line of the stacktrace being a separate CloudWatch Log message. Using JSON format, the entire stacktrace along with the rest of the error log message will be captured as a single CloudWatch Log message.

Logging in .NET Lambda functions is done through Log methods from the context.Logger property of the ILambdaContext. Alternatively the Write methods from System.Console and System.Console.Error are captured as informational log messages and error log messages, respectively. Using the logging call context.Logger.LogInformation($"User name is: {user}"); in a Lambda function, the default Text format would produce the following message:

2024-10-10T23:44:41.090Z        7400fe09-12fe-47bb-82b4-e1b66515b7a9    info    User name is: johndoe

The log message contains the date, request id, log level, and log message. By switching the format to JSON, these fields are set to separate properties in the JSON document.

{
  "timestamp": "2024-10-10T23:43:45.178Z",
  "level": "Information",
  "requestId": "2749600f-9d28-49bc-8426-8a82ceb8b2aa",
  "traceId": "Root=1-670866b1-37ec54281f2001ad21241401;Parent=7f1365fb4697b795;Sampled=0;Lineage=1:8ae6cf71:0",
  "message": "User name is: johndoe"
}

In our log message, we used C#‘s string interpolation features to create the log message. Starting with version 2.4.0 of the Amazon.Lambda.Core package, new parameterized logging APIs were added. The previous logging call can be rewritten as context.Logger.LogInformation("User name is: {user}", user);. This gives two advantages. First, you can avoid unnecessary string allocations for log messages that would be filtered out. For example, if the log message was written using LogDebug but the function is configured for INFO log level, the work to replace the string parameters in the log message is skipped. The second advantage is that each parameter to the log message becomes a property in the JSON document. In the following example, you can see that the user property has been added to the JSON document.

{
  "timestamp": "2024-10-10T23:55:58.329Z",
  "level": "Information",
  "requestId": "d72c572e-704f-417f-8bb1-7aa3974ac03f",
  "traceId": "Root=1-6708698d-005d2a3c1d07dad16fac63ed;Parent=066d58c44405276a;Sampled=0;Lineage=1:8ae6cf71:0",
  "message": "User name is: johndoe",
  "user": "johndoe"
}

With the log parameters as JSON properties, we can easily search through the logging messages for the property instead of having to parse unstructured log messages.

Composite formatting

The .NET composite formatting tokens can be used for formatting the parameters. For example, let’s say we have a cost variable with a value of 8.12345, which we need to be rounded to two decimal points in our logging. The logging call would be context.Logger.LogInformation(“The cost is {cost:0.00}”, cost);, and would produce the following JSON document.

{
  "timestamp": "2024-10-11T00:02:17.025Z",
  "level": "Information",
  "requestId": "89ee3cf9-ed71-4943-bf5e-f7c96da4f7fc",
  "traceId": "Root=1-67086b08-7fe61c095eb668ea5de71818;Parent=5188732bae5990a6;Sampled=0;Lineage=1:8ae6cf71:0",
  "message": "The cost is 8.12",
  "cost": "8.12"
}

Custom type parameters

In the user-logging example shown earlier in this post, the user parameter was a string. Now consider an alternative where the user parameter is a custom type similar to the following:

public class User
{
    public string FirstName { get; set; }
    public string LastName { get; set; }

    public override string ToString()
    {
        return $"{LastName}, {FirstName}";
    }
}

The ToString method of the custom type is used as the value that is inserted into the log statement for the JSON message property and into the JSON property value for user. It is recommended to override the ToString method of your custom types to put a meaningful value into the JSON logging. Otherwise the default .NET implementation of ToString will be used, which returns the type name. Reusing the previous logging call context.Logger.LogInformation("User name is: {user}", user); produces the following JSON log message using the ToString method for the User type.

{
  "timestamp": "2024-10-11T00:15:49.172Z",
  "level": "Information",
  "requestId": "ef7a0425-0240-4244-8631-b58a3d9c2009",
  "traceId": "Root=1-67086e34-0a03b81820601b4f0aef6eaf;Parent=273d1f289c2bfbbe;Sampled=0;Lineage=1:8ae6cf71:0",
  "message": "User name is: Doe, John",
  "user": "Doe, John"
}

If the desire is to have the object serialized in the JSON document, the @ prefix can be used in the logging call: context.Logger.LogInformation("User name is: {@user}", user);.

{
  "timestamp": "2024-10-11T00:15:49.194Z",
  "level": "Information",
  "requestId": "ef7a0425-0240-4244-8631-b58a3d9c2009",
  "traceId": "Root=1-67086e34-0a03b81820601b4f0aef6eaf;Parent=273d1f289c2bfbbe;Sampled=0;Lineage=1:8ae6cf71:0",
  "message": "User name is: Doe, John",
  "user": {
    "FirstName": "John",
    "LastName": "Doe"
  }
}

JSON serialization of types is done using System.Text.Json. To customize serialization, .NET attributes like JsonPropertyNameAttribute and JsonIgnoreAttribute can be used.

Collections

For collection parameters, the items in the collection are written to the JSON log using the ToString method. The @ prefix can be used in a similar manner to custom types to indicate that the items in the collection should be serialized into the JSON log message.

Following is an example using ToString:

context.Logger.LogInformation("Active users: {users}", users);

{
  "timestamp": "2024-10-11T20:50:19.733Z",
  "level": "Information",
  "requestId": "b4ddedf3-ecfa-4e23-9df5-86e6e5efe0f7",
  "traceId": "Root=1-67098f8b-0493863a66773b082b585d9c;Parent=3d059f6f74b110d9;Sampled=0;Lineage=1:8ae6cf71:0",
  "message": "Active users: {users}",
  "users": [
    "Doe, John",
    "Doe, Jane"
  ]
}

Following is an example using serialized items:

context.Logger.LogInformation("Active users: {@users}", users);

{
  "timestamp": "2024-10-11T20:50:19.733Z",
  "level": "Information",
  "requestId": "b4ddedf3-ecfa-4e23-9df5-86e6e5efe0f7",
  "traceId": "Root=1-67098f8b-0493863a66773b082b585d9c;Parent=3d059f6f74b110d9;Sampled=0;Lineage=1:8ae6cf71:0",
  "message": "Active users: {@users}",
  "users": [
    {
      "FirstName": "John",
      "LastName": "Doe"
    },
    {
      "FirstName": "Jane",
      "LastName": "Doe"
    }
  ]
}

Conclusion

JSON structured logging provides a lot of flexibility for searching and analyzing your Lambda function’s logs. To get started with either new or existing functions, update the log format configuration of the Lambda function.

To use the new parameterized logging API, be sure to reference version 2.4.0 or later of Amazon.Lambda.Core. If your Lambda function is using the executable assembly programming model where Amazon.Lambda.RuntimeSupport is included in the deployment package, be sure that version 1.12.0 or later is referenced.

For feedback on the logging for .NET Lambda functions, open a GitHub issue or discussion on our aws/aws-lambda-dotnet repository.