Preview 4 of AWS SDK for .NET V4

In August 2024, we announced the first preview of our upcoming version 4 of the AWS SDK for .NET. Since then we have continued making progress and released new previews as we go. At the time of writing this post, the SDK has released preview 4. In this post, we’ll take a look at some of the changes made since preview 1.

Amazon DynamoDB Object Persistence high-level library

For the V4 SDK, we are making a few minor breaking changes to the Object Persistence high-level library to steer users to best practices and improve the mockability of the library for testing.

The Object Persistence high-level library is available in the AWSSDK.DynamoDBv2 package under the Amazon.DynamoDBv2.DataModel namespace. The library allows you to use .NET types to represent items in a DynamoDB table, and provides APIs for saving and loading items as the .NET types. For example, the following code shows how a User type is used to save and load a user from a DynamoDB table.

var user = new User { Username = "johndoe", FirstName = "John", LastName = "Doe" };

await context.SaveAsync(user);

var retrievedUser = await context.LoadAsync<User>("johndoe");
Console.WriteLine(retrievedUser.FirstName);

Initialization defaults

In October 2023, we introduced new patterns for initializing the library’s DynamoDBContext object that avoid the library doing background DescribeTable calls. As mentioned in the initialization announcement, the DescribeTable call can cause performance problems for applications. In V4, we want to encourage the new patterns, but for backward compatibility, not remove the DescribeTable mechanism.

To encourage the new patterns, we have obsoleted the public constructors of DynamoDBContext that would default to the DescribeTable behavior. To construct the DynamoDBContext, we have introduced the DynamoDBContextBuilder type which will build the object in the preferred initialization mode.

DynamoDBContext context = new DynamoDBContextBuilder()
                    // Optional call to provide a specific instance of IAmazonDynamoDB
                    .WithDynamoDBClient(() => client)
                    .Build();

If your application needs the old behavior of using DescribeTable, you can use the ConfigureContext method of the builder to toggle the DisableFetchingTableMetadata property.

DynamoDBContext context = new DynamoDBContextBuilder()
                    .ConfigureContext(config =>
                    {
                        config.DisableFetchingTableMetadata = false;
                    })
                    .Build();

Operation specific configuration

For V4, we have obsoleted the methods of the DynamoDBContext object that took in the generic configuration object DynamoDBOperationConfig. Replacement overloads have been added that take a method-specific configuration object. For example, if your code was calling QueryAsync on the DynamoDBContext, then you pass in a QueryConfig object to configure the index or the filter to use.

This change was implemented because DynamoDBOperationConfig acted as a union type of all the possible configurations across the methods of DynamoDBContext. This caused some confusion because some properties of DynamoDBOperationConfig would not be applicable for some methods. For example, the IndexName on DynamoDBOperationConfig had no effect on the LoadAsync method because an item is loaded by making a GetItem call to DynamoDB with the hash and optional range key.

Enhanced Mocking

The other major change we have made to the Object Persistence library is to improve its testability. This largely means changing return types and input parameters to use interfaces. For most applications that use the var keyword, the application will just need to be recompiled. If your code was explicitly using the type names, then you need to change the type to the interface. The following code shows the required change that is needed for V4 by switching to IAsyncSearch.

// V3 code
AsyncSearch<User> search = context.QueryAsync<User>("Norm", ...

// V4 code
IAsyncSearch<User> search = context.QueryAsync<User>("Norm", ...

By making these changes, the library allows you to mock these operations, which makes it easier for you to write unit tests that don’t require actually writing to DynamoDB. The following example comes from our SDK’s repository showing how we mocked a DynamoDB query.

[TestMethod]
public async Task TestMockability_QueryAsync()
{
    var mockContext = new Mock<IDynamoDBContext>();
    mockContext
        .Setup(x => x.QueryAsync<string>(It.IsAny<object>()))
        .Returns(CreateMockAsyncSearch(new List<string> { "item1", "item2" }));

    var ddbContext = mockContext.Object;
    var asyncSearch = ddbContext.QueryAsync<string>(null);

    var results = await asyncSearch.GetNextSetAsync();
    Assert.AreEqual(2, results.Count);
    Assert.AreEqual("item1", results[0]);
    Assert.AreEqual("item2", results[1]);
}

The testability improvements were also made to the Amazon.DynamoDBv2.DocumentModel library, introducing the ITable interface and interfaces for batch operations such as IDocumentBatchWrite and IDocumentBatchGet.

Amazon DynamoDB Streams

DynamoDB Streams is a feature of DynamoDB that you can enable. When enabled, you can use a stream to read changes that have been made to your tables. In V3, the DynamoDB Streams service client, AmazonDynamoDBStreamsClient, was bundled in the AWSSDK.DynamoDBv2 package. This was inconsistent with all of our other service clients that were contained in their own packages, and made the AWSSDK.DynamoDBv2 package larger then it needed to be for users who were only using the DynamoDB API.

In the V4 SDK, we have moved DynamoDB Streams into its own package: AWSSDK.DynamoDBStreams. Besides the general V4 changes, the API for DynamoDB Streams has not changed in V4. For applications that already use the DynamoDB Streams service client, the upgrade process would be to add the new AWSSDK.DynamoDBStreams package to the application. The Amazon.DynamoDBStreams and Amazon.DynamoDBStreams.Model namespaces will need to be added wherever the service client is used.

S3 us-east-1 behavior

With the V4 SDK, the AWS SDK for .NET will follow the pattern of newer AWS SDKs and treat us-east-1 as a regional-only endpoint. If in your application you are unsure of the Region a bucket is in, the GetBucketLocation method from the S3 service client can be used to determine the Region. This method can be called for a service client configured for any Region.

In V3, the AWS SDK for .NET has legacy behavior where it treated the us-east-1 Region like a global endpoint. For example, you could configure the S3 service client for us-east-1 and make requests to a bucket in eu-west-1. This behavior originated from when AWS was just a single region.

AWS Signature Version 4 (SigV4) is the default for authenticating requests. Treating us-east-1 as a global endpoint has behavior that can make some S3 client requests take twice as long. When the SigV4 signature is computed by the SDK, the Region of the bucket is part of the signature. If the bucket being addressed is not in us-east-1, the SDK gets a signature error that the bucket needs to be signed with the bucket’s Region. The SDK then resigns the request with the Region returned in the error message and makes a second call.

In addition, many customers have rules about not accessing any resources outside of the Region that is running the application. In fact, this is a common requirement for how applications are built at Amazon to avoid inter-region dependencies. This requirement becomes difficult to enforce if the SDK treats us-east-1 as a global endpoint.

Extension libraries

The following list contains the AWS SDK for .NET extension packages for which we have published preview versions using the V4 SDK. For those who are curious about the effort to update to V4, the source changes can be viewed in the v4sdk-development branch in each of their respective GitHub repositories.

While updating each of these extension packages, we have also updated them to be trimmable, similar to the SDK packages. This makes it easier and more performant to use these packages for Native AOT Lambda functions.

We will continue releasing preview versions of the extension packages to be ready for V4. Our hope is to have all of the extension packages updated to V4 by the time we GA V4.

Conclusion

For the full list of changes in the V4 SDK, check out this pinned GitHub issue on our repository. You can subscribe to the issue to get notified of changes. We would appreciate the community testing out V4 and providing any feedback they have. You can give us feedback by opening issues or discussions on our repository.