(v3.616.0) @aws-sdk/client-s3 - ListObjectsV2Command prematurely terminate lambda function with no exception

[aws-kens]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

ListObjectsV2Command prematurely terminated in lambda function without any response nor any exception.

This issue happens on @aws-sdk/client-s3, Version 3.616.0.
Previous version (3.614.0) does not have the problem.

SDK version number

@aws-sdk/client-s3@3.616.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.x

Reproduction Steps

1. Create a lambda function with NodeJS 20.x
2. Copy the following code. Note that it is using CommonJS (index.js)

const PATH = require('path');
const {
  S3Client,
  ListObjectsV2Command,
} = require('@aws-sdk/client-s3');
const packageVersion = require('@aws-sdk/client-s3/package.json');

exports.handler = async (event, context) => {
  const {
    bucket,
    prefix,
  } = event;

  const {
    name,
    version,
  } = packageVersion;

  console.log('VERSION', name, version);

  const s3Client = new S3Client({
    computeChecksums: true,
    applyChecksum: true,
  });

  const keys = [];
  let response;
  do {
    const command = new ListObjectsV2Command({
      Bucket: bucket,
      Prefix: prefix,
      ContinuationToken: (response || {}).NextContinuationToken,
      MaxKeys: 1000,
    });

    try {
      console.log('BEFORE', 'ListObjectsV2Command');
      response = await s3Client.send(command);
      console.log('AFTER', 'ListObjectsV2Command');

      if (response.Contents !== undefined) {
        for (const item of response.Contents) {
          if (item.Key) {
            keys.push(item.Key);
          }
        }
      }
    } catch (e) {
      console.log('ERR', 'ListObjectsV2Command');
      console.error(e);
      throw e;
    }
  } while ((response || {}).NextContinuationToken);

  console.log(keys.slice(0, 10), keys.length);

  return response;
};

3. Add the aws-sdk-js-v3 layer with the @aws-sdk/client-s3@3.616.0
4. Configure the test event as follows:

{
  "bucket": "SOME_BUCKET",
  "prefix": "SOME_PREFIX/"
}

5. Run the test.
6. The CloudWatch logs never shows that the ListObjectsV2Command returns but the lambda function exits with no response data and with no exception being thrown.
7. Change the layer to use @aws-sdk/client-s3@3.614.0
8. Rerun the test
9. The CloudWatch logs shows the lambda function returns correctly with the expected response data.

Observed Behavior

Here are the CloudWatch logs with two different version of the @aws-sdk/client-s3.

CloudWatch Logs (3.614.0) -- OK

Response

{
  "$metadata": {
    "httpStatusCode": 200,
    "requestId": "6F5YEQ40P6DC8HJV",
    "extendedRequestId": "gEnu65JerfnkrNvbyKwT/V/q2zQPGK/h3UN7V1bBkgjzKZnE/j+KK2TEFK8SgcS6s5ZaQbN1QFg=",
    "attempts": 1,
    "totalRetryDelay": 0
  },
  "Contents": [
    {
      "Key": "SOME_PREFIX/mediainfo/mediainfo.json",
      "LastModified": "2024-07-21T10:19:16.000Z",
      "ETag": "\"623448607dcefc2a483690ec245da77b\"",
      "Size": 6807,
      "StorageClass": "STANDARD"
    },
    {
      "Key": "SOME_PREFIX/mediainfo/mediainfo.xml",
      "LastModified": "2024-07-21T10:19:16.000Z",
      "ETag": "\"77a4c28503625051db5b6c0196f9b9d0\"",
      "Size": 15564,
      "StorageClass": "STANDARD"
    },
  ],
  "IsTruncated": false,
  "KeyCount": 524,
  "MaxKeys": 1000,
  "Name": "SOME_BUCKET",
  "Prefix": "SOME_PREFIX/"
}

Function Logs
START RequestId: 49cb24a1-e9c4-47b4-b654-e2374721b728 Version: $LATEST
2024-07-21T13:37:37.905Z 49cb24a1-e9c4-47b4-b654-e2374721b728 INFO VERSION @aws-sdk/client-s3 3.614.0
2024-07-21T13:37:37.997Z 49cb24a1-e9c4-47b4-b654-e2374721b728 INFO BEFORE ListObjectsV2Command
2024-07-21T13:37:41.017Z 49cb24a1-e9c4-47b4-b654-e2374721b728 INFO AFTER ListObjectsV2Command
2024-07-21T13:37:41.018Z 49cb24a1-e9c4-47b4-b654-e2374721b728 INFO [
'SOME_PREFIX/mediainfo/mediainfo.json',
'SOME_PREFIX/mediainfo/mediainfo.xml',
'SOME_PREFIX/metadata/entity/output.json',
'SOME_PREFIX/metadata/keyphrase/output.json',
'SOME_PREFIX/metadata/scene/scene.json',
'SOME_PREFIX/metadata/scene/sequence-000.jpg',
'SOME_PREFIX/metadata/scene/sequence-001.jpg',
'SOME_PREFIX/metadata/scene/sequence-002.jpg',
'SOME_PREFIX/metadata/scene/sequence-003.jpg',
'SOME_PREFIX/metadata/scene/sequence-004.jpg'
] 524
END RequestId: 49cb24a1-e9c4-47b4-b654-e2374721b728
REPORT RequestId: 49cb24a1-e9c4-47b4-b654-e2374721b728 Duration: 3351.96 ms Billed Duration: 3352 ms Memory Size: 128 MB Max Memory Used: 99 MB Init Duration: 381.02 ms

CloudWatch Logs (3.616.0) -- NG

Response
null

Function Logs
START RequestId: 46df0896-3ebc-45a2-b98d-88ce20b2eb11 Version: $LATEST
2024-07-21T13:36:07.960Z 46df0896-3ebc-45a2-b98d-88ce20b2eb11 INFO VERSION @aws-sdk/client-s3 3.616.0
2024-07-21T13:36:08.052Z 46df0896-3ebc-45a2-b98d-88ce20b2eb11 INFO BEFORE ListObjectsV2Command
END RequestId: 46df0896-3ebc-45a2-b98d-88ce20b2eb11
REPORT RequestId: 46df0896-3ebc-45a2-b98d-88ce20b2eb11 Duration: 1193.08 ms Billed Duration: 1194 ms Memory Size: 128 MB Max Memory Used: 92 MB Init Duration: 410.20 ms

PROBLEMS

• AFTER ListObjectsV2Command was never reported
• The lambda function returned without any response and without any error/exception.

Expected Behavior

@aws-sdk/client-s3@3.616.0 ListObjectsV2Command should return a list of S3 objects.

Possible Solution

The last working version is @aws-sdk/client-s3@3.614.0

Additional Information/Context

No response

[kuhe]

Thank you for identifying this bug, a PR #6294 has been prepared for the next release. We expect the fix to be available by tomorrow at this time.

[kuhe]

I've untagged @aws-sdk/client-s3@3.616.0, and tagged @aws-sdk/client-s3@3.614.0 as latest.

This bug affects S3 operations in v3.616.0 with XML response bodies longer than 3kb (but not including S3::getObject). If affected, as a workaround, please use v3.614.0.

[kuhe]

The patch was released in https://www.npmjs.com/package/@aws-sdk/client-s3/v/3.617.0