S3 Multi-Region Access Point(MRAP) is not available unless with additional dependency

[AllanZhengYP]

Description

Package @aws-sdk/client-s3 launched the support for S3 Multi-Region Access Point (MRAP) feature in v3.31.0. This feature relies on signer class from aws-crt package. This dependency requires native addons that takes up to 10.5 MB on disk. Because the dependency is only required by MRAP feature, we are omitting the dependency in @aws-sdk/client-s3.

To use this feature, you need to install this dependency explicitly in your application:

$ npm install @aws-sdk/signature-v4-crt
$ yarn add @aws-sdk/signature-v4-crt

Because aws-crt contains platform-specific binary code, the following use cases are not supported in MRAP:

• browsers
• React Native
• bundle as Node.js application

The suggested use case for supporting MRAP in Node.js app is to deploy your application with the complete node_modules folder.

Describe alternatives you've considered

We are working on solutions to unblock the MRAP for all the supported platforms.

Related issues:

• @aws-sdk/client-s3 makes node_modules too large #2747
• @aws-sdk/client-s3 (@aws-sdk/signature-v4-crt) 3.29.0 introduces aws-crt and deprecated dependencies #2750
• AWS CRT binary not present in any of the following locations #2753
• Missing dependency @aws-sdk/signature-v4-crt after updating to client-s3 3.33.0 #2806
• Missing Peer Dependency | @aws-sdk/client-s3 missing @aws-sdk/signature-v4-crt #2960

[seriousme]

FYI: One might consider porting the signer class of the common runtime to webassembly. Web assembly is supported in all major browsers and in NodeJs 8+ See also: https://developer.mozilla.org/en-US/docs/WebAssembly/C_to_wasm

Kind regards,
Hans

[AllanZhengYP]

Thank you for the advice! @seriousme

We are currently weigh in the options including WASM, ASM.js, and pure JS with WebCrypto. Currently one of the obstacle is that WASM seems not supported in all React Native platforms, so we might need to fallback to use ASM.js in React Native.

On the other hand, we need to make sure the signer is cryptographically secure. If the implementation involves 3rd party crypto primitives, we need to audit them. For example, the signing algorithm in SigV4a requires constant time implementation.

[seriousme]

Good to see you have options :-)

You can mix and match, e.g. use the browsers webcrypto from WASM ;-)
e.g. https://ihsavru.medium.com/calling-javascript-code-from-c-c-using-webassembly-a9445c11bc6d

Kind regards,
Hans

[christophgysin]

Adding aws-crt blocks me from updating the deno port of aws-sdk-js-v3 past v3.33.0.

👍 for WASM, which would also allow using this in deno.

[rossng]

Just want to weigh in with the impact this issue currently has on our team. Hopefully it helps to motivate a fix. Our developers use a mix of macOS and Windows machines.

We depend on the @aws-sdk/client-s3 package. This causes the package manager to complain about the missing @aws-sdk/signature-v4-crt peer dependency.

The Windows users go ahead and add the dep to package.json, and everything appears to be resolved. Then a macOS user pulls the latest changes and suddenly package installation is completely broken.

This isn't a great experience. Given that @aws-sdk/signature-v4-crt has ~63k downloads per week, I can only imagine how many developers have run headlong into this so far.

[jekh]

I just ran into this issue, seeing the same peer dependency warnings as @rossng. Using the peerDependenciesMeta field of package.json might be a good solution: https://docs.npmjs.com/cli/v8/configuring-npm/package-json#peerdependenciesmeta

Adding this to package.json in middleware-sdk-s3 should fix this:

"peerDependenciesMeta": {
   "@aws-sdk/signature-v4-crt": {
     "optional": true
   }
 }

[AllanZhengYP]

@jekh Thanks a lot for pointing this out! It's a miss on our side. I will address them a separate PR.