Calculate AWS v4 Signature with client-side JavaScript

Photo by Ray Reyes on Unsplash.

My team recently completed a challenging 5-month product build that integrates with many Amazon Web Services (AWS) features. We had the opportunity to use some new services, namely AWS Cognito, which forced us to pioneer new solutions more than a typical project. First up: calculating AWS v4 signatures client-side to integrate EvaporateJS with AWS Cognito.

Benefits of AWS Cognito

Although some parts of AWS Cognito feel incomplete (multi-factor authentication and documentation come to mind), it offers great potential by allowing users to authenticate directly to the AWS ecosystem. When Member Matt logs in, he can run client-side JavaScript to authenticate and talk directly with the AWS API. And when Admin Amber logs in, Cognito's group feature allows her to run elevated commands that Matt cannot. This lets us securely interact with AWS without intermediary servers to proxy commands.

Case in point: uploading private files to AWS S3. We want users of our product to store sensitive files in S3 without the ability to read or overwrite another user's files. We could upload files to a custom API that identifies the user and saves their files to S3, but this adds a lot of overhead. Instead, Cognito gives each user their own AWS credentials so they can read and write directly to S3, and we can write AWS policies to isolate users (see "Segment S3 uploads by Cognito user").

Without Cognito, user uploads to custom API which proxies the file to S3

With Cognito, user authenticates and uploads directly to S3

Uploading to S3 with EvaporateJS

To upload files to S3 we chose EvaporateJS, which immediately added value like multi-part uploads, pausing, resuming, and progress callbacks.

The following code uses EvaporateJS 1.x. The current version of EvaporateJS, 2.x, adds customAuthMethod so the implementation is slightly different. See this EvaporateJS 2.x example by paolavness.

import { assign, curry, mapKeys, map } from 'lodash';
import Evaporate from 'evaporate';

// These cryptography functions are defined later in the articleimport { hashSha256ToHex, hmacSha256 } from './crypto';
/**
 * Generate the signature for AWS to authenticate the request.
 * @param {String} secretKey - The IAM secret key of the AWS user making the request
 * @param {Object} response - The response from the signerUrl or awsLambda requests
 * @param {String} stringToSign - The "string to sign" generated in the previous step
 *   of the AWS Signature v4 process.
 * @returns {String} - The signature included in the Authorization header for AWS
 *   to authenticate the request.
 */
const signRequest = (secretKey, response, stringToSign) => {
	// stringToSign is URL-encoded by EvaporateJS, but the signature must use the decoded string.
	const stringToSignDecoded = decodeURIComponent(stringToSign);
	// Signing is defined later in the article};

class S3Manager {
	/**
	 * Create an instance of S3Manager.
	 * @constructor
	 * @param {String} accessKey - Uploader's AWS access key
	 * @param {String} secretKey - Uploader's AWS secret key
	 * @param {String} sessionToken - Uploader's AWS session token
	 */
	constructor(accessKey, secretKey, sessionToken) {
		this.manager = new Evaporate({
			aws_key: accessKey,
			awsRegion: 'my-aws-region-identifier', // TODO: Specify this
			awsSignatureVersion: '4', // this is default in EvaporateJS 2.x
			bucket: 'my-s3-bucket-name', // TODO: Specify this
			// Required for AWS Signature Version 4
			cryptoHexEncodedHash256: hashSha256ToHex,
			signResponseHandler: curry(signRequest)(secretKey),
		});
		this.sessionToken = sessionToken;
	}

	/**
	 * Upload a file to AWS S3.
	 * @param {File} file - The File object uploaded to the browser
	 * @param {Object} transferHeaders - Key-value pairs of headers to add to the AWS request, like
	 *   x-amz-server-side-encryption or x-amz-meta-* headers
	 * @param {Object} options - EvaporateJS properties to set/override for this upload
	 */
	upload(file, transferHeaders = {}, options = {}) {
		const authHeaders = { 'x-amz-security-token': this.sessionToken };
		const params = assign(
			{
				file,
				name: 'encoded/upload/path.extension', // TODO: Specify this
				xAmzHeadersAtInitiate: assign({}, authHeaders, transferHeaders),
				xAmzHeadersCommon: authHeaders,
			},
			options,
		);
		return this.manager.add(params);
	}
}

To create an instance of S3Manager using our logged-in cognitoUser:

const getS3Manager = () => {
	return new Promise((resolve, reject) =>
		cognitoUser.getSession((sessionErr, tokens) => {
			if (sessionErr) {
				return reject(sessionErr);
			}

			// getSession() will populate AWS.config.credentials
			AWS.config.credentials.get(credentialsErr => {
				if (credentialsErr) {
					return reject(credentialsErr);
				}

				const {
					accessKeyId,
					secretAccessKey,
					sessionToken,
				} = AWS.config.credentials;
				const s3Manager = new S3Manager(
					accessKeyId,
					secretAccessKey,
					sessionToken,
				);
				return resolve(s3Manager);
			});
		}),
	);
};

Signing the AWS Request

Using third-party libraries to interact with AWS requires us to cryptographically sign our own requests. The AWS SDK takes care of this but other libraries do not.

Signature Version 4 is the latest method for signing AWS requests. Of the four steps, EvaporateJS handles all except the third.

3. You use your AWS secret access key to derive a signing key, and then use that signing key and the string to sign to create a signature.

The EvaporateJS docs explain how to use AWS Lambda to sign requests. This is comparable to the Custom API proxy described earlier, in that we need to use our AWS Secret Key when calling AWS but we can't reveal it to the user. But with Cognito, that's no longer an issue! How can we use the Cognito user's credentials to sign the EvaporateJS request?

There are many popular client-side cryptography libraries, and I played with the v4 signature in several. Here are examples of the signRequest function from the code above:

These examples calculate the v4 signature of the stringToSign variable. To integrate with EvaporateJS, be sure to instead sign the stringToSignDecoded variable in the signRequest function.

Still More Cryptography

Finally, EvaporateJS uses other cryptographic operations, some required (like cryptoHexEncodedHash256) and some just encouraged (like cryptoMd5Method). Here are implementations of those functions, compared side-by-side for accuracy:

I hope these examples help you dive into exciting libraries like AWS Cognito and EvaporateJS!

Drew

Drew

Hi! I'm Drew, the Wimpy Programmer. I'm a software developer and formerly a Windows server administrator. I use this blog to share my mistakes and ideas.

Comments on "Calculate AWS v4 Signature with client-side JavaScript"

Avatar for ChardChard

Hi,

I got confused on this statement

But with Cognito, that’s no longer an issue! How can we use the Cognito user’s credentials to sign the EvaporateJS request?

There are many popular client-side cryptography libraries, and I played with the v4 signature in several. Here are examples of the signRequest function from the code above:

What type of cognito user's credential are going to be used to sign the request? Is that just a jwt token that was issued by cognito, that can be verified then by an AWS Lambda when EvaporateJS tries to upload and contacts the signer_url?

Avatar for DrewDrew

Gah, I'm so sorry that I overlooked your comments. I misconfigured my server and I didn't receive the notifications that they were in the moderation queue.

I struggled to convey the details of this setup. I think you mostly followed everything I was trying to say, but I'll restate some points just in case.

Traditionally a website uses a single AWS account to upload to S3, and these account credentials must be kept on the server to avoid exposing them on the frontend. But once a Cognito user signs in, they have personal AWS credentials like an access key, secret key, and session token. We can use those personal credentials to authenticate directly to S3, without exposing any secret keys in our JavaScript and without needing a server (or Lambda) to protect our secrets.

In my approach, we set a signResponseHandler in lieu of a signerUrl. Our goal is to sign the request client-side without needing to contact a server. The signRequest() function handles this, and the content of that function varies depending on which client-side cryptography library you're using (CryptoJS, Crypto Browserify, the AWS SDK, etc.). What cryptography library are you using to sign the request?

The JWTs created by Cognito are great for many things, but not used here. Instead it's cognitoUser.getSession() followed by AWS.config.credentials.get() that will yield real AWS credentials.

Avatar for XX

I may misunderstood your diagram, but did you mean to have Lambda function as the function signer? Or you really mean that any client app can just do stuff in s3 with just Cognito credentials?

Avatar for DrewDrew

In the first diagram, "Private Uploads to S3 without Cognito", the Custom API in the middle could also be a Lambda. Basically it represents some extra layer that hides the AWS credentials from the frontend. But yes, this piece is intentionally missing from the "Private Uploads to S3 with Cognito" diagram because Cognito can provide AWS credentials for the client app to use -- no app server is required.

And just to be clear, the username/password entered by the Cognito user are not the same credentials we use to communicate with AWS. A user Bob might log in with the username "bob@example.com" and password "b0bp@ss", but once he's signed in he can get normal AWS credentials like an access key of "AKIDEXAMPLE" and a secret key of "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY".

Avatar for ginogino

Hello,

I tried your example but I get signature does not match. I double check the steps, and everything and it looks okay to me but still getting signature does not match. I'm using evaporate 2.0.8

Avatar for DrewDrew

Unfortunately all of my code was written for Evaporate 1.x. Looking at the 1.x to 2.0 migration instructions, I think item #5 is most problematic for the code in my examples.

Off the top of my head, it looks like you would modify the call to new Evaporate() to rename signResponseHandler to customAuthMethod. Then the signature of the signRequest() function would change to:

const signRequest = (secretKey, signParams, signHeaders, stringToSign, signatureDateTime, canonicalRequest) => {

The content of the signRequest() function would remain the same, except that after you generate the signature (final in my linked examples) with your cryptography library of choice, you would:

return Promise.resolve(final);

to satisfy the expectation that customAuthMethod returns a Promise.

Are you still working on this problem? I know I took a long time to respond and you've probably moved on. If not, and if the suggestion above doesn't help, then I can try to write an Evaporate 2.x example.

Also, it may help to look at the content of the "signature does not match" errors from AWS. They show a lot of detail about how AWS is signing the request, which is very helpful when trying to identically sign it in your app.

Avatar for PaolaPaola

Hi Drew, thanks for this example, it is super helpful. I'm converting it to use with EvaporateJS 2.0.x. Something I am not quite clear on - the stringToSign - do you construct this it from scratch or get it off cognitoUser or AWS credentials? Thanks again, I'll a link to my working result here for yourself and others.

Avatar for DrewDrew

Hey Paola, I'm glad you found this post useful. An example using EvaporateJS v2 would be great, thanks!

In my example, the signRequest function is passed to EvaporateJS as the signResponseHandler property when creating the new Evaporate() object. Therefore the stringToSign parameter is provided by EvaporateJS when it invokes signRequest. You can see where EvaporateJS calls the signResponseHandler function in the old Evaporate v1 code.

Hopefully you won't need to understand all the details of AWS's signing process (I sure don't) since EvaporateJS handles most of it. But basically EvaporateJS follows AWS's instructions to build a long string containing the details of the AWS request. This long string is the stringToSign. Then in signRequest you use your AWS secret key to hash the stringToSign. EvaporateJS includes the hash with the request sent to AWS. When AWS receives your request, their servers follow the same procedure to generate their own stringToSign, then hash it with your secret key (which they are privy to). Lastly they compare their calculated hash with the one in the request that you generated. If the hashes are different, the request is denied because something went wrong, such as:

  • The signature was improperly generated by the code (we'll assume this is not the case)
  • The request was signed with the wrong credentials, suggesting that someone is trying to impersonate you
  • The content of the request received by AWS does not match the content that was signed, suggesting that it was manipulated by an intermediary

Note that my signRequest function may be confusing because I'm using it with curry. In case you're not familiar, curry is basically populating the secretKey parameter immediately, while leaving the remaining function parameters to be populated by EvaporateJS later.

I still haven't worked with EvaporateJS v2, but looking at the docs this aspect hasn't changed much. Instead of passing a signResponseHandler property in the call to new Evaporate(), you will pass a customAuthMethod property to Evaporate.create(). For customAuthMethod you can pass the same value from my original article, curry(signRequest)(secretKey). The signature of signRequest will change to match the new customAuthMethod in EvaporateJS v2:

const signRequest = (secretKey, signParams, signHeaders, stringToSign, signatureDateTime, canonicalRequest) => {

The logic inside signRequest will remain the same -- you'll use a cryptographic library to hash stringToSign with your secretKey. But EvaporateJS v2 now expects the customAuthMethod to return a Promise, so you'll return the signed string ("final" in my examples) wrapped by a Promise like return Promise.resolve(final).

I hope this helps clarify some things! But my Evaporate v2 code was written on-the-fly, untested so it may contain some mistakes.

Avatar for PaolavnessPaolavness

Hi Drew, Thanks so much for this! I adapted for evaporate 2.x - see here: https://github.com/TTLabs/EvaporateJS/issues/380

Avatar for DrewDrew

Awesome, thanks Paola! I've updated the article with a link to your Evaporate 2.x example.

Avatar for GowriGowri

Great article. I tried to follow and implemented same to upload large files to cognito user bucket using EvaporateJS. But getting 403: Forbidden error. But same multipart upload is working using Amplify storage SDK to same bucket, folder, region, accesskey, securitykey. Reason i want to switch from AWS Amplify sdk to Evaporate js is lack of fine grain control over upload. Below is the code. Kindly suggest what i am missing.

const customAuth = (
  signParams,
  signHeaders,
  stringToSign,
  signatureDateTime,
  canonicalRequest
) => {
  const stringToSignDecoded = decodeURIComponent(stringToSign);
  const requestScope = stringToSignDecoded.split("\n")[2];
  const [date, region, service, signatureType] = requestScope.split("/");
  return new Promise(function (resolve, reject) {
    try {
      const round1 = cryptoJs.HmacSHA256(`AWS4` + signParams.secretKey, date);
      const round2 = cryptoJs.HmacSHA256(round1, region);
      const round3 = cryptoJs.HmacSHA256(round2, service);
      const round4 = cryptoJs.HmacSHA256(round3, signatureType);
      const final = cryptoJs.HmacSHA256(round4, stringToSignDecoded, "hex");
      resolve(final);
    } catch (err) {
      console.log("Error creating v4 sig", err);
      reject(err);
    }
  });
};

const uploadToS3 = async (accessKey, secretKey, file, filename) => {
  var config = {
    aws_key: accessKey,
    bucket: awsconfig.aws_user_files_s3_bucket,
    awsRegion: awsconfig.aws_user_files_s3_bucket_region,
    computeContentMd5: true,
    cryptoMd5Method: function (data) {
      return AWS.util.crypto.md5(data, "base64");
    },
    cryptoHexEncodedHash256: function (data) {
      return AWS.util.crypto.sha256(data, "hex");
    },
    logging: false,
    s3Acceleration: false,
    signTimeout: 200,
    maxConcurrentParts: 5,
    customAuthMethod: customAuth,
    signParams: { secretKey: secretKey },
    partSize: 1024 * 1024 * 6,
    s3FileCacheHoursAgo: 1,
    allowS3ExistenceOptimization: false,
    cloudfront: true,
  };

  var evaporate = await Evaporate.create(config);

  const addConfig = {
    name: filename,
    file: file,
    progress: function (progressVal) {
      console.log("Progress: ", progressVal);
    },
    complete: function (_xhr, awsKey) {
      console.log("Complete!");
    },
  };
  return evaporate.add(addConfig);
};
Avatar for DrewDrew

Hi Gowri! I'm glad this article helped you.

I think you need to add the x-amz-security-token header to the upload request. If I remember correctly, Cognito's security tokens are temporary and so the signed request must include the session token in the x-amz-security-token header per the Amazon docs. The session token is different from the AWS access key and secret key.

Pass the x-amz-security-token via the xAmzHeadersCommon and xAmzHeadersAtInitiate properties of addConfig, as described in the docs for evaporate.add().

I hope this fixes the 403: Forbidden errors!

Avatar for FlipFlip

Hi Drew, great post! Got everything working.

Is this Crypto JS code open source ( free to reuse under the MIT license? )

https://runkit.com/wimpyprogrammer/aws-v4-signature-with-cryptojs

Just wanted to share this with my community, happy to link back.

let me know thanks!

Avatar for DrewDrew

Hi Flip, I love hearing that my article was helpful!

I've updated all of the RunKit links with a comment to release my code into the public domain. This should allow you to use and share the code however you'd like. Credit is appreciated but not required!

Avatar for FlipListFlipList

Hi Drew,

I went ahead and built a plugin using your Crypto-JS code.

Completely open source for the no-code community at bubble.io.

https://forum.bubble.io/t/new-free-plugin-aws-signature-4/153878

With full attribution to this awesome blog!

Thanks Again.