Handle File Upload with AWS Lambda and Amazon Api Gateway

Have you ever had the need of using AWS Lambda to upload one or more files? For example, instead of exposing your S3 Bucket, you want to pass through an AWS Lambda, and obviously, an Amazon Api Gateway, in order to shield your Amazon S3 bucket from Public access. But how can we do that?

The Design

The following diagram represents my design. I have a React Application which is calling an Amazon Api Gateway by passing some Content-Type: multipart/form-data which includes Files and Objects. The content is forwarded to an AWS Lambda which uploads the Files into an Amazon S3 and the Metadata into a DynamoDB.

Solution Design

Front-end Code

For this specific example, I am going to use React with Typescript. The idea is to have an HTML Form which is going to collect some data and some Files and send it to my AWS Lambda.

async (doc: Document, { rejectWithValue }) => {
    try {
        const result = await fetch(
           `${rootUrl}api/documents/`, {
            method: 'POST',
            headers: {
            },
            body: prepareFormData(doc)
        });
        const message = await result.json() as string;
        return message;
    } catch (err) {
        // auth. failed
        return rejectWithValue({
            errorTitle: 'Fetch Error',
            errorMessage: (err as Error).message,
            type: 'application'
        });
    }
});

This action is simply collecting some information and send a POST request to my lambda, which is exposed on …/api/documents/. One gotcha of this code, look at the headers section. It is empty!

Why no Content-Type? You might be tempted here to pass Content-Type: multipart/form-data but if you do so, you will incur in the problem that the Browser won’t setup for you the Boundary and the Content.

For clarity, the request sent via Chrome Browser will look like this one:

Chrome request

The Multipart is generated inside the Content-Type, by Chrome. Something you should deal with if you plan to pass Content-Type directly from Javascript (Do not do it).

Last piece, is the function prepareFormData which is simply parsing a Typescript object and creating a FormData one:

export const prepareFormData = (doc: Document) : FormData => {
    const form = new FormData();
    form.append('data', JSON.stringify({
        name: doc.name, 
        description: doc.description})
    );
    for (let index = 0; index < doc.files.length; index++) {
        const file = doc.files[index];
        form.append(`files`, file, file.name);
    }
    return form;
}

And here is my interface used by the HTML Form:

export interface Document {
    name: string;
    description: string;
    files: Array<File>;
}

SAM CLI Template

To deploy my AWS Lambda I use SAM CLI. In order to ensure that API Gateway is capable of processing also multipart/form-data, I need to configure it via the SAM CLI template as following:

Globals:      
  Api:
    ...
    BinaryMediaTypes:
      - "multipart/form-data"
      - "application/json"

And then I configure my Lambda to accept POST and I grant permissions for the Amazon S3 bucket and for DynamoDB:

  postDocumentsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/post-documents.postDocumentsHandler
      Runtime: nodejs16.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: Upload Documents into S3.
      Policies:
        
        - DynamoDBCrudPolicy:
            TableName: !Ref DocsListingTable
        - S3CrudPolicy:
            BucketName: !Sub "${BucketNamePrefix}-draftbucket"    
      Environment:
        Variables:
          
          JOBS_TABLE: !Ref DocsListingTable
          DRAFT_BUCKET: !Ref DraftBucket
      Events:
        Api:
          Type: Api
          Properties:
            Path: /api/documents/
            Method: POST

Parsing the Files inside the Lambda Handler

Here is the tricky part. How do I parse the content received by the AWS Lambda? I use a project called lambda-multipart-parser which is available on Github here.

const parser = require('lambda-multipart-parser');

// parse the event body
const result = await parser.parse(event);

And the result will look like the following object:

{
    files: [
        {
            filename: 'test.pdf',
            content: <Buffer 25 50 6f 62 ... >,
            contentType: 'application/pdf',
            encoding: '7bit',
            fieldname: 'uploadFile1'
        }
    ],
    field1: 'VALUE1',
    field2: 'VALUE2',
}

Then I can use aws-sdk library to upload the content into an Amazon S3 bucket as following:

await s3
.putObject({ 
    Bucket: BUCKET, 
    Key: file.filename, 
    ACL: 'public-read', 
    Body: file.content })
.promise();

And now you have an AWS Lambda that can upload content into an Amazon S3 bucket.

Happy coding everyone.