Read File from S3 using Lambda

S3 can store any types of objects / files and it may be necessary to access and read the files programatically. AWS supports a number of languages including NodeJS, C#, Java, Python and many more that can be used to access and read file. The solution can be hosted on an EC2 instance or in a lambda function.

To read a file from a S3 bucket, the bucket name, object name needs to be known and the role associated with EC2 or lambda needs to have read permission to the bucket.

A role needs to be setup using lambda and then adding S3 for the role. There are multiple options for S3 - readonly or full access - and for this demo, readonly is adequate since the lambda will not write file to the bucket.

To write the code, the aws-sdk npm package must be used. The bucket name and the keyname needs to be known before the file can be retrieved and read. The s3.getObject function needs to be used to read the file from the bucket.


        var AWS = require('aws-sdk');
        var s3 = new AWS.S3();

        exports.handler = (event, context, callback) => {  
            var bucketName = process.env.bucketName;
            var keyName = event.Records[0].s3.object.key;
        
            readFile(bucketName, keyName, readFileContent, onError);
        };
        
        function readFile (bucketName, filename, onFileContent, onError) {
            var params = { Bucket: bucketName, Key: filename };
            s3.getObject(params, function (err, data) {
                if (!err) 
                    onFileContent(filename, data.Body.toString());
                else
                    console.log(err);
            });
        }
        
        function readFileContent(filename, content) {
            //do something with the content of the file
        }
        
        function onError (err) {
            console.log('error: ' + err);
        }            
            
    

Once the role has been setup, create the lambda function and then deploy the code. Then set environment variables for the bucket name and then the function can be used.

Note that for the above lambda function, the trigger is set to an object change (added, modified) for the bucket and thus the key name for the object is retrieved using the lambda event with the code snippet - event.Records[0].s3.object.key

NodeJS in Lambda can also be used to write or save contents to S3.