Creating your own Lambda function using Node.js

You can do all kinds of cool things with AWS Lambda, without having to worry about running your own servers. This is particularly useful for simple modular tasks like database backups, but it also has unlimited potential to change how we create and scale applications.

If all you want to do is extend or customise native AWS services, then you can go directly to the AWS console and start writing code in the Lambda Function configuration console. This is because, by default, Amazon includes the AWS SDK libraries you’ll need, so you do not need to add any custom libraries to interact with AWS services.

However, the more innovative uses of Lambda will require third-party libraries to upload with your code. In order to do this, you will have to make sure your code works on the machine image you wish to run your Lambda function on. This can be an issue if you have dependencies that need to compile locally on the target OS. I struggled a bit with random errors the first time I used Lambda, so I hope that this post helps others avoid the same frustration with what is otherwise a fantastic tool.

Here are the steps required to create your own custom Lambda function using Node.js. I’m assuming that you’ve already written and tested your Node.js code locally and are now ready to upload it to AWS. If you haven’t done this yet, go write your code and come back when it’s ready 😉

Pick a Region

AWS Lambda runs in different regions. Each region runs Lambda functions in a different Amazon Machine Image (AMI). The relevance of this will become obvious below, but for now just pick the region you want to run your Function in. It’s beyond the scope of this post to go into the various regional differences.

Find the AMI

Once you’ve picked a region for your Function, you will need to determine the AMI within which Lambda functions run in that region. This can be found in the list of Amazon Execution Environments. Make a note of the AMI ID – in my case it was ami-f0091d91 in the US West (Oregon) region. As I mentioned above, you need to compile any native libraries your code uses on the same AMI that your Lambda Function will run on to guarantee that your code works as expected.

Install Node.js on the AMI

In order to compile your code, you’ll need to fire up an EC2 instance of the AMI you will run your Lambda Function on and install Node.js on it. Go to the AWS Console and launch a new EC2 instance. You’ll be prompted to choose an AMI. Go to the Community AMI’s tab and in the search bar enter the AMI ID you noted in the previous step. Launch the instance and connect to it via SSH.

Amazon Linux AMI’s do not come with npm or Node.js pre-installed, nor are they available in the Amazon library, so you’re going to have to install Node.js yourself. Don’t worry though — it’s not too difficult and for the purpose of compiling code, you don’t need an enterprise grade install. So… once you’ve SSH’d into your EC2 instance, enter the following commands;

sudo yum groupinstall 'Development Tools'

This installs some dev tools that you may or may not need. If you are compiling native libraries that do things like use buffers, they’ll be written in C and you’ll need these tools. Feel free to skip this step if you’re sure you don’t require them

wget https://nodejs.org/dist/v4.3.2/node-v4.3.2-linux-x64.tar.gz

Here you’re downloading Node.js v4.3.2. As of writing, this is the version of Node being used to run Lambda Functions, but check the Lambda Execution Environments to be sure you’ve got the right version and distribution of Node.js. This may take a couple of minutes to download.

PATH=$PATH:~/node-v4.3.2-linux-x64/bin/

Add Node and npm to your PATH variable. This will save you some time when you install libraries. Your path may be different to the above, make sure you’re pointing to the right place.

So now you’ve got Node.js and npm on the EC2 instance you’re using, you need to get your code onto the same instance. Given that you want to compile your code on the same instance it will run on, your best bet is to recreate your project using npm init and npm install commands, and copying and pasting your actual Lambda Function code into a new file in that project so that your libraries are installed and built on the instance.

Upload and test your code

For my database backup script I created a directory, initialised a new project, installed the libraries I needed and ran the function on the EC2 instance to check that it all worked. My example code is below

mkdir lambdaproject

cd lambdaproject

npm init

This set up a new project in a directory called lambdaproject. I then installed the one library my function needed and created a file called index.js.

npm install mongo-backup-to-s3

vi index.js

Into index.js I pasted the code below, and then ran my code to test it.

exports.handler = function (event, context) {
  var mongoBackup = require('mongo-backup-to-s3');
  var config = {
    mongodb:{
    url: 'mongodb://user:password@mongoURL'
    },
  s3:{
    bucket:'bucket_name',
    folder:'folder_name',
    key: 'key',
    secret: 'secret'
    }
  };
  mongoBackup.dumpToS3(config,function(){context.done();});
}

Note that the Node.js code is exported in an exports.handler call. This is required by Lambda to run your function.

Once you have verified that the code works as expected, you need to zip up the code file and node_modules directory and upload the archive into your Lambda Function. There are a number of ways to do this, the easiest is to just zip up the files on the EC2 instance using the zip command and SFTP them back to your local machine. Just make sure that you zip only your code and node_modules, not the directory in which they reside, or else Lambda won’t be able to find them at runtime.

zip archive.zip index.js node_modules

Create and test the Lambda Function

At this point you’ve done all the difficult work. You just need to create the actual Lambda Function and upload the archive you’ve created in the Lambda Management Console. I’ll try to cover that in a future post.

Good luck! Feel free to post any questions in the comments section below.

RELATED LINKS

Creating a virtual service desk agent – a DevOps experiment

The CSC MyWorkStyle DevOps journey

Why Agile and DevOps fit like a hand in a glove

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: