Developing Serverless applications is a very different way of building applications we've been building for decades now. We've gotten used to having awesome local tooling for us to essentially run our entire application on our local machine. Boy does this make development easier if you can just play around with the app as you build it without even requiring access to the Internet.
Then microservices came along and things have dramatically changed. Serverless arrived to help ease that infrastructure burden that building an application consisting of many small services adds. But it still means we need to figure out what the new dev workflow looks like in a Serverless microservice world.
So lets set some goals. By the end of this post we should understand how we can reclaim to a large degree the ability we have always had to develop our applications on our local machines but still set us up for integrating our services into a larger team.
Oh, and one small caveat. With Serverless as a development methodology still in its infancy, the ideas expressed here are my own opinion born out of experience building multiple Serverless projects solo and with teams. There will be others with other ideas about how to accomplish the same thing. It doesn't make any one right or wrong; we are all still working out the best ways to accomplish our goals.
Goals
What is it we want to accomplish in this article? Lets start with a list of items that we should hopefully have some answers for.
- The ability to execute and debug code locally in a repeatable way.
- Handling API calls to cloud vendor services locally.
- Unit testing.
With that basic outline out of the way, lets get stuck in...
Local development
Because we are building microservices, we need to get used to the idea that we should not expect to run the entire application on our development machines. Having every service running constantly on your machine just so you can open up a web browser to "play" with the application offline doesn't make much sense; especially if your application is going to consist of 10's or even 100's or 1000's of seperate services.
What we need to focus on then is getting each service running in some fashion locally so that we can easily execute the code we write; our handlers that will eventually be our Lambda functions that execute our business logic.
To that end, I have, put together a Serverless bootstrap as I like to call it. It is publicly available as a Gitlab Project and is incredibly easy to use to get started. So lets get it cloned and go through some of the details.
git clone https://gitlab.com/garethm/serverless-nodejs-template.git
Getting started
Once you have cloned the template, run rm -rf .git to get rid of the current .git folder so that it is ready for you to use for your own project. You will also notice some folders:
- src: This is where we will store our handler function code as well as our unit tests and any entities, classes or any other code we write.
- templates: This contains the base templates used by one of our plugins to generate new functions and tests.
The src/functions and src/test/functions both already contain some example files for us to look at. Lets take a quick look at the src/functions/exampleFunction.js file:
'use strict'
module.exports.exampleFunction = async (event, context) => {
return {
statusCode: 200,
body: JSON.stringify({
message: 'Success!'
})
}
}As you can see, this is a very simple function. Its only purpose is as a way for us to see that our local development environment is configured correctly. The other part of that is the test file; src/test/functions/exampleFunctionTest.js
/* eslint-env mocha */
'use strict'
// tests for exampleFunction
// Generated by serverless-mocha-plugin
const mochaPlugin = require('serverless-mocha-plugin')
const dirtyChai = require('dirty-chai')
mochaPlugin.chai.use(dirtyChai)
const expect = mochaPlugin.chai.expect
let wrapped = mochaPlugin.getWrapper('exampleFunction', '../../../src/functions/exampleFunction.js', 'exampleFunction')
describe('exampleFunction', () => {
before((done) => {
done()
})
it('implement tests here', () => {
return wrapped.run({}).then((response) => {
expect(response).to.not.be.empty()
})
})
})It is this test file that really contains the brunt of our local development functionality. What do I mean? Well, if we want to actually execute the code in our function, we run this test. On the line return wrapped.run({}).then((response) => {, the run function contains whatever event object we want, in this case nothing. But it could be an API Gateway event object, SNS, S3, SQS, or any other possible event object a Lambda function could receive.
Running code locally
By default, the example function and test are linked together, so lets just execute the code in our function to see how it all works. Make sure you have the mocha plugin installed globally (npm install -g mocha) and then run the test from the root of the service:
mocha src/test/functions/exampleFunctionTest.js
What you should see are some successful results:
exampleFunction
✓ implement tests here
1 passing (8ms)
Lets make a small edit. On the line return wrapped.run({}).then((response) => { change it to:
return wrapped.run({
body: JSON.stringify({
parameter: 'value'
})
}).then((response) => {Then lets edit the function itself at src/functions/exampleFunction.js to look like:
'use strict'
module.exports.exampleFunction = async (event, context) => {
let bodyObj = JSON.parse(event.body)
console.log(bodyObj)
return {
statusCode: 200,
body: JSON.stringify({
message: bodyObj.parameter
})
}
}When you run mocha src/test/functions/exampleFunctionTest.js again you should now see our console.log included in the response.
exampleFunction
{ parameter: 'value' }
✓ implement tests here
1 passing (9ms)
Thats it. We now have local execution working. You could use this very easily to setup mocha integration with your IDE of choice and even setup debugging so you can step through your code, line by line, inspecting variables and everything else that comes with a local debugger.
One step in the right direction! But we don't want to have to copy and paste the contents of this test file and its handler function everytime we want to create a new function!
Well, we don't have to. Because of the awesome serverless-mocha-plugin that has been responsible for making our lives easier so far, we can also use a CLI command that will create a function, a linked test file as well as an eventless entry into our serverless.yml for us. The command looks something like this:
sls create function -f functioname --handler src/functions/fileName.handlerName --path src/test/functions/ --stage local
For more details about this awesome plugin, check out the Github page.
AWS Services locally
One of the benefits of Serverless (especially in the AWS ecosystem) is that we have access to an incredible array of managed services that take away a large amount of the drudgery from building complex web applications. However, when you need to test code locally that tries to communicate to services such as S3, DynamoDB, SNS, SQS amongst others, it becomes tricky. There are many different ways people try to solve this problem but my favourite is a technique called mocking.
By making use of another NPM module, aws-sdk-mock, we can capture requests that would normally go to an AWS service and then return ... whatever we want. We can return a successful response or we can even simulate an AWS service error if we wish to test how we manage failures.
To try this out, lets go back to our src/test/functions/exampleFunctionTest.js file and edit the test section to look like this:
it('implement tests here', () => {
AWS.mock('S3', 'putObject', (params) => {
return new Promise((resolve, reject) => {
resolve({})
})
})
return wrapped.run({
body: JSON.stringify({
parameter: 'value'
})
}).then((response) => {
expect(response).to.not.be.empty()
})
})All that we have done here is add a mock in to catch any call to S3's putObject API call when we run our tests. In this case we are just responding with a success and an empty object. If we had returned with a reject(new Error('Some error here!')), our function would need to catch that error and deal with it; just like in the real world.
The aws-sdk-mock module can do with this any SDK and API in the aws-sdk module and you should go check out more details here.
With the combination of those two plugins alone, we now have ways to run our functions locally, step through them with a debugger and even simulate success and failure response from the AWS services we will more than likely be using. The next step from here is to look at how we can incorporate this service we have been building in isolation into the rest of our application as a whole.
If you have any comments to add, join us on the Serverless Forum or feel free to fork the project on Gitlab and make any merge requests to improve the bootstrap template.
