Deploying node.js apps

Here at Clock, long gone are the days of FTP uploads to put a site live. We use deployment tools to automate the process, which have the advantage of being faster, while reducing the chance of human error. I'm delving into node.js development, for which I've set up my own deployment and hosting.

Here's my situation:

  • I use my local machine for development
  • I have a remote server with SSH access for testing and production
  • I'm using git for source control
  • node and npm are installed locally and remotely

And here are my objectives:

  • Encapsulate deployment process into single git push command
  • Host multiple apps on a single server
  • Have multiple instances of each app for testing and production

Read on and I'll take you through the different aspects of my solution. It is still very much a work in progress, so if you see anything which could be improved, or have a different approach then let me know.

Git

When you initialise a git repository, it creates a directory called hooks. When certain actions are invoked with git, it looks in this directory for scripts to run. The one that is particularly useful to us here is called post-receive, which gets run when a branch is pushed to the repository – after it has received everything. This means we can set up a git repository on our server, clone it on to our local machine, do some development, push those changes to the remote and have it run whatever we place in our post-receive script.

The system will assume that your post-receive is bash. However, since we're working with node, it makes sense for everything to be in js right? To have your system run it with node, simply have #!/usr/bin/env node as the first line.

On your server, initialise an empty, bare repository like so:

The --bare argument means that no working tree (all of the files that git is tracking) is stored in the filesystem, only their internal representation in git. This ensures that the repo has to be cloned to be worked with. On your local machine clone the repo like so:

You can see I've renamed the remote origin to deploy. This is purely superficial, and simply serves to make our deployment command more verbose – we'll be typing git push deploy live as opposed to git push origin live.

To achieve multiple instances of a single app on the server, we can simply use git's branching model. Personally I have removed the default master branch and have live and dev. In Clock projects we tend to have development, testing, staging and production environments – the choice is yours.

Here is what we want our post-receive script to do:

  • Create a temporary directory
  • Checkout the branch of the project that was just pushed to the repository into the temporary directory we just created
  • Run the project's build script
  • Remove the temporary directory

I won't clutter up this post with my whole script, so here it is on github (If you see references to Jake, that's my choice of build tool and it's explained later on). I have purposefully made this script do as few jobs as possible, and defer the bulk of the work to the build script – this is because the hook has to be (as far as I'm aware) placed into the git repo manually. The build script can live in the project so be project-specific and source controlled.

It is worth noting that if I had written this in bash, it would have been much shorter. For me, this was a trade-off I was willing to make in order to keep things all in one language (avoiding one I am a novice in).

Jake

Jake is a build tool written in node. It follows the node ideology and supports asynchronous tasks. In short, it is a framework for defining tasks and dependencies between tasks, and is written on top of node.

On your server you will need to install jake globally. npm normally installs packages in your current working directory, so you will need to use the global argument: npm install jake -g.

The bare minimum we want to achieve with the build script is as follows:

  • Install dependencies
  • Create a versioned directory e.g. SiteName@0.0.1
  • Move the files from our temporary directory to the versioned directory
  • Point a symlink to the new versioned directory
  • Stop old version and start new version

The folder structure for my app directory looks like this:

There are are two main advantages to this:

  • If anything goes wrong up to the symlink part of the build, then no damage will have been done to the live site and its associated files
  • If the deployment has some unexpected effect on the site, the previous version can be brought back, simply by replacing the symlink

Once again, to avoid clutter here is my Jakefile.js in full.

Upstart

If you start node from your build script, your node instance is a child process of the build process – that's not good. It's a good idea to wrap each application in a service. Based on node hosting tutorials out there, I went with upstart to achieve this.

I found my upstart service directory to be /etc/event.d/, it might be different for you. In this directory, I place my service wrappers for all of my node apps. e.g.:

Site can then be started and stopped with start site.siteName-state and stop site.siteName-state.

Monit

What happens if your node app unexpectedly dies? What if it is a strange anomaly that could be resolved (if only temporarily) by restarting the service? Monit is a utility for monitoring and managing processes. We can use it to send a simple http request to our app. If it does not respond, or responds with any other http code than 200 OK, we can assume something went wrong and restart the service.

I use a monolithic monit config file called monitc located in /usr/local/etc/ where all of my apps are defined. Here's an example of a site in my Monit config:

The first line tells Monit to run as a daemon, and to check every 300 seconds with a 60 delay.

For each site, we tell Monit the location of the upstart wrappers, so it knows how to start and stop the services. A site can be targeted individually using the host name, e.g: monit start BenGourley-dev would start the service declared above.

You can start all the services that are defined with: monit start all. This is useful if the server is restarted and we want to bring all apps back up. I have placed this in my /etc/rc.local file to bring the apps up automatically.

Summary

So that's my process from start to finish. Just one more point – since multiple apps can't all listen on port 80, you'll need a proxy to sit on port 80 and route connections through to the appropriate node instance. For this, I use nginx.

I hope this has been in-depth enough, but not too much of a heavy read. This setup works really well for me as a single developer, and I hope it will help a few more get up and running. All of my examples in full can be found on github, where they will continue to evolve as I augment things.

References & Further reading:

Join Us

Come and work for Clock

Clock is made up of bright, hard-working and talented people and we're always on the look out for more. You can browse the current jobs below or follow us @clock for the latest vacancies.

View
Jobs