Running a tiny node.js webserver hosting multiple domains with SSL on google compute engine

EDIT: 3/19/2017 - improved server security

I recently migrated a few low-traffic domains from being hosted on namecheap shared hosting to running on a private node.js server on google compute engine. This was partly because I wanted to run a node.js app on one of the domains, and partly because I was tired of the poor quality of namecheap hosting. I wanted to share what ended up working with anyone else who's trying to do something similar.

I picked Google Compute Engine for hosting partially because an app I'm developing uses google cloud datastore and partly because they had a small machine type that looked like it might work. Their pricing lists a machine type, f1-micro, with 0.6 GB which costs $.0056/hour, or around $4/month.

Let's jump in. The webserver we create will be serving pages for multiple domains. Later we'll get the IP address for the server and point multiple domains at it. Let's assume we have four domains: example1.com, www.example1.com, example2.com, www.example2.com. Let's assume our server is server.js and we run it by node server.js. Note: I've simplified the code in this post a bit from what I actually use, to remove parts that aren't relevant to this post. There's a chance there are bugs in it. Please let me know if you find anything amiss.

Let's say our directory structure for our app is:

server.js
package.json
static-example1/
static-example2/

where the static directories are where we'll serve static files for our domains.

1. server.js config. I use the express framework. We'll set it up so that HTTP redirects to HTTPS and also www.example* redirects to example*. SSL/acme/letsencrypt explained below.

   var express = require("express");
   var http = require('http');
   var https = require('https');
   var app = express();
   
   // letsencrypt verification requests will be HTTP.  Let them proceed without any of the redirection/https checking below.
   app.use('/.well-known/acme-challenge/', express.static('static-acme-challenge/.well-known/acme-challenge'));
   app.use('/.well-known/acme-challenge/',function(req,res, next) {
       res.status(404).send('letsencrypt challenge file missing');
   });
   
   app.use(function redirects(req, res, next) {
       var host = req.headers.host;
       if ((host == 'example1.com' || host == 'example2.com') && req.secure) {
           // good to go
    next();
       } else if (host == 'www.example1.com') {
           // redirect both to HTTPS as well as get rid of the www subdomain.
    res.redirect('https://example1.com' + req.url);
       } else if (host == 'www.example2.com') {
           // redirect both to HTTPS as well as get rid of the www subdomain.
    res.redirect('https://example2.com' + req.url);
       } else if (!req.secure) {
    res.redirect('https://' + req.headers.host + req.url);
       } else {
    should never get here....
       }
   });
   
   var vhost = require('vhost');
   // You could change this up so that instead of serving static files you do more interesting routing.  Beyond scope of this blog..
   app.use(vhost('example1.com', express.static('static-example1')));
   app.use(vhost('example1.com', function(req, res, next) {
       res.status(404).send('no static file found for example1.com');
   }));
   app.use(vhost('example2.com', express.static('static-example2')));
   app.use(vhost('example2.com', function(req, res, next) {
       res.status(404).send('no static file found for example2.com');
   }));
   app.use("*",function(req,res) {
      this should never happen - all requests should have been caught by one of the clauses above.
   });
   
   var httpServer = http.createServer(app);
   var httpsServer = null;
   if (fs.existsSync("./le-config/live/example1.com/privkey.pem")) {
       httpsServer = https.createServer({
           key: fs.readFileSync("./le-config/live/example1.com/privkey.pem"),
           cert: fs.readFileSync("./le-config/live/example1.com/fullchain.pem"),
           ca: fs.readFileSync("./le-config/live/example1.com/chain.pem")
       }, app);
   } else {
     console.log('No SSL certs found.  Assuming we are bootstrapping with no https');
   }
   httpServer.listen(80);
   httpsServer.listen(443);
   

2. Get code somewhere that google compute engine can find it. We put it in a git repository, with server.js in the root dir of the repository. Then it's easy to use Google cloud repositories to get it to the VM. According to this, a cloud repository is created for your project. To set up for pushing to it:

   git config credential.helper gcloud.sh
   git remote add cloud https://source.developers.google.com/p/your-project-id/
   

   Substitute "your-project-id" with your google project id (probably is something like myproj-65432. Then you can push with

   git push google master

3. Now we need to create a startup script that the VM instance will run when it first starts up. Here's the simplified version of the script I use. I put it in the file startup-script.sh, in the same directory as server.js, though it doesn't need to be.

   #! /bin/bash
   # [START startup]
   set -v
   
   # Talk to the metadata server to get the project id
   PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google")
   USERNAME=something # Replace with your username you use to log in to the server
   
   # Set up a 512MB swap partition.  The 600 MB of ram the VM has is not quite enough to do the 'npm install' command below.
   fallocate -l 512m /mnt/512MiB.swap
   chmod 600 /mnt/512MiB.swap
   mkswap /mnt/512MiB.swap
   swapon /mnt/512MiB.swap
   
   # [START the rest]
   # Debian has old version of node.  Get fresh one.  This does an apt-get update
   curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
   
   # Some necessary packages
   apt-get install -yq ca-certificates git nodejs build-essential supervisor libcap2-bin authbind unattended-upgrades
   apt-get install certbot -t jessie-backports
   
   # Create a nodeapp user. Node will run as this user.  This account does not have privileges to run a shell.
   useradd -M -d /usr/sbin/nologin nodeapp
   
   # Users seem to be added in default gropus that include a group with sudo privileges.  Remove nodeapp from those groups
   for agr in `groups nodeapp | cut -f 2 -d ':'`; do
       if [ $agr != "nodeapp" ]; then
    echo "Removing nodeapp from group $agr"
    gpasswd -d nodeapp $agr
       fi;
   done
   
   # Set default group to nodeapp
   /usr/sbin/usermod -g nodeapp $USERNAME
   # Set default permissions for new files so only nodeapp group can read them
   grep -q '^\s*umask ' /home/$USERNAME/.profile && sed -i 's/^\s*umask .*/umask 0026/' /home/$USERNAME/.profile || echo 'umask 0026' >> /home/$USERNAME/.profile
   
   # /opt/app will hold the git repo containing server.js
   mkdir /opt/app
   chown nodeapp:nodeapp /opt/app
   cd /opt/app
   
   # For authbind - let nodeapp bind to only ports 80 and 443
   touch /etc/authbind/byport/80
   touch /etc/authbind/byport/443
   chown nodeapp /etc/authbind/byport/80
   chown nodeapp /etc/authbind/byport/443
   chmod 755 /etc/authbind/byport/80
   chmod 755 /etc/authbind/byport/443
   
   
   # Create the script that cron will run every 2 weeks to renew our SSL certs
   # Note it restarts the server every 2 weeks as part of this.
   cat >/tmp/renew-cert.sh << EOF
   #! /bin/bash
   cd /opt/app
   echo "`whoami`:`pwd`: Checking renewal on `date`:"
   certbot renew --non-interactive --email subadmin@example1.com --agree-tos --debug --test-cert --config-dir ./le-config --work-dir ./le-work --logs-dir ./le-logs --webroot --webroot-path ./static-acme-challenge
   sudo supervisorctl restart nodeapp
   EOF
   chown $USERNAME:nodeapp /tmp/renew-cert.sh
   
   # Run a bunch more setup not as root
   su - $USERNAME << EOF
   # Get the application source code from the Google Cloud Repository.
   git config --global credential.helper gcloud.sh
   git clone https://source.developers.google.com/p/$PROJECTID /opt/app
   
   # Create directory for the letsencrypt challenges to be served by server.js
   mkdir static-acme-challenge
   
   # Install app dependencies specified in package.json
   # --production means to skip dev dependencies
   npm install --production
   
   # setup the cron script
   mv /tmp/renew-cert.sh .
   chmod u+x renew-cert.sh
   (crontab -l && echo "01 02 2,16 * * /opt/app/renew-cert.sh >> /opt/app/le-logs/cron-renews 2>&1") | crontab -
   EOF
   
   # Now that the npm installation has finished, we no longer need swap, so get rid of it.
   swapoff -a
   rm /mnt/512MiB.swap
   
   # Configure supervisor to run the node app.  Limit the amount of RAM node uses via the flags specified.
   cat >/etc/supervisor/conf.d/node-app.conf << EOF
   [program:nodeapp]
   directory=/opt/app
   command=authbind node --trace_gc --max_old_space_size=256 --max_semi_space_size=16 --max_executable_size=256 server.js
   autostart=true
   autorestart=true
   user=nodeapp
   environment=USER="nodeapp",NODE_ENV="production",ONGOOG="yes"
   stdout_logfile=syslog
   stderr_logfile=syslog
   EOF
   # Start the server.js.  Note it does not have SSL certs yet, it is running just so certbot can do the letsencrypt challenges
   supervisorctl reread
   supervisorctl update
   # Give server a chance to start up
   sleep 5
   
   # Make unattended apt upgrades reboot when they happen
   egrep -q "^Unattended-Upgrade::Automatic-Reboot" /etc/apt/apt.conf.d/50unattended-upgrades || echo 'Unattended-Upgrade::Automatic-Reboot "true";' >> /etc/apt/apt.conf.d/50unattended-upgrades
   egrep -q "^Unattended-Upgrade::Automatic-Reboot-Time" /etc/apt/apt.conf.d/50unattended-upgrades || echo 'Unattended-Upgrade::Automatic-Reboot-Time "02:15";' >> /etc/apt/apt.conf.d/50unattended-upgrades
   
   # Generate the SSL certificates for the domains we wish to serve.
   domains=('example1.com', 'www.example1.com',
            'example2.com', 'www.example2.com')
   # Generated a string containing each domain prefixed with a '-d'
   extraargs=""; for i in "${domains[@]}"; do extraargs="$extraargs -d $i"; done
   #debugflags="--debug --test-cert"
   debugflags=""
   # Generate the SSL certs.  Run as nodeapp
   su - $USERNAME -c "certbot certonly --non-interactive --email subadmin@example1.com --agree-tos --config-dir ./le-config --work-dir ./le-work --logs-dir ./le-logs --webroot --webroot-path ./static-acme-challenge $debugflags $extraargs"
   
   # Restart server to pick up the SSL certs
   supervisorctl restart
   
   # Application should now be running under supervisor
   # [END startup]
   

   A few notes on the script. First, the main trick in getting node to run on the f1-micro instance is managing memory. We do this by temporarily adding a swap partition to absorb the extra memory the npm install uses, and by passing flags to node (they actually are v8 flags) to limit the resident memory usage of the node binary. Node/v8 does not have great tools for managing memory usage. In particular, I saw no way to limit the virtual memory size. I tried ulimit and node would not start, failing when allocating memory.

   Regarding SSL. I went with letsencrypt since it's free. I wanted really badly to be able to use the letsencrypt-express npm package. It promised so much convenience. I banged my head against it for a few days and could not get it to work. I dug into the source code, tried fixing bugs, and eventually gave up. The source didn't seem very maintainable and did not inspire confidence. I switched to the letsencrypt natively supported certbot, specifically with their other debian-8 mode. Worked like a charm.

   letsencrypt requires that you be able to prove you control the domains you want certs for. The way I use to do this is by letting certbot put files on the server that the letsencrypt CA can then find. I put the challenges in the static-acme-challenge directory.

4. Now let's set up the server. First, setup the gcloud command-line tool. This will involve creating a project and other steps detailed in the Getting Started Guide they refer to there. In this blog, let's assume your project is called myproj. Terminology: Google calls a VM running on a machine an "instance". We'll be creating an instance that will run our node server.
5. Set the default region and zone. I picked a zone by running gcloud compute machine-types list and choosing the region listed for the machine type f1-micro (us-east1-b when I did it). Not sure if that's necessary.
6. Reserve an static external IP address. Without this, the instance will not necessary show up at the same IP each time it's (re)started. Note you are charged ($0.01/hr) if the IP address is not used by a running instance, so don't leave it lying around if you don't need it.

   gcloud compute addresses create myproj-static-ip

   You may now set any A records to point that address. This is necessary partly so the letsencrypt can verify you own the domains when generating the cert inside of startup-script.sh. That means downtime on your websites. You can avoid that by using another mechanism to prove to letsencrypt you control the domains (e.g., adding TXT records to DNS, or acting on the letsencrypt challenges using your old server), and modifying startup-script.sh, but that's outside the scope of this tutorial.

7. Create the VM instance. There are lots of options when running the create command. I create a VM based on debian because it's the default and thus I figured the least likely to have issues. I include "datastore" below because I need to access the datastore, but you can omit it. This command

   gcloud compute instances create myproj-instance --machine-type=f1-micro --image-family=debian-8 --image-project=debian-cloud --scopes userinfo-email,datastore,cloud-platform --zone us-east1-b --metadata-from-file startup-script=startup-script.sh --tags https-server --address myproj-static-ip

8. Now you can do things like

   # Get the console output from the running VM
   gcloud compute instances get-serial-port-output myprof-instance
   
   # ssh into the VM.  You can omit the 'nodeapp@' bit.
   gcloud compute ssh nodeapp@myproj-instance
   

9. Tell google to expose your new server to the internet, but opening up the ports in the firewall

   gcloud compute firewall-rules create http-and-https --allow tcp:80,tcp:443 --description "Allow http and 
   https access to http-server" --target-tags https-server
   

10. If you want to rip the thing down, you do:

    gcloud compute firewall-rules delete http-and-https
    gcloud compute instances delete myproj-instance
    gcloud compute addresses delete myproj-static-ip
    

Enjoy and hope it's helpful.