I recently migrated a few low-traffic domains from being hosted on namecheap shared hosting to running on a private node.js server on google compute engine. This was partly because I wanted to run a node.js app on one of the domains, and partly because I was tired of the poor quality of namecheap hosting. I wanted to share what ended up working with anyone else who's trying to do something similar.
I picked Google Compute Engine for hosting partially because an app I'm developing uses google cloud datastore and partly because they had a small machine type that looked like it might work. Their pricing lists a machine type, f1-micro, with 0.6 GB which costs $.0056/hour, or around $4/month.
Let's jump in. The webserver we create will be serving pages for multiple domains. Later we'll get the IP address for the server and point multiple domains at it. Let's assume we have four domains: example1.com, www.example1.com, example2.com, www.example2.com. Let's assume our server is server.js and we run it by node server.js
. Note: I've simplified the code in this post a bit from what I actually use, to remove parts that aren't relevant to this post. There's a chance there are bugs in it. Please let me know if you find anything amiss.
Let's say our directory structure for our app is:
server.js package.json static-example1/ static-example2/where the static directories are where we'll serve static files for our domains.
server.js
config. I use the express framework. We'll set it up so that HTTP redirects to HTTPS and also www.example* redirects to example*. SSL/acme/letsencrypt explained below.var express = require("express"); var http = require('http'); var https = require('https'); var app = express(); // letsencrypt verification requests will be HTTP. Let them proceed without any of the redirection/https checking below. app.use('/.well-known/acme-challenge/', express.static('static-acme-challenge/.well-known/acme-challenge')); app.use('/.well-known/acme-challenge/',function(req,res, next) { res.status(404).send('letsencrypt challenge file missing'); }); app.use(function redirects(req, res, next) { var host = req.headers.host; if ((host == 'example1.com' || host == 'example2.com') && req.secure) { // good to go next(); } else if (host == 'www.example1.com') { // redirect both to HTTPS as well as get rid of the www subdomain. res.redirect('https://example1.com' + req.url); } else if (host == 'www.example2.com') { // redirect both to HTTPS as well as get rid of the www subdomain. res.redirect('https://example2.com' + req.url); } else if (!req.secure) { res.redirect('https://' + req.headers.host + req.url); } else { should never get here.... } }); var vhost = require('vhost'); // You could change this up so that instead of serving static files you do more interesting routing. Beyond scope of this blog.. app.use(vhost('example1.com', express.static('static-example1'))); app.use(vhost('example1.com', function(req, res, next) { res.status(404).send('no static file found for example1.com'); })); app.use(vhost('example2.com', express.static('static-example2'))); app.use(vhost('example2.com', function(req, res, next) { res.status(404).send('no static file found for example2.com'); })); app.use("*",function(req,res) { this should never happen - all requests should have been caught by one of the clauses above. }); var httpServer = http.createServer(app); var httpsServer = null; if (fs.existsSync("./le-config/live/example1.com/privkey.pem")) { httpsServer = https.createServer({ key: fs.readFileSync("./le-config/live/example1.com/privkey.pem"), cert: fs.readFileSync("./le-config/live/example1.com/fullchain.pem"), ca: fs.readFileSync("./le-config/live/example1.com/chain.pem") }, app); } else { console.log('No SSL certs found. Assuming we are bootstrapping with no https'); } httpServer.listen(80); httpsServer.listen(443);
- Get code somewhere that google compute engine can find it. We put it in a git repository, with server.js in the root dir of the repository. Then it's easy to use Google cloud repositories to get it to the VM. According to this, a cloud repository is created for your project. To set up for pushing to it:
git config credential.helper gcloud.sh git remote add cloud https://source.developers.google.com/p/your-project-id/
Substitute "your-project-id" with your google project id (probably is something likemyproj-65432
. Then you can push withgit push google master
- Now we need to create a startup script that the VM instance will run when it first starts up. Here's the simplified version of the script I use. I put it in the file
startup-script.sh
, in the same directory as server.js, though it doesn't need to be.#! /bin/bash # [START startup] set -v # Talk to the metadata server to get the project id PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google") USERNAME=something # Replace with your username you use to log in to the server # Set up a 512MB swap partition. The 600 MB of ram the VM has is not quite enough to do the 'npm install' command below. fallocate -l 512m /mnt/512MiB.swap chmod 600 /mnt/512MiB.swap mkswap /mnt/512MiB.swap swapon /mnt/512MiB.swap # [START the rest] # Debian has old version of node. Get fresh one. This does an apt-get update curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - # Some necessary packages apt-get install -yq ca-certificates git nodejs build-essential supervisor libcap2-bin authbind unattended-upgrades apt-get install certbot -t jessie-backports # Create a nodeapp user. Node will run as this user. This account does not have privileges to run a shell. useradd -M -d /usr/sbin/nologin nodeapp # Users seem to be added in default gropus that include a group with sudo privileges. Remove nodeapp from those groups for agr in `groups nodeapp | cut -f 2 -d ':'`; do if [ $agr != "nodeapp" ]; then echo "Removing nodeapp from group $agr" gpasswd -d nodeapp $agr fi; done # Set default group to nodeapp /usr/sbin/usermod -g nodeapp $USERNAME # Set default permissions for new files so only nodeapp group can read them grep -q '^\s*umask ' /home/$USERNAME/.profile && sed -i 's/^\s*umask .*/umask 0026/' /home/$USERNAME/.profile || echo 'umask 0026' >> /home/$USERNAME/.profile # /opt/app will hold the git repo containing server.js mkdir /opt/app chown nodeapp:nodeapp /opt/app cd /opt/app # For authbind - let nodeapp bind to only ports 80 and 443 touch /etc/authbind/byport/80 touch /etc/authbind/byport/443 chown nodeapp /etc/authbind/byport/80 chown nodeapp /etc/authbind/byport/443 chmod 755 /etc/authbind/byport/80 chmod 755 /etc/authbind/byport/443 # Create the script that cron will run every 2 weeks to renew our SSL certs # Note it restarts the server every 2 weeks as part of this. cat >/tmp/renew-cert.sh << EOF #! /bin/bash cd /opt/app echo "`whoami`:`pwd`: Checking renewal on `date`:" certbot renew --non-interactive --email subadmin@example1.com --agree-tos --debug --test-cert --config-dir ./le-config --work-dir ./le-work --logs-dir ./le-logs --webroot --webroot-path ./static-acme-challenge sudo supervisorctl restart nodeapp EOF chown $USERNAME:nodeapp /tmp/renew-cert.sh # Run a bunch more setup not as root su - $USERNAME << EOF # Get the application source code from the Google Cloud Repository. git config --global credential.helper gcloud.sh git clone https://source.developers.google.com/p/$PROJECTID /opt/app # Create directory for the letsencrypt challenges to be served by server.js mkdir static-acme-challenge # Install app dependencies specified in package.json # --production means to skip dev dependencies npm install --production # setup the cron script mv /tmp/renew-cert.sh . chmod u+x renew-cert.sh (crontab -l && echo "01 02 2,16 * * /opt/app/renew-cert.sh >> /opt/app/le-logs/cron-renews 2>&1") | crontab - EOF # Now that the npm installation has finished, we no longer need swap, so get rid of it. swapoff -a rm /mnt/512MiB.swap # Configure supervisor to run the node app. Limit the amount of RAM node uses via the flags specified. cat >/etc/supervisor/conf.d/node-app.conf << EOF [program:nodeapp] directory=/opt/app command=authbind node --trace_gc --max_old_space_size=256 --max_semi_space_size=16 --max_executable_size=256 server.js autostart=true autorestart=true user=nodeapp environment=USER="nodeapp",NODE_ENV="production",ONGOOG="yes" stdout_logfile=syslog stderr_logfile=syslog EOF # Start the server.js. Note it does not have SSL certs yet, it is running just so certbot can do the letsencrypt challenges supervisorctl reread supervisorctl update # Give server a chance to start up sleep 5 # Make unattended apt upgrades reboot when they happen egrep -q "^Unattended-Upgrade::Automatic-Reboot" /etc/apt/apt.conf.d/50unattended-upgrades || echo 'Unattended-Upgrade::Automatic-Reboot "true";' >> /etc/apt/apt.conf.d/50unattended-upgrades egrep -q "^Unattended-Upgrade::Automatic-Reboot-Time" /etc/apt/apt.conf.d/50unattended-upgrades || echo 'Unattended-Upgrade::Automatic-Reboot-Time "02:15";' >> /etc/apt/apt.conf.d/50unattended-upgrades # Generate the SSL certificates for the domains we wish to serve. domains=('example1.com', 'www.example1.com', 'example2.com', 'www.example2.com') # Generated a string containing each domain prefixed with a '-d' extraargs=""; for i in "${domains[@]}"; do extraargs="$extraargs -d $i"; done #debugflags="--debug --test-cert" debugflags="" # Generate the SSL certs. Run as nodeapp su - $USERNAME -c "certbot certonly --non-interactive --email subadmin@example1.com --agree-tos --config-dir ./le-config --work-dir ./le-work --logs-dir ./le-logs --webroot --webroot-path ./static-acme-challenge $debugflags $extraargs" # Restart server to pick up the SSL certs supervisorctl restart # Application should now be running under supervisor # [END startup]
A few notes on the script. First, the main trick in getting node to run on the f1-micro instance is managing memory. We do this by temporarily adding a swap partition to absorb the extra memory the
npm install
uses, and by passing flags to node (they actually are v8 flags) to limit the resident memory usage of the node binary. Node/v8 does not have great tools for managing memory usage. In particular, I saw no way to limit the virtual memory size. I triedulimit
and node would not start, failing when allocating memory.Regarding SSL. I went with letsencrypt since it's free. I wanted really badly to be able to use the letsencrypt-express npm package. It promised so much convenience. I banged my head against it for a few days and could not get it to work. I dug into the source code, tried fixing bugs, and eventually gave up. The source didn't seem very maintainable and did not inspire confidence. I switched to the letsencrypt natively supported certbot, specifically with their other debian-8 mode. Worked like a charm.
letsencrypt requires that you be able to prove you control the domains you want certs for. The way I use to do this is by letting
certbot
put files on the server that the letsencrypt CA can then find. I put the challenges in thestatic-acme-challenge
directory. - Now let's set up the server. First, setup the gcloud command-line tool. This will involve creating a project and other steps detailed in the Getting Started Guide they refer to there. In this blog, let's assume your project is called myproj. Terminology: Google calls a VM running on a machine an "instance". We'll be creating an instance that will run our node server.
- Set the default region and zone. I picked a zone by running
gcloud compute machine-types list
and choosing the region listed for the machine type f1-micro (us-east1-b
when I did it). Not sure if that's necessary. - Reserve an static external IP address. Without this, the instance will not necessary show up at the same IP each time it's (re)started. Note you are charged ($0.01/hr) if the IP address is not used by a running instance, so don't leave it lying around if you don't need it.
gcloud compute addresses create myproj-static-ip
You may now set any A records to point that address. This is necessary partly so the letsencrypt can verify you own the domains when generating the cert inside of startup-script.sh. That means downtime on your websites. You can avoid that by using another mechanism to prove to letsencrypt you control the domains (e.g., adding TXT records to DNS, or acting on the letsencrypt challenges using your old server), and modifying startup-script.sh, but that's outside the scope of this tutorial.
- Create the VM instance. There are lots of options when running the create command. I create a VM based on debian because it's the default and thus I figured the least likely to have issues. I include "datastore" below because I need to access the datastore, but you can omit it. This command
gcloud compute instances create myproj-instance --machine-type=f1-micro --image-family=debian-8 --image-project=debian-cloud --scopes userinfo-email,datastore,cloud-platform --zone us-east1-b --metadata-from-file startup-script=startup-script.sh --tags https-server --address myproj-static-ip
- Now you can do things like
# Get the console output from the running VM gcloud compute instances get-serial-port-output myprof-instance # ssh into the VM. You can omit the 'nodeapp@' bit. gcloud compute ssh nodeapp@myproj-instance
- Tell google to expose your new server to the internet, but opening up the ports in the firewall
gcloud compute firewall-rules create http-and-https --allow tcp:80,tcp:443 --description "Allow http and https access to http-server" --target-tags https-server
- If you want to rip the thing down, you do:
gcloud compute firewall-rules delete http-and-https gcloud compute instances delete myproj-instance gcloud compute addresses delete myproj-static-ip
Enjoy and hope it's helpful.
No comments:
Post a Comment