The Power of Scripting: A Deploy Script
Feb 22, 2021 • posted by Michael Hartl
Michael Hartl here from the Rails Tutorial and Learn Enough. This post is about the power of scripting, which refers to the creation of programs (known as scripts) that are generally shorter and simpler than full-size applications while still being highly useful. The most common kind of script is a shell script, which is typically run at the command line.
The specific example we’ll be looking at is an only slightly simplified version of a script that I use all the time, often multiple times per day. It’s called deploy
, and its purpose is to deploy a wide variety of different project types using a method appropriate to each type. For example, deploy
can be used to deploy books, static websites, Ruby gems, and Rails apps, all using the same command.
The language we’ll be using is Ruby, an elegant object-oriented programming language that’s excellent for writing shell scripts (among many other things). Because Ruby is generally easy to read, there’s a good chance you’ll be able to follow this post even if you don’t have much (or any) experience with the language. (If you’re interested in learning Ruby, check out Learn Enough Ruby to Be Dangerous and the Ruby on Rails Tutorial.) This is a useful aspect of technical sophistication: being able to read and generally understand a computer language you don’t necessarily know.
Note: In a post of this sort, it’s impossible to anticipate the exact background of every possible reader. Many clarifying links are provided, but some googling may be required; for a systematic introduction to all the relevant topics in a carefully designed order, see the Learn Enough All Access subscription (7-day free trial, 60-day money-back guarantee).
1 A bunch of different projects
Like many developers and other tech-savvy people, I maintain several different kinds of projects:
- Static websites (news.learnenough.com (i.e., this site), michaelhartl.com)
- Dynamic web apps (learnenough.com, railstutorial.org, tauday.com)
- Ruby gems (softcover, polytexnic, jekyll-latex, git-utils)
- Books, essays, and articles (Learn Enough Command Line to Be Dangerous, Learn Enough Text Editor to Be Dangerous, et al.; the Ruby on Rails Tutorial; The Tau Manifesto)
- Various directories versioned as Git repositories (
txt
directory, personalbin
directory, etc.)
A frequent task common to all these projects is deploying them, defined loosely as moving them from a local development machine to some remote server. Because I typically deploy things in this generalized sense multiple times a day, it can be cognitively costly to have to remember which deployment method to use. In order to lower the resulting cognitive load, some years ago I began developing a custom deploy
script, which I’ve been refining ever since.
The basic idea is that, no matter which kind of project directory I’m working in, I can just type
$ deploy
and the deploy script will do the right thing for the corresponding project type. If it’s a book, deploy
publishes the book; if it’s a Rails app, deploy
updates the app on the live server; and so on. Let’s see how it works!
2 A Git deploy script
We’ll get started with the simplest version of deploy
, a script to deploy a Git repository. For the purposes of a plain Git repo (as opposed to one that’s also a Rails app, a Ruby gem, etc.), we’ll define “deploy” as “push to a remote server like GitHub, GitLab, or Bitbucket.” As discussed in Learn Enough Git to Be Dangerous, this involves running the command git push
, which pushes the changes in the repository to origin
, i.e., the default place to push changes.
To make an executable script, we’ll work in the local bin
directory, which should be included on the PATH
(as described in Learn Enough Text Editor to Be Dangerous) so that it can be executed from anywhere:1
$ cd ~/bin # Changes directory to bin relative to home dir ~
$ touch deploy # Creates a new file called deploy
$ chmod +x deploy # Changes the mode to make the script executable
The final line here arranges for us to be able to run the deploy script as a regular command-line program, like this:
$ deploy # Doesn't do anything yet
For the script itself, we’ll use a “shebang” line to identify it as a Ruby script (Learn Enough Ruby):
#!/usr/bin/env ruby
Now we need to detect whether or not the current directory contains a Git repository. Our repository-detection method is to look for the presence of a special hidden directory called .git
(read “dot git”),2 which is where Git stores repository-specific information. We can test for the presence of a .git
directory using Ruby’s File.directory?
method:
File.directory?(".git") # Returns true if .git is a directory
The question mark ?
at the end of directory?
is Ruby’s way of indicating a boolean method, that is, a method that returns true
or false
.
If there is a .git
directory, we’ll deploy the repository by “shelling out” to the underlying system using the system
command in order to run git push
. Opening ~/bin/deploy
in a text editor and putting everything together gives the proto-script shown in Listing 1.
~/bin/deploy
#!/usr/bin/env ruby
# Deploys a Git repository.
if File.directory?(".git")
system "git push"
end
2.1 Refactoring Git deploy
We can make a small but important refinement to the script in Listing 1 by introducing an abstraction layer between the repository detection—in this case, detecting the presence of a .git
directory—and the rest of our program. We can do this by defining a new method called git_repo?
:
def git_repo?
File.directory?(".git")
end
Here we’ve included a question mark ?
at the end of git_repo?
to follow the Ruby convention discussed in Section 2, in this case indicating that the method in question returns true
if it is a Git repo and false
if it isn’t.
Refactoring the code in Listing 1 slightly then gives the equivalent script shown in Listing 2.
~/bin/deploy
#!/usr/bin/env ruby
# Returns true for a Git repository.
def git_repo?
File.directory?(".git")
end
# Deploys a Git repository.
if git_repo?
system "git push"
end
Note that the refactored script in Listing 2 is actually longer than the script in Listing 1. This is often the case when adding abstraction layers, but what we lose in brevity we gain in clarity. The meaning of
if git_repo?
system "git push"
end
is immediately understandable in a way that
if File.directory?(".git")
system "git push"
end
is not. Moreover, the benefit of adding abstraction layers compounds quickly, as we’ll see in Section 3.2.
3 Extending the script
With the foundational work laid in Section 2, we’re now in a position to extend deploy
by adding support for other kinds of deployable projects. Here’s a list of the most common projects I deploy, together with the methods of deployment:
- Static website using Jekyll:
git push
toorigin
(typically GitHub Pages) - Rails or Sinatra app:
git push heroku
- Ruby gem: run standard
rake release
- Softcover book or article:
softcover deploy
- Git repo:
git push
For each project type, we simply need to define an analogue of git_repo?
(Listing 2) for that project. We can then auto-detect the project type of the current directory and use system
to take the corresponding action.
We can actually accomplish everything we need with just three different Ruby techniques:
File.directory?("name")
: As discussed in Section 2, this returnstrue
ifname
is a directory andfalse
otherwise.File.exist?("name")
: Returnstrue
ifname
exists andfalse
otherwise.3Dir["*.gemspec"].any?
: CombinesDir
from the Ruby Standard LibraryFileUtils
module andany?
to returntrue
if there are any files in the current directory matching the pattern"*.gemspec"
, i.e., ending in the string".gemspec"
.
3.1 Adding Jekyll sites and Softcover books
One of the easiest ways to determine a project type is looking for a directory characteristic of that project. We saw this with the .git
directory in Section 2, and it works for other project types as well.
For example, every site built using the Jekyll static-site generator (covered in Learn Enough CSS & Layout to Be Dangerous and used to build this blog) has a directory called _site
where the static site is stored. This means we can add a jekyll_site?
method in analogy with git_repo?
from Listing 2:
def jekyll_site?
File.directory?("_site")
end
Similarly, Softcover books have a latex_styles
directory, so we can define softcover_book?
like this:
def softcover_book?
File.directory?("latex_styles")
end
This suggests an augmented deploy
script as shown in Listing 3. Note that services like GitHub Pages automatically build Jekyll sites, so in this case a simple git push
is all that’s needed to deploy. As a result, we can combine Jekyll sites and plain Git repos into one step using the ||
(“or”) operator.
~/bin/deploy
#!/usr/bin/env ruby
# Deploys a project.
# Returns true for a Softcover book.
def softcover_book?
File.directory?("latex_styles")
end
# Returns true for a Jekyll site.
def jekyll?
File.directory?("_site")
end
# Returns true for a Git repository.
def git_repo?
File.directory?(".git")
end
# The main script.
if softcover_book?
system "softcover deploy"
elsif jekyll? || git_repo?
system "git push"
end
3.2 Completing the script
The final projects to support in our deploy
script are Ruby gems and Ruby web apps (Rails4 or Sinatra5). For the first case, we’ll use the presence of the gemspec
file needed by all Ruby gems, which we can test for as follows:
require "fileutils"
Dir["*.gemspec"].any? # Returns true if any files end in ".gemspec"
The first line here is needed because, as noted briefly in Section 3, Dir
is part of the FileUtils
module in the Ruby Standard Library. This allows us to define a ruby_gem?
boolean like this:
# Returns true for a Ruby gem.
def ruby_gem?
Dir["*.gemspec"].any?
end
Finally, the trickiest test is for a Ruby web app. The complication is that most of the things characteristic of Ruby web apps are present in Jekyll projects and Ruby gems as well. For example, we could test for the presence of a file called Gemfile
, but both Jekyll and Ruby gems have that, too. We could test for a test
directory, but Ruby gems often have the same thing. But what does work is checking for both a Gemfile
and (&&
) a test
directory but not (!
) a Ruby gem:
# Returns true for a Ruby-based web app.
# Matches Rails and Sinatra apps.
def ruby_web_app?
File.exist?("Gemfile") && File.directory?("test") && !ruby_gem?
end
Putting everything together gives the final deploy
script shown in Listing 4. Making your way through this script is an excellent exercise in reading code (especially if you don’t already know Ruby!). Note that the use of abstraction layers (Section 2.1) means the main part of the script (at the bottom of the file) is readable even if you don’t understand all the implementation details.
~/bin/deploy
#!/usr/bin/env ruby
require "fileutils"
# Deploys a project.
# Returns true for a Softcover book.
def softcover_book?
File.directory?("latex_styles")
end
# Returns true for a Ruby gem.
def ruby_gem?
Dir["*.gemspec"].any?
end
# Returns true for a Ruby-based web app.
# Matches Rails and Sinatra apps.
def ruby_web_app?
File.exist?("Gemfile") && File.directory?("test") && !ruby_gem?
end
# Returns true for a Jekyll site.
def jekyll?
File.directory?("_site")
end
# Returns true for a Git repository.
def git_repo?
File.directory?(".git")
end
# The main script.
if softcover_book?
system "softcover deploy"
elsif ruby_gem?
system "rake release"
elsif ruby_web_app?
system "git push heroku"
elsif jekyll? || git_repo?
system "git push"
end
With the code in Listing 4, a simple
$ deploy
is all that’s required to deploy any of the project types supported by the script.6
The deploy
script in Listing 4 is already quite useful. For my personal setup, I include a few refinements and extensions, including the following:
- Include Rails or Sinatra apps that use RSpec for testing (which involves looking for a
spec
directory in addition totest
). - Handle Jekyll sites that are built locally as well as those that are built on GitHub Pages (Box 1).
- For all projects, automatically pull Git repos before pushing.
- Automatically push Git repos for non-Git-only projects (e.g., do a
git push
before deploying a Rails app withgit push heroku
).
To implement 3 & 4, my standard deploy
script includes a ship
method as shown in Listing 5, which replaces the system
command from Listing 4.
ship
method to sync Git repos automatically. ~/bin/deploy
#!/usr/bin/env ruby
require "fileutils"
# Deploys a project.
.
.
.
# Ships a product.
def ship(command)
pull = "git pull"
push = "git push"
full_command = command == push ? "#{pull} && #{command}" :
"#{pull} && #{push} && #{command}"
puts full_command
system full_command
end
# The main script.
if softcover_book?
ship "softcover deploy"
elsif ruby_gem?
ship "rake release"
elsif ruby_web_app?
ship "git push heroku"
elsif jekyll? || git_repo?
ship "git push"
end
With the ideas in this post (and maybe the Learn Enough All Access subscription), you’ll be in a good position to add these enhancements to your own version, as well as to write and extend utility scripts of your own. Good luck!
The present site actually belongs to the final refinement discussed in the text: it’s a Jekyll site built locally and then uploaded in its entirety as static HTML. The corresponding step in deploy
involves detecting the presence of a docs
directory, which is where GitHub Pages looks for a statically generated site. If File.directory?("docs")
is true, the deploy
builds the site locally and then initiates the upload to GitHub Pages.
The amazing thing is that I had forgotten that I did this. I didn’t remember if news.learnenough.com was built locally or on GitHub Pages. Being able to forget such details is the entire purpose of the deploy
script; rather than keeping track of the exact steps to deploy a particular project—in this case, a static site built using Jekyll—I can just type deploy
and be confident that the script will take care of the details for me.
PATH
)..git
is what makes it hidden. This is the general convention for Unix systems, including Linux and macOS.File.exist?(".git")
for the .git
directory instead of File.directory?(".git")
, but this would yield a false positive in the rare edge case of a project with a file (but not a directory) called .git
. In general, it’s a good idea to use the most specific criterion available in order to avoid such edge cases.deploy
using alias d='deploy'
in my .zshrc
file. (See Learn Enough Text Editor to Be Dangerous and “Using Z Shell on Macs with the Learn Enough Tutorials” for more.) That way I only ever have to type d
to deploy a project. Brevity FTW!