The Power of Scripting: A Deploy Script

Feb 22, 2021 • posted by Michael Hartl

Michael Hartl here from the Rails Tutorial and Learn Enough. This post is about the power of scripting, which refers to the creation of programs (known as scripts) that are generally shorter and simpler than full-size applications while still being highly useful. The most common kind of script is a shell script, which is typically run at the command line.

The specific example we’ll be looking at is an only slightly simplified version of a script that I use all the time, often multiple times per day. It’s called deploy, and its purpose is to deploy a wide variety of different project types using a method appropriate to each type. For example, deploy can be used to deploy books, static websites, Ruby gems, and Rails apps, all using the same command.

The language we’ll be using is Ruby, an elegant object-oriented programming language that’s excellent for writing shell scripts (among many other things). Because Ruby is generally easy to read, there’s a good chance you’ll be able to follow this post even if you don’t have much (or any) experience with the language. (If you’re interested in learning Ruby, check out Learn Enough Ruby to Be Dangerous and the Ruby on Rails Tutorial.) This is a useful aspect of technical sophistication: being able to read and generally understand a computer language you don’t necessarily know.

Note: In a post of this sort, it’s impossible to anticipate the exact background of every possible reader. Many clarifying links are provided, but some googling may be required; for a systematic introduction to all the relevant topics in a carefully designed order, see the Learn Enough All Access subscription (7-day free trial, 60-day money-back guarantee).

1 A bunch of different projects

Like many developers and other tech-savvy people, I maintain several different kinds of projects:

A frequent task common to all these projects is deploying them, defined loosely as moving them from a local development machine to some remote server. Because I typically deploy things in this generalized sense multiple times a day, it can be cognitively costly to have to remember which deployment method to use. In order to lower the resulting cognitive load, some years ago I began developing a custom deploy script, which I’ve been refining ever since.

The basic idea is that, no matter which kind of project directory I’m working in, I can just type

$ deploy

and the deploy script will do the right thing for the corresponding project type. If it’s a book, deploy publishes the book; if it’s a Rails app, deploy updates the app on the live server; and so on. Let’s see how it works!

2 A Git deploy script

We’ll get started with the simplest version of deploy, a script to deploy a Git repository. For the purposes of a plain Git repo (as opposed to one that’s also a Rails app, a Ruby gem, etc.), we’ll define “deploy” as “push to a remote server like GitHub, GitLab, or Bitbucket.” As discussed in Learn Enough Git to Be Dangerous, this involves running the command git push, which pushes the changes in the repository to origin, i.e., the default place to push changes.

To make an executable script, we’ll work in the local bin directory, which should be included on the PATH (as described in Learn Enough Text Editor to Be Dangerous) so that it can be executed from anywhere:1

$ cd ~/bin             # Changes directory to bin relative to home dir ~
$ touch deploy         # Creates a new file called deploy
$ chmod +x deploy      # Changes the mode to make the script executable

The final line here arranges for us to be able to run the deploy script as a regular command-line program, like this:

$ deploy    # Doesn't do anything yet

For the script itself, we’ll use a “shebang” line to identify it as a Ruby script (Learn Enough Ruby):

#!/usr/bin/env ruby

Now we need to detect whether or not the current directory contains a Git repository. Our repository-detection method is to look for the presence of a special hidden directory called .git (read “dot git”),2 which is where Git stores repository-specific information. We can test for the presence of a .git directory using Ruby’s File.directory? method:

File.directory?(".git")    # Returns true if .git is a directory

The question mark ? at the end of directory? is Ruby’s way of indicating a boolean method, that is, a method that returns true or false.

If there is a .git directory, we’ll deploy the repository by “shelling out” to the underlying system using the system command in order to run git push. Opening ~/bin/deploy in a text editor and putting everything together gives the proto-script shown in Listing 1.

Listing 1: Deploying a Git repository. ~/bin/deploy
#!/usr/bin/env ruby

# Deploys a Git repository.
if File.directory?(".git")
  system "git push"
end

2.1 Refactoring Git deploy

We can make a small but important refinement to the script in Listing 1 by introducing an abstraction layer between the repository detection—in this case, detecting the presence of a .git directory—and the rest of our program. We can do this by defining a new method called git_repo?:

def git_repo?
  File.directory?(".git")
end

Here we’ve included a question mark ? at the end of git_repo? to follow the Ruby convention discussed in Section 2, in this case indicating that the method in question returns true if it is a Git repo and false if it isn’t.

Refactoring the code in Listing 1 slightly then gives the equivalent script shown in Listing 2.

Listing 2: A refactored Git deploy script. ~/bin/deploy
#!/usr/bin/env ruby

# Returns true for a Git repository.
def git_repo?
  File.directory?(".git")
end

# Deploys a Git repository.
if git_repo?
  system "git push"
end

Note that the refactored script in Listing 2 is actually longer than the script in Listing 1. This is often the case when adding abstraction layers, but what we lose in brevity we gain in clarity. The meaning of

if git_repo?
  system "git push"
end

is immediately understandable in a way that

if File.directory?(".git")
  system "git push"
end

is not. Moreover, the benefit of adding abstraction layers compounds quickly, as we’ll see in Section 3.2.

3 Extending the script

With the foundational work laid in Section 2, we’re now in a position to extend deploy by adding support for other kinds of deployable projects. Here’s a list of the most common projects I deploy, together with the methods of deployment:

  • Static website using Jekyll: git push to origin (typically GitHub Pages)
  • Rails or Sinatra app: git push heroku
  • Ruby gem: run standard rake release
  • Softcover book or article: softcover deploy
  • Git repo: git push

For each project type, we simply need to define an analogue of git_repo? (Listing 2) for that project. We can then auto-detect the project type of the current directory and use system to take the corresponding action.

We can actually accomplish everything we need with just three different Ruby techniques:

  • File.directory?("name"): As discussed in Section 2, this returns true if name is a directory and false otherwise.
  • File.exist?("name"): Returns true if name exists and false otherwise.3
  • Dir["*.gemspec"].any?: Combines Dir from the Ruby Standard Library FileUtils module and any? to return true if there are any files in the current directory matching the pattern "*.gemspec", i.e., ending in the string ".gemspec".

3.1 Adding Jekyll sites and Softcover books

One of the easiest ways to determine a project type is looking for a directory characteristic of that project. We saw this with the .git directory in Section 2, and it works for other project types as well.

For example, every site built using the Jekyll static-site generator (covered in Learn Enough CSS & Layout to Be Dangerous and used to build this blog) has a directory called _site where the static site is stored. This means we can add a jekyll_site? method in analogy with git_repo? from Listing 2:

def jekyll_site?
  File.directory?("_site")
end

Similarly, Softcover books have a latex_styles directory, so we can define softcover_book? like this:

def softcover_book?
  File.directory?("latex_styles")
end

This suggests an augmented deploy script as shown in Listing 3. Note that services like GitHub Pages automatically build Jekyll sites, so in this case a simple git push is all that’s needed to deploy. As a result, we can combine Jekyll sites and plain Git repos into one step using the || (“or”) operator.

Listing 3: An augmented deploy script. ~/bin/deploy
#!/usr/bin/env ruby

# Deploys a project.

# Returns true for a Softcover book.
def softcover_book?
  File.directory?("latex_styles")
end

# Returns true for a Jekyll site.
def jekyll?
  File.directory?("_site")
end

# Returns true for a Git repository.
def git_repo?
  File.directory?(".git")
end

# The main script.
if softcover_book?
  system "softcover deploy"
elsif jekyll? || git_repo?
  system "git push"
end

3.2 Completing the script

The final projects to support in our deploy script are Ruby gems and Ruby web apps (Rails4 or Sinatra5). For the first case, we’ll use the presence of the gemspec file needed by all Ruby gems, which we can test for as follows:

require "fileutils"
Dir["*.gemspec"].any?    # Returns true if any files end in ".gemspec"

The first line here is needed because, as noted briefly in Section 3, Dir is part of the FileUtils module in the Ruby Standard Library. This allows us to define a ruby_gem? boolean like this:

# Returns true for a Ruby gem.
def ruby_gem?
  Dir["*.gemspec"].any?
end

Finally, the trickiest test is for a Ruby web app. The complication is that most of the things characteristic of Ruby web apps are present in Jekyll projects and Ruby gems as well. For example, we could test for the presence of a file called Gemfile, but both Jekyll and Ruby gems have that, too. We could test for a test directory, but Ruby gems often have the same thing. But what does work is checking for both a Gemfile and (&&) a test directory but not (!) a Ruby gem:

# Returns true for a Ruby-based web app.
# Matches Rails and Sinatra apps.
def ruby_web_app?
  File.exist?("Gemfile") && File.directory?("test") && !ruby_gem?
end

Putting everything together gives the final deploy script shown in Listing 4. Making your way through this script is an excellent exercise in reading code (especially if you don’t already know Ruby!). Note that the use of abstraction layers (Section 2.1) means the main part of the script (at the bottom of the file) is readable even if you don’t understand all the implementation details.

Listing 4: The final deploy script. ~/bin/deploy
#!/usr/bin/env ruby
require "fileutils"

# Deploys a project.

# Returns true for a Softcover book.
def softcover_book?
  File.directory?("latex_styles")
end

# Returns true for a Ruby gem.
def ruby_gem?
  Dir["*.gemspec"].any?
end

# Returns true for a Ruby-based web app.
# Matches Rails and Sinatra apps.
def ruby_web_app?
  File.exist?("Gemfile") && File.directory?("test") && !ruby_gem?
end

# Returns true for a Jekyll site.
def jekyll?
  File.directory?("_site")
end

# Returns true for a Git repository.
def git_repo?
  File.directory?(".git")
end

# The main script.
if softcover_book?
  system "softcover deploy"
elsif ruby_gem?
  system "rake release"
elsif ruby_web_app?
  system "git push heroku"
elsif jekyll? || git_repo?
  system "git push"
end

With the code in Listing 4, a simple

$ deploy

is all that’s required to deploy any of the project types supported by the script.6

The deploy script in Listing 4 is already quite useful. For my personal setup, I include a few refinements and extensions, including the following:

  1. Include Rails or Sinatra apps that use RSpec for testing (which involves looking for a spec directory in addition to test).
  2. Handle Jekyll sites that are built locally as well as those that are built on GitHub Pages (Box 1).
  3. For all projects, automatically pull Git repos before pushing.
  4. Automatically push Git repos for non-Git-only projects (e.g., do a git push before deploying a Rails app with git push heroku).

To implement 3 & 4, my standard deploy script includes a ship method as shown in Listing 5, which replaces the system command from Listing 4.

Listing 5: Defining a ship method to sync Git repos automatically. ~/bin/deploy
#!/usr/bin/env ruby
require "fileutils"

# Deploys a project.
.
.
.
# Ships a product.
def ship(command)
  pull = "git pull"
  push = "git push"
  full_command = command == push ? "#{pull} && #{command}" :
                                   "#{pull} && #{push} && #{command}"
  puts   full_command
  system full_command
end

# The main script.
if softcover_book?
  ship "softcover deploy"
elsif ruby_gem?
  ship "rake release"
elsif ruby_web_app?
  ship "git push heroku"
elsif jekyll? || git_repo?
  ship "git push"
end

With the ideas in this post (and maybe the Learn Enough All Access subscription), you’ll be in a good position to add these enhancements to your own version, as well as to write and extend utility scripts of your own. Good luck!

Box 1. The power of forgetting.

The present site actually belongs to the final refinement discussed in the text: it’s a Jekyll site built locally and then uploaded in its entirety as static HTML. The corresponding step in deploy involves detecting the presence of a docs directory, which is where GitHub Pages looks for a statically generated site. If File.directory?("docs") is true, the deploy builds the site locally and then initiates the upload to GitHub Pages.

The amazing thing is that I had forgotten that I did this. I didn’t remember if news.learnenough.com was built locally or on GitHub Pages. Being able to forget such details is the entire purpose of the deploy script; rather than keeping track of the exact steps to deploy a particular project—in this case, a static site built using Jekyll—I can just type deploy and be confident that the script will take care of the details for me.

1. If you don’t have such a directory, see Section 3.3 of Learn Enough Text Editor to Be Dangerous for the details on how to set one up (including how to add it to the PATH).
2. The leading dot in .git is what makes it hidden. This is the general convention for Unix systems, including Linux and macOS.
3. We could actually use File.exist?(".git") for the .git directory instead of File.directory?(".git"), but this would yield a false positive in the rare edge case of a project with a file (but not a directory) called .git. In general, it’s a good idea to use the most specific criterion available in order to avoid such edge cases.
5. To learn the basics of Sinatra, see Chapter 10 of Learn Enough Ruby to Be Dangerous.
6. I’m so lazy that I actually made an alias for deploy using alias d='deploy' in my .zshrc file. (See Learn Enough Text Editor to Be Dangerous and “Using Z Shell on Macs with the Learn Enough Tutorials” for more.) That way I only ever have to type d to deploy a project. Brevity FTW!
MORE ARTICLES LIKE THIS:
shell-script , command-line , deploying , ruby , tutorials