Developing and deploying applications with Fabric and Subversion.

Twenty-four hours ago, I was deploying applications from development to production environments in about 5-6 steps. Today, tomorrow and every day in the future, I'm doing it in 1 step, with Fabric.

Fabric is:

It is awesome. But don't let me tell you, let me show you.

We have lots of projects floating all over the place, all neat and tidy in our Subversion repository. When I'm ready to start building a new feature or fix a bug, I like to have a copy of the production database for that application on my local machine for development. Most of the time, these applications are Drupal or Django based. When I'm ready to start building, I do something like this:

cd ~/Documents/Scripts
bash update-project.sh
cd ~/Code/project

The shell script I run essentially does this:

• log into the remote production server
• take a snapshot of the production database
• save the dumpfile
• log out of the server
• transfer the file from the production server to my workstation
• remove the existing development database from my local MySQL installation
• create a new database to contain the production database
• import the production database into the new database

It's nice that the script takes care of all of that for me — but, you see, there are 10-20 of those files (one for each project). This becomes an enormous headache when we change servers or add a new feature to the script. We needed a better solution.

Enter Fabric. I got it up and running in about 5 minutes, and began to migrate that shell script into one 'fabfile'. However, since I needed to use exactly the same functions for every project, we needed to share the functions with all projects. Fabric makes this easy.

In each project root, there is a file named 'fabfile.py', with the following contents:

# Load our global function definitions.
load('/path/to/fabric_global.py')
 
# Set project server settings.
config.fab_hosts = ['production.server.com']
set(
    fab_user = 'sshuser',
    fab_host = config.fab_hosts[0],                         # Shouldn't have to do this, but $(fab_host) doesn't appear to work.
    prod_dir = '/var/www/vhosts/project.com/httpdocs'       # Directory that the production codebase exists in.
)
 
# Set production server MySQL credentials.
set(
    prod_mysql_host = 'localhost',      # This will usually be localhost (beacuse it's being run on the server itself)
    prod_mysql_user = 'dbuser',
    prod_mysql_pass = 'dbpass',
    prod_mysql_db   = 'dbname'
)
 
# Set local server MySQL credentials.
set(
    local_mysql_host = 'localhost',
    local_mysql_user = 'locdbuser',
    local_mysql_pass = 'locdbpass',
    local_mysql_db   = 'locdbname'
)

Important note: The reason I use config.fab_hosts instead of 'set(fab_hosts = ['...'])' is because I've built my Fabric installation from the git master branch. If you've downloaded the 0.0.9 package, use:

set(fab_hosts = ['production.server.com'])

The fabfile sets some basic variables for the server we're connecting to, the user we connect with, and MySQL credentials. The very first thing the file does is import a file named 'fabric_global.py', which contains the following:

def update():
    "Update development MySQL database with production clone."
 
 
    require(
        'prod_mysql_host', 'prod_mysql_user', 'prod_mysql_pass', 'prod_mysql_db',
        'local_mysql_host', 'local_mysql_user', 'local_mysql_pass', 'local_mysql_db'
    )
 
    # Run the database dump on the server and store as file.
    run("mysqldump --user=$(prod_mysql_user) --password=$(prod_mysql_pass) --host $(prod_mysql_host) $(prod_mysql_db) > $(prod_mysql_db).dump")
 
    # Download the database dump from the server and store as file.  Will be stored as $(prod_mysql_db).dump.$(fab_host).
    download('$(prod_mysql_db).dump', '$(prod_mysql_db).dump')
 
    # On our local workstation, create a fresh instance of the database and import the production dump into it.
    local("""
        `mysql -u$(local_mysql_user) -p$(local_mysql_pass) -h$(local_mysql_host) << EOF
        DROP DATABASE IF EXISTS $(local_mysql_db);
        CREATE DATABASE $(local_mysql_db);
        exit`
 
        `mysql -u$(local_mysql_user) -p$(local_mysql_pass) -h$(local_mysql_host) $(local_mysql_db) < $(prod_mysql_db).dump.$(fab_host)`
    """)
 
def deploy(**kwargs):
    "Deploy code changes to the production server."
 
    # If a message is provided (fab deploy:m='a message'), run an SVN commit, first.
    if 'm' in kwargs:
        set(message = kwargs['m'])
        local("svn ci -m '$(message)'")
    else:
        local("echo 'WARNING: running deploy without SVN commit.\n##########'")
    run("svn update $(prod_dir)")

At first glance, it may look a little confusing, but this code is something like 10% of what it would be if it were duplicated for each project (and with all the additional commands).

The 'fabric_global.py' file defines two functions that we can run on a codebase. One is for updating the development database with the production database, and the other is for simply committing file changes and updating them on the server.

Now, when I need to grab the production database at the beginning a project, I simply do this:

fab update

Well crap, that's a lot easier.

When it's time to deploy my code changes, I simply do:

fab deploy:m='my subversion message'

If I had already committed my changes and simply wanted to update them on the production server, I would just do:

fab deploy

Quite amazing, if you ask me.

Also, Fabric can do a whole lot more than what I just demonstrated, so checkout the docs.

I'd like to thank the Fabric team for probably preserving a few years of my lifespan. Also, it should be noted that the current version of Fabric is 0.0.9, which should give you an idea of the amount of awesomeness to eventually come to newer releases.