Jade's weblog

Occasionally, however, the test is hard to fix, or the person is unavailable to fix it. Then things start piling up and it's difficult to tell when something broke.

Also, sometime you get a regression and write a new test that duplicates the failed test, and you want to find out when it initially broke.

The usual tool for finding when something broke it using git bisect. It does a binary search and finds the exact commit that broke something. And, it even can use the output of a script to automatically determine the exact commit, based on the result of a test script.

However, a big problem in Rails is that your environment gets out of sync with the codebase pretty quickly. You have to run bundle each time git bisect updates, and then migrate your database to keep it in sync. It can be quite a pain to do it manually, and these things tend to prevent you from running the automatic scripts.

I hacked together a very rough script (based loosely on this one: http://pivotallabs.com/users/jnoble/blog/articles/1678-using-git-bisect-to-find-where-something-broke) that automates everything. It's probably slower than if you did it manually, but it IS automatic, so you can go get a cup of coffee and come back with the answer to where it broke.

WARNING: This will blow away your database, AND any uncommitted work!

Here's a very early version of these scripts (I'll probably modify them as I use them more):

script/git-bisect.sh


#!/bin/bash
set -e

if [ -z "$1" ]; then
echo This script automates the process of finding what broke a particular test
echo To use it, go to the base directory of the core site project, and type:
echo
echo $0 path_to_ruby_test.rb
echo
echo note the ruby test script should not be saved in git
echo The script will use git bisect to find the initial commit that broke the test
exit
fi

echo We assume it is currently broken, and that it was okay a month ago

WEEKS_AGO=`ruby -rdate -e '(Date.today - ARGV.first.to_i).display' 31`

git bisect start
git bisect bad
git checkout `git rev-list -n 1 --before="$WEEKS_AGO" master`
git bisect good

git bisect run script/git-bisect-run-one.sh $1


script/git-bisect-run-one.sh


#!/bin/bash
set -e

if [ -z "$1" ]; then
echo usage: use git-bisect.sh, which calls this
exit
fi

git checkout .
bundle
rake db:drop
rake db:create
rake db:migrate
rake db:test:prepare
ruby -Itest $1