Rails A/B testing with Split on Heroku

I recently decided to start doing some A/B testing on www.checkthesock.com. The last time I tried A/B testing I used Google website optimizer, it worked fairly well but I did not like how difficult it was to setup a test. This time I looked around for A/B testing tools for rails. I ended up finding split and so far I have been very happy with the results. The documentation is fairly good but it did take a little fiddling to get it working on Heroku.

First off add the Redis to Go add-on in Heroku, the Nano plan is free and has been sufficient for my needs so far. Install redis on the dev server. I use a mac so that was as simple as "brew install redis" and then I followed the brew instructions for getting redis to start on login.

To install the split gem add the following to the Gemfile

 gem 'split', '0.3.3', :require => 'split/dashboard'

To set the development environment up in a way that is compatible to how Redis to Go is configured on Heroku add the following to config/environments/development.rb

ENV['REDISTOGO_URL'] = 'redis://localhost:6379'

Then create an initializer for redis in config/initializers/redis.rb and put the following in that file.

uri = URI.parse(ENV['REDISTOGO_URL'])
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :username => uri.user)

To configure split to use the applications redis configuration create config/initializers/split.rb and add the following.

Split.redis = REDIS

To setup the split dashboard add the following to config/routes.rb

mount Split::Dashboard, :at => 'split'

To add authentication to the dashboard add the following to config/initializers/split.rb

Split::Dashboard.use Rack::Auth::Basic do |username, password|
  username == 'username' && password == 'password'
end

Setting up an A/B test is super easy just follow the instructions in the split README.

Resources:

• https://github.com/andrew/split
• Heroku Redis to Go documentation

Note: These instructions are for rails 3.x, the listed resources do have instructions for earlier versions of rails.

Testing to ensure code gets cleaned up.

The other day I was working on a change on www.checkthesock.com that involved getting rid of some pages. It is generally a good idea to redirect those URL's to a relevant location so that search engines and people can get to a useful location. Eventually no one uses the old URL anymore and whatever code is redirecting it is just sitting around gathering dust.

I wanted to make sure I eventually cleaned up the code I had doing the redirection. In the past I have tried several different ways of remembering to do this; try to remember, put a note somewhere, or put a reminder in my calendar. None of those have ever ended up working out. None of those methods involve anything that is part of my normal working process.

The solution I came up with was to write a test that checks to see if the code is still around at some future date. I run my tests all the time as part of my normal workflow and I never release code without fixing a broken test. Now at some future point in time the test will break and I will not forget to clean up my unused code. Seems like a potentially useful pattern. In a few months, when my tests start breaking to remind me to clean stuff up, I will discover if it is useful or just really annoying.