Rails A/B testing with Split on Heroku

I recently decided to start doing some A/B testing on www.checkthesock.com. The last time I tried A/B testing I used Google website optimizer, it worked fairly well but I did not like how difficult it was to setup a test. This time I looked around for A/B testing tools for rails. I ended up finding split and so far I have been very happy with the results. The documentation is fairly good but it did take a little fiddling to get it working on Heroku.

First off add the Redis to Go add-on in Heroku, the Nano plan is free and has been sufficient for my needs so far. Install redis on the dev server. I use a mac so that was as simple as "brew install redis" and then I followed the brew instructions for getting redis to start on login.

To install the split gem add the following to the Gemfile

 gem 'split', '0.3.3', :require => 'split/dashboard'

To set the development environment up in a way that is compatible to how Redis to Go is configured on Heroku add the following to config/environments/development.rb

ENV['REDISTOGO_URL'] = 'redis://localhost:6379'

Then create an initializer for redis in config/initializers/redis.rb and put the following in that file.

uri = URI.parse(ENV['REDISTOGO_URL'])
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :username => uri.user)

To configure split to use the applications redis configuration create config/initializers/split.rb and add the following.

Split.redis = REDIS

To setup the split dashboard add the following to config/routes.rb

mount Split::Dashboard, :at => 'split'

To add authentication to the dashboard add the following to config/initializers/split.rb

Split::Dashboard.use Rack::Auth::Basic do |username, password|
  username == 'username' && password == 'password'
end

Setting up an A/B test is super easy just follow the instructions in the split README.

Resources:

• https://github.com/andrew/split
• Heroku Redis to Go documentation

Note: These instructions are for rails 3.x, the listed resources do have instructions for earlier versions of rails.