I recently decided to start doing some A/B testing on www.checkthesock.com. The last time I tried A/B testing I used Google website optimizer, it worked fairly well but I did not like how difficult it was to setup a test. This time I looked around for A/B testing tools for rails. I ended up finding split and so far I have been very happy with the results. The documentation is fairly good but it did take a little fiddling to get it working on Heroku.
First off add the Redis to Go add-on in Heroku, the Nano plan is free and has been sufficient for my needs so far. Install redis on the dev server. I use a mac so that was as simple as "brew install redis" and then I followed the brew instructions for getting redis to start on login.
To install the split gem add the following to the Gemfile
gem 'split', '0.3.3', :require => 'split/dashboard'
To set the development environment up in a way that is compatible to how Redis to Go is configured on Heroku add the following to config/environments/development.rb
ENV['REDISTOGO_URL'] = 'redis://localhost:6379'Then create an initializer for redis in config/initializers/redis.rb and put the following in that file.
uri = URI.parse(ENV['REDISTOGO_URL']) REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :username => uri.user)To configure split to use the applications redis configuration create config/initializers/split.rb and add the following.
Split.redis = REDIS
To setup the split dashboard add the following to config/routes.rb
mount Split::Dashboard, :at => 'split'To add authentication to the dashboard add the following to config/initializers/split.rb
Split::Dashboard.use Rack::Auth::Basic do |username, password| username == 'username' && password == 'password' end
Setting up an A/B test is super easy just follow the instructions in the split README.
Note: These instructions are for rails 3.x, the listed resources do have instructions for earlier versions of rails.