Monday, March 15, 2010

Rails 3: Code coverage with rcov

UPDATE: Added support for ruby 1.9 and rails 3.1

I use rcov to make sure I have 100% test coverage on my rails projects. In earlier versions of rails I used the rails_rcov plugin to accomplish this task. Since rails 3 allows rake tasks to be imported by gems, I decided to write a gem with a rake task that reports on code coverage.

rails_code_qa
Currently rails_code_qa runs the unit, functional, and integration tests for a Rails 3 app. It generates 2 different coverage reports. Unit test coverage is calculated on models, helpers, and lib; functional test coverage is calculated on the controllers. Coverage is not calculated on integration tests. In my experience I have found this to be a good way to calculate coverage because it encourages you to write unit tests that cover your core code very well. 

Install
All you need to do to install rails_code_qa in your rails 3 app is add the following line to your Gemfile.
gem "rails_code_qa"
Then update the bundle for the rails app with the following command.
bundle install
Bundler automatically pulls from rubygems.org and installs any dependencies (rcov) that rails_code_qa needs.

Usage
rails_code_qa adds a few useful rake tasks to your rails app.
rake rcqa              #all tests with coverage reports  
rake rcqa:units        #unit tests with coverage
rake rcqa:functionals  #functional tests with coverage

Future
I plan to add other code quality tools to this gem in the future. Likely candidates include flog, flay, and roodi. Feel free to leave a comment if you know of another code quality tool that you think I should include.

13 comments:

cimm said...

Looked promising but the tasks are not available after a bundle install, running Rails 3.0.0.beta4. Is it possible it's not yet working with beta4? Thanks.

Nathan said...

I have been running this on an app that I started in rails 3 beta1 and upgraded through each beta to beta4. I just created a brand new rails3 beta4 app and put it in my Gemfile and after running bundle install all of the tasks were available and functioning.

I am running bundler version 0.9.26, I have had some issues with different versions of bundler in the rails3 beta releases, so you might try updating bundler if you have an older version.

One other thing I thought of, I have only tested this on ruby 1.8.7 on mac and linux.

Salvus said...

A useful and highly reusable, modular gem.

A nice addition would be to generate an index.html file in ./coverage that contains links to ./coverage/units/index.html and ./coverage/functionals/index.html.

Nathan said...

I have been planning on getting that set up at some point. I would also like to get flog, flay, and any other tools generating some html output that could also be linked to.

Nathan said...

Thanks for the feedback Galtzo. I looked at the code a little after I read your comment and realized that I have been a little sloppy with setting the versions on dependencies (rcov, flog, and flay). So I just pushed an update that depends on specific versions of those gems.

I don't know if that will fix the problems you encountered, but it should at least get rid of some variables.

sloser said...
This comment has been removed by the author.
sloser said...

how to use rails_code_qa with rspec?

Nathan said...

I don't normally use rspec myself so I have not yet taken the time to figure out how to get proper coverage reporting of rspec tests working.

Richard Hurt said...

I am using Ruby 1.9.2 and it doesn't like the PLATFORM env var. I changed it to RUBY_PLATFORM and it worked much better.

Galtzo said...

Richard & Nathan: When wrestling with a platform issue I found this (both PLATFORM and RUBY_PLATFORM are wrong!):
http://blog.emptyway.com/2009/11/03/proper-way-to-detect-windows-platform-in-ruby/

Nathan: I am almost done with my updates to the gem. The latest Cucumber (0.9.2) definitely does not work with my windows environment. I used the article linked above to make the dependency optional at build time and load rescues and defined? tests in the code to get around problems when it is missing... More to come.

Stefan Haslinger said...

First: nice work, thanks.
But I see the same problem as in standard rcov: Comment lines are dealt as not run. Do you see any chance to fix this?

Nathan said...

I am planning on upgrading this to work with ruby 1.9.x at some point. When I do that I plan to switch from rcov to the built in coverage library that ruby 1.9.x provides. I think that may do a better job of handling comment lines.

Adam said...

Isn't this similar to metrical? https://rubygems.org/gems/metrical