NCover and code coverage analysis
I recently attended a seminar about code coverage and testing. I have used the Compuware BoundsChecker product for years on unmanaged code (it works great). And the presenter spoke about their newer DevPartner products also, but he mentioned a number of open source or low cost alternatives that I didn’t know about. What is code coverage? Wikipedia has a great article about code coverage. The basics are that code coverage is a measure of how much of your code has been run through a series of runs. The metric is used a lot to determine how many of your lines of code are untested. Some tools go on to tell you how many decision points you have in your code, and then to tell you how many of them you actually tested. It is almost impossible to get 100% code coverage on a complex application, but the goal most companies strive for is 80% of their non GUI code. Then other 20% are special cases or GUI driven tests that you perhaps need a human to actually step through the code and cause manually (these are referred to as manual cases). While these manual cases should be documented and performed regularly the obvious goal is to automate as much as possible. NUnit is an excellent tool to perform these types of tests. You write the tests to exercise your code, and verify it performs correctly. Running a code coverage tool shows you how much of your code is being tested by those NUnit tests. NCover NCover is a free code coverage tool for .NET. Release 1.5 is very mature and used by a lot of teams to test their code coverage. It is a command line tool that runs your app and measures the coverage. The output is a very simple XML file that can be used for an automated in house report, or you can use the GUI tool NCover Explorer to look at the results (other GUIs available also). After loading the NCover explorer you can see each of the major namespaces and objects along with a percentage of the code that has been covered. (I included my NCover.bat file in the latest setup for the NUnit tests if you want to look). How are we doing? When I first took over VistaDB the tests ran covered 2% of the engine code. Now with the public and internal Unit tests (271 tests and growing) we have just passed 36%. Some of the engines, like the DDA parser, are nearly 85%. I hope that the stability of the engines show that increase as well. I have another 250 functions stubbed out that need to be written for tests. And running NCover helped me to see the parts of the engine that I am not even hitting yet. By far the hardest tests are yet to come for the SQL parser. First 30 days Today marks the end of my first 30 days with the VistaDB code. I think we have made an excellent start at improving stability and our testing, but we still have a long way to go. One of my Emerald programmers is going to start working on VistaDB part time to help us get the code coverage and unit testing completed as quickly as possible.
