Testing, Code Coverage, and Other Ways You Could Be Wasting Time

I’ll be the first to say that testing and code metrics can improve software quality and increase productivity, but an overzealous application of either could incur a heavy cost.

Tests are code, code is overhead, and while some overhead is necessary and even advisable, overhead is debt and should be minimized whenever possible.

There is no perfect product, bugs will be deployed to live systems (no matter how many tests and quality checks are in place), and for the most part, customers not only tolerate this, they expect it to one degree or another.  What will really win over clients is how fast bug fixes are delivered.

Time is often better spent refactoring existing code, and improving feedback and logging systems. Making code testable will make it far easier to diagnose problems and expedite repairs.  Tests should be put in place to prevent the same bug from surfacing more than once, but more time should be spent improving existing code quality than extending a safety net of tests to compensate for poor code quality.

I hear the term “code coverage” tossed around as if it were a panacea, it’s like the thought is, “if we improve our code coverage, it will solve all our quality problems”.  Code coverage is useful tool, but certainly not the most important measure of quality. Depending on the code, 30% code coverage could be plenty, or in another case, 70% might not be enough – and keep in mind, that even if a particular piece of code is 100% covered, it may still need more testing. Code coverage is only one metric, and at best it’s an incomplete indicator of test coverage. All too often, a rule that requires 80% code coverage encourages testing of things like:

class Foo
{
    public string Bar { get; set; }

    public Foo(string bar)
    {
        Bar = bar;
    }
}

[Test]
public void TestFoo()
{
    var foo = new Foo("bar");

    Assert.AreEqual("bar", foo.Bar);
}

This is not testing the “Foo” class or business logic, and while it does increase code coverage, ultimately all that’s being tested is the C# language itself. This is not a good use of time since you can trust the language to do its job correctly.

Testing, code coverage, static code analysis, and related tools and practices, while important, have lower value than improving code quality and testability. Ultimately, no amount of success in testing can compensate for failure in code quality.