Random header image... Refresh for more!

Test-Driven Disaster

When first considering using Test-Driven Development, many people will consult their local tester.  This is, of course, the wrong thing to do, because their local tester doesn’t actually care about Test-Driven Development.  It’s not a testing methodology, it’s a development methodology.  Asking a tester about it because it has the word “Test” in it is just as wrong as asking a bus driver about it because it has the word “Drive” in it.  And if you’re assigning your tester the task of “Test-Driven Development”, you need to stop before you damage something, because you’re doing it wrong.

For those who don’t know, “Test-Driven Development” is an Agile methodology for designing classes and developing code, where you begin with writing an automated test, stubbing out a method as you go, then after you have the test, you implement the method until the test case passes.  What many people seem to miss is that testing is merely a side-effect, it’s not the central goal.  Instead, the goal is to develop code from the top down by looking at how you’re going to use it.  There’s virtually no difference between TDD and stubbing out a low level module while implementing a higher level module.  The approach is the same:  You focus on how you’re going to use the functions you’re writing by actually using them, rather than trying to list all the operations you might need without the context on how they’ll be called.  If you don’t understand that’s what you’re doing, you’re bound to foul it up and hurt something.  You can’t do TDD after you’ve already specced out the method signatures, so don’t even try.

Beyond the rampant misunderstanding regarding what TDD is, my biggest reservation about it is the fact that most people who want to try it don’t know how to write tests.  Bad tests are far worse than no tests.  By strictly adhering to the principles behind TDD, you’ll write five tests and think you have everything covered because you red-green-refactored your way to perfection.   In fact, since TDD and the Wide World of Agile will encourage you to use code coverage you’re guaranteed to have tested everything.  Trouble is, you’ve only covered five specific cases, not the five hundred cases that would be apparent if you actually looked at the problem and nowhere near the five thousand cases your users will throw your way.  Code coverage will only tell you that you’ve executed lines of code, not that they’re correctly executing according to your plan.  By exactly following the process of Test-Driven Development, you’re pretty much assured to write nice shiny code that doesn’t actually work, even though it passes all of your tests.  To successfully write effective unit tests, you have to go beyond the initial requirements (Which are incomplete) and the initial design (Which has changed) and look at the solution you actually implemented to see where the problems are.  In other words, your job isn’t over after “Refactor”, you still need to go back and enhance your initial suite of tests before you’re done.

Take, for example, the requirement that you write a function that takes any two numbers and returns their sums.  In the world of TDD, you start with a test:

[Test]
public void TestSumMethod()
{
    Assert.AreEqual(5, SumMethod(2, 3), "2 + 3 = 5 Check");
}

Okay, so, we have the test.  Now we need the function stub.

public int SumMethod(int a, int b)
{
    return 0;
}

Compile, run tests, and blammo!  Your test failed, as it should, because you haven’t implemented anything yet.  This was the “Red” step.  Now it’s time to go Green.  For that, you implement the function you stubbed out.

public int SumMethod(int a, int b)
{
    return a + b;
}

Run the test again and it’ll pass.  Run code coverage and -look at that- 100%.  Now you refactor, rerun the tests and code coverage, and everything’s green, so you’re done!

Except…  You’re not done.  The requirements said “Any two numbers”.  So, you need more tests.  Does it work with negatives?  Big numbers?  Small numbers?  Any tester worth paying will immediately break your function by trying to pass in 2147483647, 2147483647.  And that’s just the beginning.  What about real numbers?  Can it do π + e?  And let’s not even get into complex numbers.  During your refactoring, did you change the inputs to a type that includes NaNs and Infinities?

The point is that following Test-Driven Development left you thinking that you had written a function that was adequately tested, when in reality, it was woefully under-tested.  Obviously, this was a simplified example.  The consequences of doing this with some real, worthwhile code could be disastrous.

Now, I like developers writing unit tests (Well, I like it when they write good unit tests, as I’ve already written…).  I don’t care if they come first, last, or during.  Just don’t fool yourself into thinking that practicing TDD will mean that you’ll have all of your unit tests written as part of your red-green-refactor cycle.

All in all, I like what TDD tries to do.  Thinking about code before you write it and writing unit tests are generally good things to do.  What I don’t like is the shiny glowing path to failure that TDD sets out for those who are unprepared.  I have no doubt that certain Agile practitioners can do TDD and do TDD well.  Unfortunately, I don’t think those people live in The Real World.  If you run your own software consulting firm, then great, go off and do things how you want.  But for everyone else, you’re going to get fired if you try to do that sort of thing, because you’re going to do it wrong and waste a lot of time in the process.

0 comments

There are no comments yet...

Kick things off by filling out the form below.

Leave a Comment