Man, I’m going to catch it for that title. Okay, let me get this out up front: this post is a hypothesis that I consider probable. That’s why there’s a question mark in the inflammatory title. Okay, asbestos underwear in place, let’s continue.

The Hypothesis

Here’s the logic flow for my hypothesis.

1. Unit testing (and its bosom buddy automated testing) provides substantial benefits when used regularly. These benefits are well-understood.
2. Ensuring or regularizing unit testing is the primary value proposition of Test-driven Development (TDD).
3. A majority of vocal unit testing advocates are also TDD proponents.
4. Developers searching for unit testing information are more likely to find references that describe unit testing as subservient to or at least included in TDD than they are to find references that are simply about unit testing.
5. Faced with the substantial alterations in habit and development methodology inherent in TDD, many developers and development managers eschew unit testing altogether who wouldn’t have had a problem with simpler unit testing methods.

Simple enough, really. If 1, 2, 3, and 4 then 5.

I’m not going to bother trying to verify statements 1 and 2. You might be able to quibble with 2, but it’d take some real mental gymnastics to do so.

Testing Propositions 3 & 4

Propositions 3 and 4 are key to this hypothesis being correct so let’s explore them. Here’s a simple test: what are the top 10 links on Google for unit testing and are the addresses they link to substantially advocates for test-first development. This test assumes that the top results from a Google search will be representative of what unit testing advocates are saying. Since Google earns money by the metric ton to make sure this is the case, I’m going to let the results stand for both 3 and 4.

To help with the analysis, I’ll include two rankings for each link. Both rankings are my subjective opinion; included so that you can check my figures for yourself if you wish by seeing how much you agree with my evaluation (or if you agree at all, I suppose). Both are a scale from 0 to 5.

The first scale, Helpful, will indicate whether a developer looking to find initial unit testing information is likely to read the whole thing. A five is something I think most developers will stop to read and digest. A 2 or 3 is either too general or too specific and thus might or might not be interesting to a developer looking for initial unit testing information. Lower scores indicate links that make me question Google’s ranking system.

The second scale, Test-first Focus, will indicate how focused that link is on writing unit tests before coding. A five is something that puts unit testing squarely in the realm of test-first development and includes substantial explanations on how to do so or a base assumption that testing first will be practiced by those reading. A 3 means that it contains information on testing first, but in a more neutral tone—i.e. unit tests will be written when the developer determines it is important with testing first given as a (mere) option. Lower scores indicate that testing first is mentioned in passing if at all. And yes, I am going to lump TDD into XP and other test-first methodologies because they will tend to lead to step 5 as much as formal TDD will.

Okay, here’s the top 10 Google hits for unit testing as of this moment:

Well, fancy that. Over half the entries are a 4 or 5 for their test-first focus. Of those that aren’t directly test-focused, one is wikipedia and the others are unit testing frameworks (I think—that last one is weird).

I’m going to call propositions 3 and 4 proven.

So Then

Here’s the question: given that 3 and 4 turn out to be true, do you think that "Faced with the substantial alterations in habit and development methodology inherent in TDD, many developers and development managers eschew unit testing altogether who wouldn’t have had a problem with simpler unit testing methods"? Personally, I think this is likely the case.

Since the barrier to implementation is higher for TDD than it is for POUT, you have to acknowledge that more developers will be willing to spend the resources on POUT than will do so for TDD. The question is how many are we talking here? How can we find out? I find TDD a much more substantial resource hit than POUT. Thus, at least some people would be willing to spend resources on POUT that wouldn’t spend them on TDD. You can double-check this by trying to imagine someone being willing to spend resources on TDD who would hesitate to spend them on POUT. Do you think there are any? I don’t.

If this is a trend in the general population, then what happens when someone who would opt for POUT if they knew about it is only able to find information for TDD? It’d be only natural if they skipped unit testing altogether.

TDD folks gripe about projects with no unit testing. Non-TDD folks resist and resent being badgered about implementing a methodology they aren’t convinced will justify its cost. I wonder if we aren’t locked in an unnecessary struggle between poles with each side providing ammunition to the other.

I wonder what would happen if the TDD folk started with encouraging people to unit test regularly. After all, TDD folks would have to admit that having some unit tests is better than no unit tests. How about starting your testing discussions with POUT? Or how about defaulting to POUT when you run into resistance for TDD and push for more later if you can? How badly do you want regular unit testing? Badly enough to let go of presumptive TDD discourse? I’ve noticed that TDD advocates never discuss unit testing outside of a TDD framework. Since many of the most popular developers fit this category, I wonder if we aren’t unwittingly encouraging a TDD or nothing effect in the wild.

I wonder what would happen if every TDD advocate went on the record that POUT is way better than no unit testing (with examples and/or encouragement and without going on to propound the vast superiority of TDD). Will you TDD advocates reading this accept the challenge from me to do so?

And here’s an interesting question you can address: how much of the advantages of TDD do you think can be achieved with just good design and timely unit testing? 90%? 50%? 20%?

Because Unit Testing is the plain-Jane progenitor of Test Driven Development, it’s kind of unfair that it doesn’t have an acronym of its own. After all, it’s hard to get programmer types to pay attention if they don’t have some obscure jargon to bandy about. UT is too awkward, besides being a state abbreviation in the U.S., so for this post (and, if it catches on, future posts as well) I’ll borrow from the telco folks and call unit testing Plain Old Unit Testing.

The Best of all Possible Worlds

Part of my problem with TDD has been that it claims to provide complete testing. You see, even if this is true, it runs up against my gut feeling that worse is better when it comes to unit testing. Or, to throw in yet another development aphorism, it is my suspicion that unit testing lies squarely in the realm of the 80/20 rule—80% of the value of unit testing comes from 20% of your unit tests.

While it is functionally impossible to know in advance exactly which 20% of your unit tests will be important later on down the line, that doesn’t mean that we’re throwing darts at a board. Experienced developers can make pretty good guesses about what the problem areas are likely to be. And they’re pretty good about developing unit tests that cover those problem areas.

Which is my way of saying (or restating, really) that number of tests is no more important to me than number of lines of code as a measurement of value in programming.

Crossing the Beams

I think some of the confusion with TDD discussions is that TDD is an intensified version of POUT. Both POUT and TDD use unit tests. Both do so as part of a process. Both are concerned with future maintenance and functionality. The primary substantive difference is that TDD writes the unit tests before developing anything else.

Unfortunately, the popularity of TDD followed the widespread use of POUT very closely. This hurts because we never had a chance to get comfortable with POUT on its own before TDD appeared on the scene. Indeed, many people went from no POUT (or just beginning to learn) straight into TDD.

What that means in practical terms is that we have a tough time separating the value of POUT from the value of TDD. Few people have done both thoroughly enough to judge them based on this relatively singular point of differentiation. The inability to distinguish or track the value of each separately lies at the heart of much of the cross-talk surrounding TDD and TDD evangelism. Indeed, many of the testimonials I’ve seen (including many comments on a prior post) seem to assume that without TDD, no testing is done at all.

Those of us who are happy with POUT (and looking at the extra effort needed for TDD with some distaste) are left wondering what TDD offers us that we don’t already have. We don’t want some hogwash about forcing us to do our job. We see the value of testing and have learned to take time out for POUT (Oh. There’s a slogan that you can paste up on your very own office wall "Time out for POUT!" Remember you read it here first.). Frankly, it seems to me that we’re getting the goodies the TDD folk go on about just fine and without having to retrain how we develop.

A Call for Differentiation

So here’s my request for those of you who are using and enjoying TDD who want to invite us all to partake of its superior benefits: please couch your arguments in the assumption that I am already POUTing, that my POUTing already delivers substantial benefit, and that adopting TDD is a non-trivial training and practice burden. Thank you.