TDD Considered Harmful?

by Jacob 18. February 2008 23:30

Firefighter Man, I’m going to catch it for that title. Okay, let me get this out up front: this post is a hypothesis that I consider probable. That’s why there’s a question mark in the inflammatory title. Okay, asbestos underwear in place, let’s continue.

The Hypothesis

Here’s the logic flow for my hypothesis.

  1. Unit testing (and its bosom buddy automated testing) provides substantial benefits when used regularly. These benefits are well-understood.
  2. Ensuring or regularizing unit testing is the primary value proposition of Test-driven Development (TDD).
  3. A majority of vocal unit testing advocates are also TDD proponents.
  4. Developers searching for unit testing information are more likely to find references that describe unit testing as subservient to or at least included in TDD than they are to find references that are simply about unit testing.
  5. Faced with the substantial alterations in habit and development methodology inherent in TDD, many developers and development managers eschew unit testing altogether who wouldn’t have had a problem with simpler unit testing methods.

Simple enough, really. If 1, 2, 3, and 4 then 5.

I’m not going to bother trying to verify statements 1 and 2. You might be able to quibble with 2, but it’d take some real mental gymnastics to do so.

Testing Propositions 3 & 4

Propositions 3 and 4 are key to this hypothesis being correct so let’s explore them. Here’s a simple test: what are the top 10 links on Google for unit testing and are the addresses they link to substantially advocates for test-first development. This test assumes that the top results from a Google search will be representative of what unit testing advocates are saying. Since Google earns money by the metric ton to make sure this is the case, I’m going to let the results stand for both 3 and 4.

To help with the analysis, I’ll include two rankings for each link. Both rankings are my subjective opinion; included so that you can check my figures for yourself if you wish by seeing how much you agree with my evaluation (or if you agree at all, I suppose). Both are a scale from 0 to 5.

The first scale, Helpful, will indicate whether a developer looking to find initial unit testing information is likely to read the whole thing. A five is something I think most developers will stop to read and digest. A 2 or 3 is either too general or too specific and thus might or might not be interesting to a developer looking for initial unit testing information. Lower scores indicate links that make me question Google’s ranking system.

The second scale, Test-first Focus, will indicate how focused that link is on writing unit tests before coding. A five is something that puts unit testing squarely in the realm of test-first development and includes substantial explanations on how to do so or a base assumption that testing first will be practiced by those reading. A 3 means that it contains information on testing first, but in a more neutral tone—i.e. unit tests will be written when the developer determines it is important with testing first given as a (mere) option. Lower scores indicate that testing first is mentioned in passing if at all. And yes, I am going to lump TDD into XP and other test-first methodologies because they will tend to lead to step 5 as much as formal TDD will.

Okay, here’s the top 10 Google hits for unit testing as of this moment:

Link Description Helpful Test-first Focus
Unit testing - Wikipedia, the free encyclopedia This is the wikipedia entry for unit testing. As is common for wikipedia, the article is concise and well-focused. TDD is mentioned but only briefly and as a "related topic". 3 1
Effective Unit Testing A magazine article supposedly about unit testing. Third sentence: "What if I told you we are going to start with unit testing?" 2 5
Extreme Rules A page on unit testing at extremeprogramming.org. "you should try to create your tests first, before the code." 4 4
Six Rules of Unit Testing A developer’s list of six rules for unit testing. Good post, actually. "1. Write the test first" 5 5
Chapter 13. Unit Testing A chapter from Dive Into Python. The link itself is only the introduction to the chapter, but the very next section leads with this: "Now that you’ve completely defined the behavior you expect from your conversion functions, you’re going to do something a little unexpected: you’re going to write a test suite that puts these functions through their paces and makes sure that they behave the way you want them to. You read that right: you’re going to write code that tests code that you haven’t written yet." 2 5
JUnit, Testing Resources for Extreme Programming JUnit is an automated unit test framework for java. I’ve never used it, but I understand it is the gold standard for Java unit testing frameworks and the conceptual ur-progenitor for unit testing frameworks in other environments. 3 0
Unit Test A page from WardsWiki. I’ve no idea what that is, but its primary focus seems to be software development. It’s almost more of a forum than a wiki, though. That said, the introduction for the topic includes this: "Key here is to CodeUnitTestFirst. " 1 4
Testing, fun? Really? An old blog post (March 1st, 2001) about unit testing in Extreme Programming. "Unit tests are so important that you should write your tests before you write the code." 5 5
Unit Testing in scriptaculous wiki A wiki for the script.aculo.us Java Script unit test framework. I didn’t know they had those. Cool. 2 0
Unit Testing in BlueJ A pdf manual for unit testing in BlueJ (I’ve no idea what BlueJ is and don’t feel like looking it up). Chapter 10: Writing tests first. This is actually a little misleading, though because the chapter is about how to test first if you want to. 1 3

Well, fancy that. Over half the entries are a 4 or 5 for their test-first focus. Of those that aren’t directly test-focused, one is wikipedia and the others are unit testing frameworks (I think—that last one is weird).

I’m going to call propositions 3 and 4 proven.

So Then

Here’s the question: given that 3 and 4 turn out to be true, do you think that "Faced with the substantial alterations in habit and development methodology inherent in TDD, many developers and development managers eschew unit testing altogether who wouldn’t have had a problem with simpler unit testing methods"? Personally, I think this is likely the case.

Since the barrier to implementation is higher for TDD than it is for POUT, you have to acknowledge that more developers will be willing to spend the resources on POUT than will do so for TDD. The question is how many are we talking here? How can we find out? I find TDD a much more substantial resource hit than POUT. Thus, at least some people would be willing to spend resources on POUT that wouldn’t spend them on TDD. You can double-check this by trying to imagine someone being willing to spend resources on TDD who would hesitate to spend them on POUT. Do you think there are any? I don’t.

If this is a trend in the general population, then what happens when someone who would opt for POUT if they knew about it is only able to find information for TDD? It’d be only natural if they skipped unit testing altogether.

TDD folks gripe about projects with no unit testing. Non-TDD folks resist and resent being badgered about implementing a methodology they aren’t convinced will justify its cost. I wonder if we aren’t locked in an unnecessary struggle between poles with each side providing ammunition to the other.

I wonder what would happen if the TDD folk started with encouraging people to unit test regularly. After all, TDD folks would have to admit that having some unit tests is better than no unit tests. How about starting your testing discussions with POUT? Or how about defaulting to POUT when you run into resistance for TDD and push for more later if you can? How badly do you want regular unit testing? Badly enough to let go of presumptive TDD discourse? I’ve noticed that TDD advocates never discuss unit testing outside of a TDD framework. Since many of the most popular developers fit this category, I wonder if we aren’t unwittingly encouraging a TDD or nothing effect in the wild.

I wonder what would happen if every TDD advocate went on the record that POUT is way better than no unit testing (with examples and/or encouragement and without going on to propound the vast superiority of TDD). Will you TDD advocates reading this accept the challenge from me to do so?

And here’s an interesting question you can address: how much of the advantages of TDD do you think can be achieved with just good design and timely unit testing? 90%? 50%? 20%?

Technorati Tags: ,,

Tags: , ,

Programming

E-mail | Kick it! | DZone it! | del.icio.us
Permalink | Comments (24) | Post RSSRSS comment feed

Comments


February 23. 2008 00:01
Dave^2
I disagree with 2. Ensuring unit testing is a side effect of TDD. TDD is primarily about design (as mentioned in the comments of your previous TDD posts).

No mental gymnastics required Smile


February 23. 2008 07:19
Jacob
@Dave: I wish you had tried some gymnastics before trying to dismiss my point with something so flimsy. Here's the thing: TDD is a methodology. The only relationship that a design has to a methodology is as a limitation. Design literally cannot be a benefit of a methodology. Why is that, you might ask? Simple, because in the absence of the methodology (in this case, TDD), a developer is free to select the very best design based on a project's specific attributes and needs. If a methodology forces a particular design (or limits the use of particular designs), then there is a chance that the optimal design for a project will be excluded by use of that methodology.

I know this because a big part of my frustration with TDD is being forced into strict SoC. Say what you want about the virtues of SoC, the fact is that no design is perfect for every situation (just as every design has a space where it rules supreme--however small that space may be). What I am saying is that even if I thought strict SoC was the bees knees and absolutely required in every project I tackle, TDD has absolutely no bearing on that fact, nor does it have any bearing on my ability to implement SoC to my heart's content.


February 23. 2008 08:49
Dave^2
I'd try some mental gymnastics on ya, but I'm a bit underpowered in the intellect department <dopeyGrin />. Instead I'll resort to flimsy arguments.

"The only relationship that a design has to a methodology is as a limitation. Design literally cannot be a benefit of a methodology... In the absence of the methodology (in this case, TDD), a developer is free to select the very best design"

I disagree. TDD does not limit your design options. It gives you rapid feedback on your design decisions, starting before you even write the production code. It might be tricky to write a test for code that doesn't have SoC -- that is feedback from the process telling you your code is tightly coupled. You can ignore that feedback. You advocate POUT, so you'll be writing a test for the unit regardless of how tricky it is, right? Same design, same test written, but if you are using TDD the coupling is immediately obvious, giving you a opportunity to change the design before you have committed to it.

That argument and semantics aside, it doesn't change the fact that the *intention* of TDD is to help developers produce a good design. That is explicitly stated in all the literature on TDD I have ever read (books by Beck, Martin, Feathers, blogs by Jeremy Miller, Phil Haack et al), and is supported by my experience with TDD. The whole push behind BDD is to help stop the misconception that the point of TDD is encouraging unit testing.

Improved design is the "primary value proposition" of TDD. To pretend otherwise, as you do in point 2, is misleading.

Now I know you believe that TDD does not improve design, but that's a different argument entirely (and one we've discussed before) Smile


February 23. 2008 10:17
Dave^2
Despite our disagreement over point 2, I should add that I completely agree with you that everyone should be encouraged to do unit testing, regardless of whether they use TDD. All the TDD material I can recall reading has supported that explicitly.

Another point I'm not entirely convinced by is your assumption that POUT is significantly easier than TDD. For most of the people and teams I've worked with, writing tests that work with their design is the hard part.

If their design is keeping them from unit testing, then is it possible TDD might give them a lower barrier to entry for unit testing than going straight to POUT?

And in answer to the last question on your post: "how much of the advantages of TDD do you think can be achieved with just good design and timely unit testing?". If by "advantages" you mean "positive results", I would say 100%. TDD is a tool to help you reach a goal. It is not the only way to get there.


February 25. 2008 00:39
Mark Levison
My reply was far too long for a comment. So I blogged it instead.

See: www.notesfromatooluser.com/.../...-driven-dev.html


February 25. 2008 04:35
William Pietri
People have been talking about unit testing for decades, but almost nobody did it. Now they are. I think it's no coincidence that most of the people serious about unit testing are of the TDD school.

In my view, test-first development is much easier than test-after, and for me it's much more effective. I would be very hesitant to suggest to somebody that they start with test-after; I don't think it will be a stepping-stone to good testing.

Instead, I think it will be what it has been for many years: a way to convince people that testing is too hard and that they should just skip it if they want to get anything done.


February 25. 2008 05:11
Miles Thompson
thought my example might shed some light.. our development shop approach to these things went like this..
1. "TDD/Unit Tests/Wha?"
2. Read a bunch of stuff including TDD hype and JUnit things, attend meetings/conferences where the hype gets spread further.
3. Figure it out by reading code. Figure out how the Unit test libraries work. Play with green light GUIs for a bit, realise that they are kinda pointless.
4. Install (in our case) NUnit and TestDriven.NET. Start writing tests.
5. Occasionally say to each other "you know you/we really I guess you 'ought' to write the tests first".. generally, though, we just write the tests when it suits - which is generally (thought not always) just after writing the code when you first want a nice simple/fast-to-start way to start 'testing' out the code you just wrote.

Its a fact that I remember saying 'hey you know you 'ought' to write the tests first'. I do try to do that sometimes (for clarity) but generally just do it when it suits.

Practically wins over talk in our case.. and I suspect in most peoples cases. Programmers look to code for examples of what they should do - and all the code examples are POUT basically. 'Methodologies' have a very weak hold in the real world of programming, 'out there'.


February 25. 2008 06:23
David Arno
You have it spot on William: the history shows that OUT has been around for decades and no one did it. Along came TDD and now some people unit test.

The reason is simple: TDD forces you to do unit tests first and so they get done. Traditional unit testing was scheduled after the coding was done. Coding always runs late and so unit testing always got shelved.

This is a case of the "bleedin' obvious" and it is a puzzle that Jacob refuses to see it.


February 25. 2008 07:13
Alex McMahon
In my opinion, one of the problems is that people don't recognise the difference between what Jacob calls POUT and the traditional perception of unit testing; As far as I am aware Jacob hasn't suggested that unit tests be written after all the development work is complete, rather they are written iteratively as you write bits of code.

I would definately say that POUT is better than no unit testing and also better than 'traditional' unit testing (test-at-the-end). I like the idea of TDD, and I think unit testing needed an exciting idea that people could promote such as TDD. I struggle to see the big selling point for moving from POUT to TDD from a unit testing perspective. I can see the argument that TDD is not really about unit testing, and it's more of a bridge to BDD.

Jacob, keep up the good work in putting forward a different view. Whether you are right or wrong, healthy debate, rather than blind following has got to be beneficial all round.


February 25. 2008 13:33
Troy DeMonbreun
Here, Here!

Yes, my humble request to all  TDD evangelists, at least explain that POUT is a valuable subset of TDD and is SIGNIFICANTLY more effective than NO TESTING at all.

No matter how simple you (as a TDD evangelist) believe TDD is to learn and implement, it is more of a barrier to entry than POUT.


 Jeff Santini 
February 25. 2008 15:08
Jeff Santini
I think the gymnastics are the sticking point.  The idea that TDD is not primarily about design seems less than self-evident.  Agreed based on the chosen acronym, and as other's have said the BDD crowd are intent on rectifying this situation.  

What seems to follow from the concept of TDD being about Unit Testing is that a linear progression exists from no testing to Unit Testing to TDD.

If you acknowledge that, for some TDD practitioners, the practice is more about design than unit testing then the linear progression breaks down and we can move past the idea that TDD is a step beyond just Unit Testing.  

Of course, many people do TDD and there are probably many reasons to do it.  But it is not hard to find references in the literature about TDD providing early and clear pointers towards a clean design.
www.developer.com/design/article.php/3622546
blog.objectmentor.com/.../generated-tests-and-tdd
www.codeodor.com/.../1980

Not claiming the above links are correct, just pointing out they are easy to find.  These came from a Google search on "TDD Design Benefits"

One other reason I would suggest for doing TDD is clarity of purpose.  If I have stated what I want to achieve via a test, before I begin achieving it, I will have a more difficult time veering off track, or progressing beyond the needs of the client, or for that matter, stopping short of the needs of the client.  The test expresses my understanding of the problem.  For me, without the test written, I can fool myself into thinking I know what I should do without having to provide any evidence.  It is analogous to the idea that if you think you understand a concept, go explain  it to someone who doesn't.  If you can't explain it clearly to them, you probably don't understand it very well yourself.  If you can't write a test to state your expectations of the code you probably aren't ready to solve the problem yet.  And if you do understand the problem well enough to write an executable test for it, then why not write it?

I have seen the position above countered with the argument "I don't need TDD to force good design on me".  That same argument can be made about seatbelts and car safety, a healthy diet and longevity, or modern programming languages and GUI driven applications.  There are ways to achieve the second item in the combo without the using the first, but statistics favor users of the first item when trying to achieve the second.



 Jeff Santini 
February 25. 2008 18:09
Jeff Santini
Sorry, in my post above I start by stating "The idea that TDD is not primarily about design seems less than self-evident".  I meant to say "The idea that TDD is not primarily about TESTING seems less than self-evident"


February 25. 2008 18:35
Troy DeMonbreun
Jacob,

As first I read the statement "Ensuring or regularizing unit testing is the primary value proposition of Test-driven Development (TDD)", I thought to myself "Boy, Jacob, I hope you ARE wearing your asbestos undies."

However, short of Dave^2's comments, there seems to be little resistance to the idea so far.

I am shocked, as I figured the last thing that most TDD evangelists would accept was that TDD was a sort of POUT+ or "POUT sandbox".

Maybe the threat of "gymnastics" was too intimidating.  Or maybe the  sound of this shotgun blast hasn't reached the coasts, yet. Wink



February 25. 2008 19:22
Jacob
@jeff:
I have seen the position above countered with the argument "I don't need TDD to force good design on me". That same argument can be made about seatbelts and car safety, a healthy diet and longevity, or modern programming languages and GUI driven applications. There are ways to achieve the second item in the combo without the using the first, but statistics favor users of the first item when trying to achieve the second.

I've made that argument myself and stand by it. All of your examples are fatally flawed as a comparison to TDD and good design. The TDD for design argument is about protecting me from myself from an outcome my actions will absolutely and solely determine (or that a team determines collectively). My own analogy would be wearing safety goggles while cross-stitching to that I won't poke myself in the eye. It could be useful if I were prone to poking myself in the eye while holding a needle, but as I'm not I don't see the point.


 Jeff Santini 
February 25. 2008 21:03
Jeff Santini
Perhaps it is a difference in world views at issue.  This will sound a little corny.  But the me that starts a project is often not the same person that ended it.  Hopefully during the course of a project I will have learned something more about the business domain in which I am working, maybe learned a new tool, library, or language.  With that learning comes different decisions.  With learning you have introduced an unknown into the development process.  So something besides myself(the agent that taught me) has at least partially determined the outcome of my project.  If you introduce a team into the equation, then I think absolute and sole control of outcomes becomes much more tenuous.  

So in my view, though I may be in complete control of me, I have no idea what tomorrow will bring in my software creation world.  For that reason I chose the analogies I chose, and stand  by the form I chose.  Though I do not begrudge you the position that you don't need the help TDD offers in building towards a stronger design.

OK.  So what if I drop the "You are never in an absolute position of sole control" argument.  Even if sole and absolute control were possible there are helpful and unhelpful tools.  Let me attempt an example more strictly analogous to yours.  For me, in this context, TDD would be more like MS Word for an author.  I can write my novel by hand, but it will be much easier if Word is around to help.

OK.  so that was all just a bit of wordplay yeah?  Getting back to the point of your post, I would reiterate from my original comment, that TDD people do not say "At least do Unit Testing" because for most TDD practitioners TDD is about something besides Unit Testing.  The evidence is easy to find, and I think it would be hard to take a serious look at TDD and not see what I am talking about.  I have been doing TDD for 5 years now, and one of the first lessons I learned is that writing tests first is a means to an end, and the end is not test coverage.

Here are some early examples from the mists of 2000 to bolster the examples given in my previous comment.

From the xp site.  Here Design is mentioned as an advantage, but not as THE advantage(It was early days after all).
www.extremeprogramming.org/rules/testfirst.html

Here is Micheal Feathers leading us through an example of TDD and discussing the design benefits.  He talks about Growing an application.  The gist is that TDD helps you grow a design.
www.xprogramming.com/xpmag/test_first_intro.htm

These links were found by searching "Test First" on Google.  Test First was what it was called BEFORE people realized that design was such a large part of TDD.

Hope this is helpful.


February 26. 2008 07:00
Paul Ingles
@jacob: The poking yourself in the eye analogy is interesting. In my experience (and I've worked with a fair few number of teams across all types of businesses) you'd likely never know, until it was too late. Less like poking yourself in the eye and more like losing your ticket for a train, boarding, and then realising when the conductor comes you've lost it.

Of course, you may be the exception to the rule, but I would imagine that you probably do make mistakes- you are after all human.

For example, I'm working with a team at one of the largest UK investment banks on their trading and pricing application. There's little evidence of any test first development (the system is *way* too coupled for one). But, there are flourishes of unit testing (or POUT if you're so inclined towards cute acronyms). Unfortunately, since they were done after-the-fact, they *only* prove the system works. They have not provided any other benefit. Of course, understanding what it means when I say it *works* is itself a problem- since the tests are too large and difficult to understand, and are more a statement the code does what the author wrote it to do (as opposed to describing what the code *ought* to do), but I'm willing to accept it's possible to write good unit tests _without_ writing them first (since that's where I started 6 years ago). Just that it's orders of magnitude more difficult and as a result, less likely.

As it stands our productivity is excruciatingly and painfully slow - I've worked on teams where we've delivered whole releases in the time it takes a single (small) feature to be developed on this system. Productivity is low because the design isn't good. And by good design I don't mean *my* interpretation of how *I'd* like things to look and feel, but the ability to maintain and change things- it's about having a team go quicker with more confidence.

I would suggest that by ignoring the immediate feedback of writing clients for your code you've poked yourself (and others) in the eye a few times, you just don't know it Smile


February 26. 2008 13:52
William Pietri
It could be, Jacob, that your design skills cannot be improved upon even an iota by writing tests first. That's what I'm taking from your comment, anyhow.

But consider that a lot of people who think TDD improved their design skills thought they were pretty good designers before. Some of them, like Martin Fowler and Bob Martin, were published OO experts. Kent Beck himself was the OOPSLA program chair a decade before publishing about test-first programming.

So you might be right; you might just be a naturally better designer than every single person who thinks--like I do--that TDD helped step up their game. You might have nothing to learn. But given your strong skepticism toward others' bold claims, you can see how others might be reluctant to just take your word for it, yes?


February 26. 2008 20:23
Jacob
@Paul & Jeff: Good comments. I think our disagreement actually ends up highlighting scruffylookingcatherder.com/.../...house-call.aspx">some things scruffylookingcatherder.com/.../...y-bad-self.aspx">I've brought scruffylookingcatherder.com/.../...evelopment.aspx">up before: our assumption that we're all doing the same kind of work is a bad one. I probably ought to start off each post with a disclaimer linking to those articles just so that my personal development context is explicit.

I work for a small company that needs internal customization of our business processes. Few of my projects last longer than a week, but I also end up revisiting projects over and over. In this environment an ability to refactor ruthlessly is king. Unit testing is a good skill to develop as well, but mainly for things that are tricky or reused in a lot of places.

Not that I haven't been in (slightly) larger development shops. For example, I was a development manager in a larger in-house shop (~20 developers) that is roughly analogous to Paul's bank experience. While there, we got unit testing (among other things) going and found good results with it. Building a unit test around a bug before fixing it was an excellent technique for getting unit testing built up around prior projects, for example. At any rate, I've experienced a broad sampling of programming environments and been in charge in enough of them to have confidence in my opinions.

So while I certainly admit that I'm no more immune to mistakes than the next guy, I have also experienced the beauty of scruffylookingcatherder.com/.../...e-acronyms.aspx">tight iterative cycles and their corrective effects on development projects. Tight cycles and solid feedback trump pretty much every other methodology I've ever tried. The thing is, where you test in each cycle is immaterial to the results I've experienced. In a tight cycle, testing first doesn't really carry much design benefit because any coupling weaknesses you have are revealed just as quickly.


February 26. 2008 20:47
Jacob
@William: You have some important logical fallacies here.

First off, blindly mimicing others just because that's what the cool kids are doing was never actually a ticket to success (not even in High School). They are not you. They are not me, either. My software development needs are substantially different from anything Martin Fowler, Bob Martin, or Kent Beck are likely to find themselves doing. While their experiences are good to know and I certainly attempt to learn all that I can from people with the respect of their peers, any extrapolation of their experiences has to be done with care and intelligence.

Second, just because TDD was instrumental in the discovery and exploration of certain design patterns for a group of developers doesn't mean that TDD is the only (or even the best) way to learn those design patterns. If you could only learn a design pattern by adopting a specific methodology, I'd question the utility of that design pattern. It'd be like only being able to learn math in a schoolroom with blackboards in it.

Third, ad hominem attacks aren't as effective as you seem to think that they are. Those petty little jabs end up hurting your argument more than any counter-argument I could come up with. You might get encouragement from people who already agree with you, but those wavering even a little bit will find it off-putting.


 andrew 
March 6. 2008 23:37
andrew
I went the "the sound of one man testing" site, and was rather amused that there were grammatical errors in the text... words that were spelled right, but used incorrectly.... which to me highlights the problem with trusting testing tools (spelling/grammar checkers) too much - it can give you a false sense of confidence.... then again, so can not testing... I wrote a routine, the level of confidence was high - until I wrote a test script (after the development) and found a couple of cases where there was an issue... then I wasn't so confident and wrote some more tests... I guess the question is, would I have known to write those tests before development?  would I have written enough test?  could I have given myself just as much confidence by writing one or two tests that it passed? What if you get lazy writing the tests and don't cover the full scope?  Personally I figure it's always better to get someone else to do the testing - someone with an incentive to break it.  You just can't rely on people to do their own testing... If you can't trust someone to code something correctly, then how can you trust them to test correctly (completely) either?  Anyway, yes some testing is better than no testing... at least the words were spelled correctly!


March 7. 2008 21:11
Jacob
@andrew: Yes, some blog writers misspell or misuse words slightly. We have enough non-native English speakers in the industry, though, that I try to look past that to the ideas they're trying to express. I'm not sure about Derik Whittaker's background, but his ideas tend to be clear enough, even if his English usage is sometimes technically off a bit. I disagree with his one post, but his points are clear and interesting.

As for getting someone else to test, yes, that's definitely valuable. I've made the case for scruffylookingcatherder.com/.../...evelopment.aspx">in-house QA before and expanded it with scruffylookingcatherder.com/.../...a-argument.aspx">how to convince a business to pull the trigger. So.  QA == good.

That said, I think unit tests are best written by the developer who wrote (or will write) the code. It'd be interesting to expand on this, but it comes down to two main reasons (off the top of my head): a) the cost of understanding the problem domain in enough detail to write the code would have to be repeated to write the tests and b) there's a synergy bonus to both testing and coding if they're done closely enough together (as the developer jumps from one to the other for some final refactoring).


May 5. 2008 23:27
Jeff L.
I've done TDD for about 9 years now on many different sorts of applications. I have done "test-after" (or "POUT") many times since, as well. It's simply not as effective, nor as enjoyable.

POUT is a lower barrier to entry, but most of the benefits of TDD do not exist, primarily the drive toward lower coupling and high cohesion. I'd say maybe 20% of the benefits accrue, so little as to suggest that efforts are best spent elsewhere.

Sure, a good developer will be able to produce a pretty good design using POUT. However, the majority of developers aren't that good, and thus they suffer a similar quandary--the barrier to entry for *really* understanding how to produce a good design is probably similar to that for TDD.

One other thing that's rarely mentioned is what happens to the tests over time. In a TDD environment, they are central to what the team does, and they stay "live." In contrast, where TDD is sporadically practiced, or POUT is the norm, many of the tests fall into disrepair over time, and often get turned off. Even if they ever had value, the ROI for those tests goes to 0, suggesting that they were most likely a waste of time (sure, they told us that the code as shipped was probably working, but the reason such tests go into disrepair is that the system is changing, which is when the defects start occurring).

If POUT makes you and your team feel better, then by all means go for it. Same for the TDD proponents.

Perhaps neither choice is wrong, but I've seen results from very good TDD teams (i.e. places where the developers "got it") and from very good non-TDD teams. The non-TDD code was what most people would consider good, but the TDD code on the *good* TDD teams (which are admittedly rare) dramatically outshone the other systems. My choice is made. Make your own.


 Marcel Popescu 
December 29. 2008 19:54
Marcel Popescu
Very interesting site. I have been reading more than 20 of your articles in the last few days, including all the commentaries. The end result of that is that now I'm convinced that TDD is a better idea than just POUT Smile Weird that it took me so long, now that I think about it - after all, I have tried to do POUT and I gave up because it was too difficult and I couldn't see the point. (Most of the stuff I wanted to test was impossible to actually write without changing a lot of code... which I was afraid to do without any tests.)

Also interesting, I continue to believe that the service locator / dependency injection pattern is a good idea. (Even though I have only recently started using it, because I didn't want to learn a complicated DI container, and I *hate* XML configuration files.)

That doesn't mean that your articles aren't interesting, of course... just that I think some of the arguments for TDD above are quite compelling - and that your insistence that TDD is about testing is ridiculous (call it "BDD" or "Specification driven design" or "definitely not about testing design" if it helps you grok it).


December 30. 2008 20:57
Jacob
@Marcel: Thank you for the considered response. I particularly appreciate that you've considered the processes as they apply to your own experience and situation. I think that's the best way to approach this over all. Personally, when I write the tests has no impact on appropriate SoC or developing testable code. But I concede that is a personal (and unverifiable) statement of preference.

Also, I really like what I'm seeing from the BDD folks (and, to some extent, DDD as well). I'm not sure BDD has to be test-first, pre se, but that's a quibble. Testing to the specs is a profound shift in focus and answers the questions other methodolgies failed to ask: i.e. what we should be testing (as opposed to how or when).

Comments are closed

Information

    Recent Posts

    Calendar

    <<  September 2010  >>
    MoTuWeThFrSaSu
    303112345
    6789101112
    13141516171819
    20212223242526
    27282930123
    45678910

    View posts in large calendar
    Disclaimer
    The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

    © Copyright 2010 Scruffy-looking Cat Herder