Testy Development

by Jacob 21. February 2008 21:16

Peeved One of the things that irks me when discussing certain technical topics is a tendency towards boosterism that can hinder, or even halt, deeper evaluation and discussion. Some topics, tools, or practices arrive with a divine imprimatur of authority that is hard to resist. Since I believe in being personally responsible for the software development I undertake, I sometimes find myself exploring murky territory that seems like it should be better mapped. The latest of these pre-hallowed principles is Test-driven Development.

What comes across as a casual assumption of superiority gets my hackles up. I’ll show what I mean using a recent post by Derik Whittaker titled Sound of One Man Testing. I want to be careful up front, though, because my purpose isn’t to pick on Derik. Indeed, I really liked a previous post of his describing his love of being a software developer. I suspect that I’d like him if I knew him personally. I’m just using his post so that I have concrete illustrative examples for what I’m reacting to. His post is in good (and numerous) company.

TDD or A Poke in the Eye with a Sharp Stick

I mentioned this before, but it keeps happening. The comparison to TDD seems to always be a complete lack of testing. This is always a flawed comparison. It doesn’t matter if you’re addressing heathens or saints, it simply carries no useful information. It promotes sloppy thinking and lazy arguments because you set such a low hurdle to beat. It’s worse than unconvincing because it irritates people who disagree with you and thus erects barriers to communication.

It is a textbook case of the Straw Man logical fallacy. Defeating a straw man isn’t really much of an achievement and pretending that it is doesn’t make it so. Skipping straight to 50 when counting out pushups is only impressive to someone who didn’t notice. It doesn’t actually do you any good and will certainly never result in those ripped abs your SO goes on about.

Going Mental

Derik says "Following TDD is more about your mind set and attitude [than] it is about the process in my opinion." David Tchepak recently made a similar point (admittedly tangential to his post) in his generous response to my earlier comments, "I feel TDD has the potential to change the way you think about coding."

I get their sentiment and I’m not trying to dismiss these statements out of hand. Attitude, mind set, and perspective matter, and things that keep you focused or have other purely mental benefits have a place in software development.

It’s just that these benefits are highly personal and hard to make relevant. If you include them, it is helpful if you can make sure that they are an important part of your point and that you develop them in a way that stands a good chance of connecting with others. Failing to connect them to your larger point makes them stand out, weakening the whole.

Clique Here

Many posts about TDD are aimed at those who are already convinced. Derik’s post is one such—he is clearly addressing those who already believe in TDD who want arguments to help them bring TDD into non-TDD environments.

Which would be fine in a discussion group or forum, but is more problematic in a blog post. Blogs are, by nature, public (I’ve seen private blogs but I really don’t see their point). Blog posts are likely to be read by people who don’t share your basic assumptions. It is wise to bear this in mind when discussing potentially controversial topics.

Addressing an in-crowd comes across very poorly for those who don’t consider themselves part of your community. It’s like those jerks in High School who talk about others in public as if they weren’t there—which is essentially what you are doing.

Take this section of Derik’s post, for example:

Benefits the project will gain

  • Better code coverage
    Even if you only can cover 5% of the code in your tests, that is 5% less code to worry about
  • Increased code quality
    Do I really need to explain this?
  • Increased code reliability
    Do I really need to explain this?
  • As foundation to grow on
    Because you have [taken the initiative] to create the test plumbing (i.e. configuration, db setup, logging setup, etc) you have reduced the barrier of entry for other developers. I have found that this barrier of entry is a common reason people don’t want to start testing.  I know it is lame, but I have heard it.

Someone not already convinced of the superiority of TDD can take what looks like a casual disregard for their perspective poorly. My inner spite-demon pops up like a jack-in-the-box and says things like "Yes Derik, you do really need to explain this." or "Do you really mean to say that covered code == code you don’t have to worry about?" or "Taking the initiative to provide infrastructure only you use sounds a lot like wasting time and resources to me."

While I got over myself (and reminded myself again that I liked his previous post), the reminder that I’m not in Derik’s clique is off-putting. If you don’t want to re-hash arguments that you feel have been established elsewhere (a perfectly reasonable impulse), give a link (or at least a reference). Bear in mind that there are reasonable people out there who don’t share your convictions and if they show up you would probably like it if they would hear you out.

Remembering that Reasonable People Disagree

There is an unfortunate human tendency to believe that all smart people agree with you. The corollary is that anyone who disagrees with you is stupid. The thing is that even if you are right, communicating from that perspective doesn’t work out very well. Reminding yourself from time to time that people disagree with you and some of them might be worth reaching out to is a good habit to cultivate. I know that I’d personally take it as a kindness.

Technorati Tags: ,,

Tags: , ,

Programming

Post Comments

by Jacob 29. January 2008 12:58

BlahBlah The discussion spawned from my TDD post has been interesting to me. I’ve enjoyed the comments by Phil on his post and by him and others on my own. I’m particularly flattered that one of the authors of the original study dropped by. I disagree with some of his points, but this isn’t where I want to address them. Instead, I want to examine the discussion itself because this is definitely something that we as a software development community could be better at.

The Report

First, I want to emphasize something that I could have made clearer in my original post. The original study was extremely well designed and I have no issues with how it was conducted or how it was structured. I described the report in my prior post but neglected to give my opinion of the setup. Let me correct that now: this is an exceptionally well-designed, well-executed, and well-written study. And I’ll point out that my criticisms were not with the data collection or design. If you go back over what I said, you’ll see that my concerns are with presentation and interpretation.

On Commenting

As I said, I enjoyed most of the comments posted both on my post and on Phil’s. That said, here’s some advice for those expressing opinions online.

First off, if you are going to criticize something, it’d probably be best if you actually read it. Or at the very least, read what is said about it very carefully. I can tell that many of those supporting or criticizing the report hadn’t bothered to get to know the study at all because of little misrepresentations they made (like assuming that two teams from a group of 24 must be split evenly—they weren’t). Here’s the thing, if you make little errors of fact, it becomes hard to take the rest of your points seriously.

Second, if you are going to criticize the way a study is put together, it’d help if you know what you are talking about. Take sample size, for example. While 24 seems like a small sample size to end up with, there are very detailed and specific methods to adjust for sample size in statistical mathematics. This fact is at the heart of the authors' use of the term "statistically significant". In essence (and strongly backed by decades of research) if a bias is strong enough, sample sizes don’t have to be very big to be representative enough to be useful. A little cross-discipline humility goes a long way here.

Now, I have no problem with someone who might say "that seems like an awful small sample size" because that allows you to express your doubt but also leaves room for further enlightenment and avoids making an unsupportable assertion of failure. The thing is that this study wasn’t put together by amateurs. This is an academic paper presented to the IEEE. Like all good academic papers, it includes an entire section titled "Threats to Validity". Reading it, you’ll notice that sample size is not listed in this section. Asking (yourself or in general) why a sample size of 24 isn’t considered a threat to validity is a good idea and potentially interesting in a geeky kind of way.

Third, the force of your conviction doesn’t ameliorate the burden of making your point clearly and rationally if you want to convince someone else. Too often, advocates will substitute emotion for reason when discussing their Great New Thing™. That’s a perfectly human thing to do. It is also perfectly human to reject overwrought arguments out of hand. Seriously, you do your cause no good if you cannot be bothered to make your case in clear and rational terms.

Finally, if you don’t know people you can simultaneously respect and disagree with, the problem isn’t that people who disagree with you are all stupid jerks. The limits of a system isn’t in the brilliance of those arguing for it, it’s the brilliance of those arguing against it. Untested theories and unquestioned understanding are as useful as untested code. Personally, for any given topic, I want the best arguments that I can find. It’s not enough to examine both sides of some new thing, I want the very best arguments that both sides have to make.

Which is why I appreciate the efforts of people like the authors of this study. And why I appreciate guys like Phil who take the time to discuss their theories and to explain and expound them when challenged. None of this stuff is a no-brainer. If it were, people wouldn’t pay us to do it.

Technorati Tags: ,,

Tags: , ,

Observations

Undercutting Your Own Argument

by Jacob 7. June 2007 17:29

Ironic door sign There's a reason that my personal blog is named The Rabid Paladin—I form opinions easily and express them strongly, even as I attempt to maintain an even keel through my sense of integrity. What this means is that on those occasions when I enter an argument with the purpose of informing and/or convincing others, I try to remain open to valid points from other perspectives and the possibility that I might be wrong.

Letting Bias Show

Too often, people arguing their case will paint alternatives in as bad a light as possible—perhaps believing that their misrepresentations make their arguments stronger. The true effect is that it weakens their argument when it becomes clear that their biases have colored their analysis. Technologists are particularly ill-served by this tendency. Computers function logically so there's an expectation that those who are able to understand and manipulate computers will be similarly logical.

So it's often disheartening to me when I see blatant bias in developers and other technologists. Note, I'm not talking about people whose opinions differ from mine or who value some aspects differently and thus favor other technologies or solutions or approaches. I'm talking here about people who want things to be a certain way and let that desire cloud their judgement and rhetoric.

There are a couple of good examples of this at work in the comments to my post about Microsoft. I actually struggled about doing this in a full-blown post because I don't want to pick on people who took the time to post here. I want to encourage people leaving feedback and don't want to give the impression that I'm trolling my own blog for people to make fun of. That said, the bias against Microsoft is something that I've been thinking about a lot lately and this gives me an opportunity to illustrate things that I find off-putting.

Sloppy Equations and Florid Adjectives

The very first comment is by "LKM".

How about "they engaged in blatantly illegal business practices in order to destroy their competition"?
It's not like they were not found guilty of this by a US court.

Now, it's not clear if the reference is to Microsoft being a monopoly or the broader point of Microsoft being evil, but the terms used here clearly show a bias against Microsoft. First off, being found guilty of violating a U.S. law is hardly proof of being either evil or having a monopoly. Name one company Microsoft's size and age that hasn't been sued successfully.

The cold, hard, unfortunate fact of corporate life in the U.S. is that large companies make attractive targets for lawsuits from individuals, corporations, and government entities. Being ethical (or "not evil") isn't protection from being sued, nor is it a shield against conviction. The worst you can say about Microsoft from the fact that they were sued successfully is that they were sued successfully. Any further conclusions that you wish to draw from that conviction still have to be backed up by reason and logic.

Second, LKM's description of Microsoft's business practices as "blatantly" illegal is an example of an intensifier that ends up undermining instead of emphasizing. If their practices had been blatantly illegal, then the trial would have been extremely short if it had gotten to court at all. Microsoft didn't get to where they are today by being stupid about how they spend their money and there's nothing dumber than trying to defend the indefensible.

Frankly, describing Microsoft's actions as blatant also indicates LKM's opinion of those who don't agree. Used maliciously, framing it that way could be seen as a way to cut off debate by implying that you'd have to be blind to disagree. I'm not saying that LKM was deliberate in this or that it was intended as such. Still, this is a form of what I meant in the conclusion of my prior article when I say that painting Microsoft's users and/or supporters as stupid is not a useful way to convince them that you are right.

Ignorance and Hyperbole

Syd from Fairground Town is a little more articulate, but in the end comes off as much more biased.

For me, the incident which summed-up MS was Genuine Advantage. It wasn't the fact that they did it, which I guess was fair enough (what with it being their software 'n'all) but their spokesperson on BBC Radio telling us that "Customers have been crying out for a way to know if their copy of Windows is genuine." This was a breath-taking "big lie", worthy of Goebels, and it was the "last straw" that made me shut down my ten-year-old Hotmail account and buy a Mac.

The beginning here seems like Syd is going to be reasonable. It's a good first step to acknowledge that someone has the right to do something even if you (presumably) disagree with them doing it. He undermines this by displaying both ignorance and extreme hyperbole further in, though.

Now, I didn't hear the spokesperson in question so I can't properly contextualize the quote Syd offers, but I'm willing to bet that Syd doesn't work in a large IT department—a context where the spokesman might actually make sense. While it may be the case that BBC Radio failed to contextualize the quote he heard (making his initial ignorance not his fault), it's important for someone actually trying to be reasonable to look for how a statement that seems so ludicrous actually could be true. Few people lie deliberately or blatantly in a forum guaranteed to reach millions of people. Microsoft's spokesman had to have had a reason for this statement, even if you end up disagreeing with it. Understanding what that reason is should be the very first step in an incident you plan to describe as motivating some action on your part.

Personally, I have worked with a couple of companies where one of the largest headaches of the systems guys was tracking licenses so I understand where the spokesman could be coming from. Licensing was a particularly onerous task at XanGo because our growth was so extreme. The anti-piracy folks at the BSA have managed to win enough cases and levy heavy-enough fines that companies really don't want to go through the expense of an audit that finds that they are out of compliance. One aspect of Genuine Advantage includes the ability for sysadmins to track this automatically and I suspect this is what the Microsoft spokesperson was talking about.

But really, Syd loses me entirely with the Nazi reference. As we gain distance from the horrors of WWII, people seem to be increasingly willing to use Third Reich references to vilify people they disagree with. It makes me wonder if they know anything about the reality behind those events except as a way to put their opponents beyond the pale. It's an obnoxious rhetorical trick and one I've come to despise. I'm to the point any more where I automatically discount anyone making a comparison to anything Third Reich, even if I suspect I might agree with them. The strength of the hyperbole automatically disqualifies them as someone I'm willing to take seriously, let alone support. I mean, if you break it down, Syd is implying that a spokesman for Microsoft is on the same level as someone who covered for the wholesale slaughter of men, women and children.

This comparison is so extreme that it leads me to suspect not just Syd's sincerity but his veracity as well. I suspect he already owned a Mac and already intended to shut down his Hotmail account (assuming he had one). In fact, it's reminiscent of the rhetorical device where a commenter will claim an affiliation they don't have in order to lend weight to their statements (like someone describing themselves as a lifelong member of a political party even as they regurgitate the talking points of the opposition).

Passion, Conviction, and the Right to be Wrong

Now, I'm not saying that either commenter had no right to their opinion or passion. I'm too passionately opinionated myself for that to fly. What I'm saying is that having passionate opinions doesn't mean you should paint counter arguments as less than they are. Doing so diminishes all participants in a discussion, starting with yourself. Indeed, both Syd and LKM ended up undermining themselves to me more than they did Microsoft with their arguments.

I like to tell people that I maintain the right to be wrong. I know that I've been wrong in the past (with a stellar example on this blog just last week). I know that I'll be wrong in the future. Knowing this, I have very few convictions that aren't open to reasoned argument. But please note the modifier there. I don't think that I'm alone in being put off by argumentation that is long on passion and short on logic. Particularly when the terms of those arguments attempt to railroad me or use rhetorical trickery.

And I continue to believe that implying someone is blind, stupid, or allied with murderers is a poor way to convince them to take you or your arguments seriously.

Tags: , , , , , ,

General IT

scruffylookingcatherder.com

Information

    Recent Posts

    Calendar

    <<  September 2010  >>
    MoTuWeThFrSaSu
    303112345
    6789101112
    13141516171819
    20212223242526
    27282930123
    45678910

    View posts in large calendar
    Disclaimer
    The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

    © Copyright 2010 Scruffy-looking Cat Herder