The discussion spawned from my TDD post has been interesting to me. I’ve enjoyed the comments by Phil on his post and by him and others on my own. I’m particularly flattered that one of the authors of the original study dropped by. I disagree with some of his points, but this isn’t where I want to address them. Instead, I want to examine the discussion itself because this is definitely something that we as a software development community could be better at.

The Report

First, I want to emphasize something that I could have made clearer in my original post. The original study was extremely well designed and I have no issues with how it was conducted or how it was structured. I described the report in my prior post but neglected to give my opinion of the setup. Let me correct that now: this is an exceptionally well-designed, well-executed, and well-written study. And I’ll point out that my criticisms were not with the data collection or design. If you go back over what I said, you’ll see that my concerns are with presentation and interpretation.

On Commenting

As I said, I enjoyed most of the comments posted both on my post and on Phil’s. That said, here’s some advice for those expressing opinions online.

First off, if you are going to criticize something, it’d probably be best if you actually read it. Or at the very least, read what is said about it very carefully. I can tell that many of those supporting or criticizing the report hadn’t bothered to get to know the study at all because of little misrepresentations they made (like assuming that two teams from a group of 24 must be split evenly—they weren’t). Here’s the thing, if you make little errors of fact, it becomes hard to take the rest of your points seriously.

Second, if you are going to criticize the way a study is put together, it’d help if you know what you are talking about. Take sample size, for example. While 24 seems like a small sample size to end up with, there are very detailed and specific methods to adjust for sample size in statistical mathematics. This fact is at the heart of the authors' use of the term "statistically significant". In essence (and strongly backed by decades of research) if a bias is strong enough, sample sizes don’t have to be very big to be representative enough to be useful. A little cross-discipline humility goes a long way here.

Now, I have no problem with someone who might say "that seems like an awful small sample size" because that allows you to express your doubt but also leaves room for further enlightenment and avoids making an unsupportable assertion of failure. The thing is that this study wasn’t put together by amateurs. This is an academic paper presented to the IEEE. Like all good academic papers, it includes an entire section titled "Threats to Validity". Reading it, you’ll notice that sample size is not listed in this section. Asking (yourself or in general) why a sample size of 24 isn’t considered a threat to validity is a good idea and potentially interesting in a geeky kind of way.

Third, the force of your conviction doesn’t ameliorate the burden of making your point clearly and rationally if you want to convince someone else. Too often, advocates will substitute emotion for reason when discussing their Great New Thing™. That’s a perfectly human thing to do. It is also perfectly human to reject overwrought arguments out of hand. Seriously, you do your cause no good if you cannot be bothered to make your case in clear and rational terms.

Finally, if you don’t know people you can simultaneously respect and disagree with, the problem isn’t that people who disagree with you are all stupid jerks. The limits of a system isn’t in the brilliance of those arguing for it, it’s the brilliance of those arguing against it. Untested theories and unquestioned understanding are as useful as untested code. Personally, for any given topic, I want the best arguments that I can find. It’s not enough to examine both sides of some new thing, I want the very best arguments that both sides have to make.

Which is why I appreciate the efforts of people like the authors of this study. And why I appreciate guys like Phil who take the time to discuss their theories and to explain and expound them when challenged. None of this stuff is a no-brainer. If it were, people wouldn’t pay us to do it.

In one of his characteristically long essays, Bill Whittle expounded at length on the genius of John Boyd. The essay is political and not necessary to understand my point, so follow the link at your own risk.

At any rate, Boyd proved a tactical theory of aerial combat by repeatedly winning a challenge to the best Air Force pilots in Nevada. The challenger would start by having Boyd in a perfect kill position at thirty thousand feet over the Nevada desert. Once the challenger yelled "Fight’s on!", Boyd had 40 seconds to reverse their positions.

Winning that challenge repeatedly and consistently laid the ground work that allowed Boyd to teach even hardened veterans his combat theories. It opened minds that thought they already knew all there was to know about aerial combat.

And it’s a good thing he did because his theory is counter intuitive. At its heart it claims that making the best decisions doesn’t actually matter in combat. Interestingly, I think his theory might apply profitably to software development.

OODA—Not Just the Sound a Tuba Makes

Responsible action in general follows a very particular pattern—one most of us use all the time without really thinking about it. It’s pretty simple, really, and only the foundation for Boyd’s theory so I’ll be brief in describing it. It’s just four steps.

1. Observe-check out the terrain, look at a problem, see an opportunity.
2. Orient-figure out where you stand in relation to what you have observed.
3. Decide-come up with what you want to do based on your relation to what you have observed.
4. Act-take the action you decided upon.

This is pretty basic, really.

Most people who want to make wise decisions will look at this pattern and figure that the quality of your action at the end will depend on the quality of the prior three steps. They’ll try to make the best, most accurate observations possible. They’ll go over their relation to the situation in detail. And they’ll carefully search out the very best action they can decide to take.

Which would probably work out very well in a vacuum.

The Decision Loop

Boyd’s key insight is that actions in a rapidly evolving situation are not a monolithic whole but rather a series of iterations. In combat, you are not the only one taking action and a lot of what you do has to take the activities and movement of everything else on the battlefield into account. Boyd’s insight is that on the battlefield, victory goes to the most agile, not the most prepared, the strongest, or the one with the best technology. Firing at the position somebody held 20 minutes ago is only useful if they are slower than you are.

Boyd’s point was that taking a reasonable series of actions rapidly beats finding the perfect action to take. More importantly, if you are cycling fast enough a bad action becomes less important because it can be corrected in the next cycle. In other words, if you can get three rounds off with 75% accuracy in the time your opponent gets off his single 100% accurate shot, you win.

Evaluating Agile Processes

It’s the word "agile" that piqued my interest while reading Bill’s essay. Agile development has been around long enough that I doubt it’s a new concept to anyone reading this. To me, Agile development can be seen as an expression or form of Boyd’s theories. If this is true (I’m positing the relationship as a theory to explore), then we have an interesting benchmark to use in comparing or evaluating our Agile practices.

It has long been evident that not all Agile implementations are created equal. It turns out that Agile itself is no more a guarantee of success than any other software development tool.

Two Key Refining Questions

In looking at the effectiveness of an agile process, it seems to me that Boyd’s OODA decision loop doctrine prompts two questions that will help expose weaknesses, explain failure, or provide opportunities to improve.

Are we responsible decision makers?

For Boyd’s theory to work at all, you have to have a solid decision making process. Each OODA step has to be taken, in order, and thoroughly enough that the final action isn’t fundamentally compromised. Missing or partial steps lead to random action and while that might keep you alive for a while, it isn’t going to produce actual victory. Anything that prevents you from observing, orienting, deciding, or acting essentially prohibits success. Whether it’s company politics, personal vendettas, or simple incompetence, this lack means that you are pretty much screwed from the start no matter what action you eventually take.

Can we iterate faster?

Once you have a solid OODA loop, the question becomes how fast can you go? I’ve heard of companies that standardize on monthly or even weekly iterations. I’ve wondered for a while why it is that they do so. Oh, I can see that it can be a motivating factor to have a predictable rhythm. And it creates an atmosphere of accountability when you know for certain-sure that you have to push your baby out the door on a known date.

But Boyd’s theory pretty much means that faster is better. Always. My question is if the benefits of a regular schedule are real and if they are, do they outweigh the inherent benefit of iterating faster? Do you really need to create an atmosphere of accountability in your software development shop and if you do, is a regular schedule the only or even best way to do it? Does your software development team need to be motivated by development policy, and if they do, is a predictable rhythm the one or best way to do it?

After all, a regular schedule applied to variable activities means, by definition, that you are padding your activities to meet the schedule. And there are few activities more variable than software development. Indeed, it seems like a classic example of Parkinson’s Law to me.

A Caution and a Caveat

This post is an exploration of my initial thoughts trying to relate a transformative battlefield doctrine to software development. I think it’s a relevant comparison and one that can provide useful insight. I have a personal tendency to make strong statements even when my certainty is relatively low, however, and want to make that explicit in this case. I never begrudge counter arguments and I personally hold no idea inviolate, even when it happens to be my own.

Also, I want to note that while fast iterations might be good for a quality end product, what you do with your product at the end of each iteration needs to be considered carefully. Jay Barnson explains some of the pros of releasing frequent updates with specific relationship to games. Soon Hui recently explored some of the downside. Both include important considerations that are mainly concerned with the business side of software development. I’ll sum up my caution by saying that what is best for your code isn’t always wise to expose to your customers. I think that this applies equally in both in-house software development and software product development.

Frequent iteration doesn’t necessarily mean frequent release.

Though work and family keep me pretty busy, everyone needs a hobby. Mine is online gaming. My current addiction of choice is City of Heroes where my favorite class to play is the Scrapper. Scrappers engage in close-range fighting with the enemy and are good at dishing out damage, which means they tend to go after the biggest bad-guys they can find in a group leaving others on the team to manage the lesser minions. Since this puts the scrapper in the middle of chaos with a limited scope of influence, scrappers often acquire what’s called "scrapper-lock"—they lose awareness of external contexts and miss important developments (like that their own health is in the red and it’d be a good time to pop a health inspiration).

This came to mind recently during a podcast on Dot Net Rocks. In this episode Carl and Richard talk with Udi Dahan about SOA. For all that Udi tends to discuss issues way above my scale enterprise-wise, I enjoy hearing him flog his hobby horse. He has the ability to make complex concepts understandable by using solid examples that relate well to his points.

Near the end of this episode (around 57:08), Carl asks "If there’s one big mistake that people tend to make, what is it?" Udi answers with our tendency to view challenges through the lens of our current obsessions and assumptions. His response reminded me strongly of scrapper-lock. It is really easy to look for solutions in terms of our current abilities and mind-sets. It is really difficult to take a step back and make sure you are taking on a problem in its entire context.

Stay on Target

There are two important factors to achieving IT goals. The first is identifying the target. This is actually the harder of the two factors because we bring so many unacknowledged assumptions to the task. The first assumption that messes us up is that we work for IT. We don’t. We work for companies, universities, governments, non-profit organizations, ad-hoc teams, and ourselves (and a multitude of other entities too varied to name). I tend to concentrate on corporate development so I’ll talk about working for a company below, but you can substitute any of the above just as easily.

In considering a goal, it is important to have not just a complete understanding of the goal itself, but of the reasons behind that goal. This is what makes software development such a challenge, by the way. You could train pretty much anyone to write the actual code. It’s learning to map the code to the needs of your company that causes real strain.

Take, for example, the requirement that you "hit a target roughly six inches in diameter from a distance of 150 feet." Which target did you see?

In this situation, you can ask all kinds of clarifying questions about the target (what color, outdoors or indoors, how much force etc) but the one question that is going to provide the best guidance is "why?"

Understanding the purpose gives you a load of related context that allows you to skip a lot of stupidity and identify buried assumptions. The reason this is so difficult is that in order to understand the purpose, you often have to understand the business (domain, university, non-profit et al) and most IT people resist doing so.

Gearing Up

Once you understand the target, deciding how you are going to reach it offers its own batch of problematic assumptions. Too often, we let our pet projects or obsessions intrude in this process. Are you creating a web interface because it’s cool and what you know or because it’s the best way to meet your business need? Are you implementing a new enterprise framework because the company needs it or because you really want to try out the whole SOA thing?

This is where integrity becomes important because you’ll often find that the best way to go is less interesting than the one you’d actually prefer. Every project requires trade-offs and those can leave you lots of room to nudge things in directions that might offer a slight advantage, but might also just scratch a personal itch to do things the "right" (i.e. your) way.

As the subject matter experts in our organizations, we have a lot of trust. It is easy to abuse that trust unintentionally by allowing ourselves to give too great a weight to our own interests (it’s also easy to abuse this trust intentionally, but that’s another post entirely). This tendency drives many of the fads you see sweeping through software development now and then (Extreme Programming anyone? Some good things are there, sure, but not nearly to the extent many of us convinced ourselves existed).

Acquiring a Strategic Overview

Because lock-in is so easy to fall into, it’s important that we develop some habits that overcome it. Here are some of my ideas, but please add or modify them in the comments if I’ve missed something useful.

• Develop some key general principles. It’s important that these be both "general" and "principles"—i.e. fundamental ideas that are valid regardless of programming environment or language. YAGNI is a good example of a general principle and one of my core tenets.
• At the beginning of any new project, make sure you ask the most important question: why? Further, make sure you understand the answer. It also helps if you take steps to understand your domain. If you program for businesses, for example, introductory courses in accounting and management are extremely useful. I use information from the one college accounting class I took way more often than information from my one college computer science class.
• Periodically review your tools. Is there a new kid in the Unit Testing world that might have some sweet capabilities (and an easy conversion utility)?  Has someone slain the dependency injection demon for Mock objects?
• Periodically review alternative environments. This is the hardest for me personally because I’m rather attached to MS Visual Studio and the .Net framework. Why should I bother with REST or Ruby or Eclipse? The important thing to learn here is perspective and the important perspective is not your own but that of the company (or whatever) that you work for. However comfortable you may be with Asp.Net, for example, it may be that your company can realize a substantial benefit by adopting Ruby. I’ll never forget the roadblock one developer on my team overcame in a day using python that I was afraid would set us back a week.
• Finally, make sure you are aware of and take the time to understand the current crop of buzzwords. Agile, SOA, TDD, Grid Computing, all of these have value or people wouldn’t be buzzing about them. It’s easy to become jaded after you’ve applied your third silver bullet and the slavering monster is still healthy and looking your way. Declaring all silver bullets useless and hibernating isn’t going to help you. When management comes to you with a new solution they read about in their trade magazine, you’ll get a lot farther if you can speak knowledgeably about its strengths and applicability in your environment. And who knows, you may just find something that slays one of your own dragons while you’re at it.

I was thinking today about something I learned the hard way and how I wish someone had taught me what to look for earlier in my career. Then it occurred to me that perhaps I could perform that service for others.

He Hates Me!

Have you ever found yourself in a situation where someone important where you work doesn’t seem to like you very much? I have. In fact, at my first real programming job (programming in Pick of all things) my manager’s boss obviously had a hard time with me. I would hear from my manager that her boss was concerned that I wasn’t "cutting it" and that I should make sure I highlight my accomplishments when we interacted. And there were definitely a couple of times when he cut me pretty short shrift.

Since then, I’ve been in other situations where someone important seems to have prejudged me in some way that is, at the least, inaccurate, and at worst potentially career limiting.

That experience left me insecure enough that I went looking for a new job.  I knew I was better than the raise I wasn’t getting and programmers were very scarce then. I found a position that paid better and where I had more responsibility (though still programming in Pick). This turned out to be a good move because it let me leave on good terms with the company I was at.

Going Detective

Since there was such a pronounced disconnect between what I considered reality and the perception of others at the company, I decided to dig a little. After a couple of months, I approached some of the insiders there. I asked them what their perception of my work had been, if or when it had changed, and what they thought of me leaving as I did. This turned out to be an instructive exercise.

Long story short, it seems that my manager, the one that I thought was on my side and happy with my work, had been denigrating me in meetings with her superiors. A project couldn’t be completed in the desired timeframe, for example, not because the project was complex but because I couldn’t handle the requirements.

Here’s the Scoop

Since this happened, I’ve seen subtler indications of the same pattern. The thing I eventually came to realize is that work relationships, unlike pretty much any other, start off in the black. When someone hires you for a job, they do so with the conviction that you can do the work required. Any manager or company grand poobah that you interact with will be prejudiced in your favor initially as a result. The instinct is to validate the hire decision.

Relationships with company managers start off without all the typical prejudices we deal with in more social circles—not because managers are nicer, fairer people but because they worked those prejudices out before hiring you (or trusting others to hire you for them). Indeed, your immediate supervisor’s attitude should be the default attitude towards you for all company higher ups.

Which is why, when someone you don’t work with treats you differently than your immediate supervisor does, it should send up a major red flag. They are almost certainly getting feedback that you aren’t privy to. Finding out what that feedback is and where it is coming from could be vital to your career. Expending a little effort and/or political capital can head off major pain headed your way.

What I’m saying is that a negative attitude has to be generated. It is not innate. It comes from somewhere and had to overcome an initial positive bias in order to become strong enough for you to perceive.

Caveats and Addendum

Any one of the following could probably be its own post but here are some things to keep in mind.

• I’m not saying that you should go looking for indications that people hate you. Insecurity breeds reasons to be insecure. If the disconnect manifests, though, be aware that the fault probably isn’t in your professional conduct and no amount of "trying harder" is going to be useful.
• When you screw up (and you will screw up), expect to see that show up in the attitude of your manager and possibly in higher ups (depending on the severity and scope of your screw up). If, over time, you cannot reaffirm the good sense your manager had to hire you in the first place, it’s probably a good time to look for a new job. If, over time, you cannot regain that good attitude in higher ups, it probably means your manager isn’t communicating well. How you handle that depends on the manager. My tendency, however, is to nudge my manager to repair the damage for me because I’m in no position to do so myself. Just letting it go is a bad idea because your manager is letting bad impressions accumulate.
• When you come into a company, you enter as part of a group. There will be prejudices between groups so expect to see that manifest. Marketing managers aren’t going to like programmers. Adjust accordingly.
• Oh, and there is a very small possibility that you are simply a jerk. If every new job ends up with you experiencing prejudice, misunderstandings, and outright betrayal, you might want to take a closer look at that pattern. Close inspection will reveal that the common denominator in all those situations is you...

I noticed something interesting from Google Analytics for this site. It seems that three of the top ten keywords that lead people here are combinations of Microsoft and evil. It seems that in addition to being a popular meme on the Internet, it's also a way to drive traffic to your site.