One of the things I am seeing less of lately is the understanding that reasonable people can and will disagree with one another—without either of them being any the less reasonable or intelligent for doing so. It seems to me that people become so invested with the “rightness” of their ideas that they deny the possibility that those who disagree with them may be equally intelligent and well-informed. You see it a lot in politics, but I think that this attitude has crept into development discussions as well.

I saw a manifestation of this in action after a recent Stack Overflow podcast wherein Joel had the temerity to question Robert Martin's SOLID principles (SOLID was the topic of a recent podcast with Scott Hanselman that Joel had apparently heard). I highly recommend both podcasts. It didn't take long for some bright stars in the Alt.Net universe to talk about Joel jumping the shark or the state of his imperial wardrobe. I'm not sure why the impulse to denigrate those who disagree with you is so strong, but it appears to be nearly universal. We go from the belief that somebody is wrong to the conclusion that they are incompetent or ignorant without taking time to draw breath, really.

I think this impulse is not only wrong, but damaging. It represents a voluntary limitation of our ability to engage in important dialog and stretch our understanding.

Education

One crutch of those who participate in this hidden hubris is the belief that people who disagree must be missing data. You can see this most commonly when someone tells you to “educate yourself” or its milder form “try it and you'll see”. The underlying message in those statements is that if you knew what they know, you'd do what they do.

And that may be correct. It really could be the case that someone is missing data and if they had that data they might agree. Since software development is so change-driven, much of development blogging is motivated by the desire to teach others things they might not have heard about before. You have to honor the often unrewarded efforts of those who take the time to put information out there for the benefit of those seeking education.

The problem is that telling someone who questions your tools or methods to educate themselves is arrogant, even if you are completely sincere and honestly well-meaning. You see, there are really only two possibilities for someone questioning you: they lack data or they have arrived at a different conclusion after weighing the data themselves.

In the first case, telling them to educate themselves notifies them of your superiority (you are in possession of data they lack) at the same time you deny access to yourself as a resource (you are directing them elsewhere for acquiring that data). It's a dismissive brush off. In the second case, telling them to educate themselves is a judgement of their experience and conclusion without the benefit of explanation or debate. In both cases you are saying that you are better than they are and that they shouldn't be allowed to participate in the discussion as a result.

If you honestly believe that someone disagrees with you due to lack of information, a better response would be a series of questions or even challenges geared towards examining their point better. “Have you considered” questions or “I disagree because” statements are more helpful; they give others the chance to respond and have the courtesy of taking someone seriously enough to invest in the discussion.

Weighty Matters

What if I'm already educated and I still disagree? What if I've done the recon, seen the scene, danced the dance, bought the souvenir and I still beg to differ with your Great Truth? How can two intelligent people look at the same data, having access to the same information, and arrive at two legitimately different conclusions?

Allow me to educate you. (heh)

You see, most decisions are complex and involve competing principles. Do I take the time to add an abstraction layer that will be useful later or do I YAGNI my way through with the simplest solution that works right now? Do I hassle with System.Data.Common to support multiple database providers or allow a strong dependency on SQL Server? Do I use test first to drive out my design or am I confident that my planned design is decoupled enough and tests ex-post-hackto will be adequate? All of these decisions break down into factors that we weight according to our experience and expectations.

I'll illustrate what I mean. Lets take, for example, deciding if Georgette Heyer is better than Meg Cabot. It can be argued (and I would, indeed, so argue) that they have comparable ability with characterization, prose, story, and plot. However, Meg Cabot sets her characters in modern settings and Georgette Heyer's books are chiefly set in Regency England. Two people could easily disagree over which author is best if one has a strong weight on “historical regency setting” without either being any the less intelligent or in need of education.

Back to software development. Consider some of the factors that might go into deciding to use TDD.

• Force strongly decoupled design
• Ensure a minimum level of test coverage
• Early API exploration and (pseudo) documentation
• Commit project to a test framework
• Commit project to a mock framework
• Commit project to automated testing infrastructure
• Writing tests without the benefit of intellisense

How you weight each factor will determine your judgement of TDD. Someone who is not worried about forcing decoupled design (either because decoupling doesn't pay off in their environment or because they feel they can achieve decoupling without force) will be less likely to chose TDD for their project.

And it gets even more complicated when you realize that each factor can, in turn, be comprised of additional factors with their own weight and trade offs (you could easily break “decoupled design” down further, for example).

Learning to look for underlying principles and the weights that others may apply can be interesting as well as useful. More importantly, it opens up the gray areas and opportunities for honest disagreement (without denigration) that are vital when working with IRealWorld.

Educating Yourself

As software professionals, we owe it to ourselves, our employers, and our clients, to be educated in our craft. That's a not inconsiderable burden in a field that grows by leaps and bounds year after year with no sign of slowing any time soon. It is hard enough learning all the concepts, patterns, and practices (not to mention tools, environments, and platforms) that it is often tempting to find a core of experts that you rely on to make decisions for you (and there is some benefit in doing so initially as you come up to speed on the intricacies of our field).

But to be truly educated, you have to go beyond what your experts say and learn about the principles that exist underneath. That's extra work but more importantly it is extra responsibility. It may be that when you break a practice down into its constituent principles that you will find yourself in a situation where “Best Practices” aren't, in fact, best. To me, it is the ability to not only make that determination but to then act on it that truly makes a “professional”.

I'm on record as dissenting from some of the planks of the Alt.Net bandwagon. I question design for testability and have extended that to questioning what I consider to be the over-use of Dependency Injection (though I've also talked about using it successfully in a project that I believe warranted its use). Further, in my last post, I asked if the Alt.Net folks couldn't expand their treatments of design principles to include contra-indications or fault points. I also decried those who actively stifle alternative viewpoints, though I left it vague about who I think might do so.

Given all of that, it's easy to see how some Alt.Net folks might see me as an antagonist and read my posts from an adversarial perspective.

For the Record

Realizing this, I thought I'd take a post to set the record straight. I'm glad that the Alt.Net movement exists and that there are so many quality .Net developers putting so much effort into that community. Software development is a non-trivial task with a huge range of concerns and diversity of environments. Having a group of people with an eye on broader software developments who then take the time to translate advances in other spheres into .Net is an invaluable service. Many folks in the Alt.Net crowd not only teach and explain software development advances, they work with awesome intensity to bring tools to the .Net environment that enable and promote these methods and practices.

Their passion and energy have advanced .Net development immensely and I would do nothing to take away from or diminish their achievements. We (as a .Net community) benefit from their work even if we don't implement the practices, tools or patterns they espouse. Not only do we gain from the visibility they give our platform, but we enjoy improvements that make it back into our development environment (whether through Microsoft or through tools vendors who incorporate and enable better designs).

Elitist?

So is the Alt.Net community elitist as they have been accused by some? I'd be careful about making such an accusation. Any group is going to be “elitist” if only from the simple fact that for a group designation to be useful you have to have people both inside and outside that group. Attributing a personality to Alt.Net is complicated, though, by it's amorphous definition. There's no official designation, no tribunal, no excommunication procedure—there's not even an official, endorsed doctrine. Alt.Net is a convenient designation to describe a general group of practices (and/or developers who favor or promote those practices) the details of which are in a more or less constant state of flux.

Generally speaking, those who identify themselves as a part of Alt.Net are some of the best and brightest in the .Net development community. They're smart and they have strong opinions (however loosely held) and they are willing, even eager, to express those opinions when given half a chance. Smart people expressing strong opinion can come across to others as elitist but I don't think that is an accurate description of Alt.Net as a rule. At worst, you can describe them as having high standards and the guts to promote the advantages of adhering to those high standards.

That's not to say that there aren't those in their ranks who sometimes behave arrogantly. As I described in my last post, I was stung by the public remarks of one prominent Alt.Netter and his casual disregard for my post and for me personally. I wouldn't, however, paint that incident as either representative of Alt.Net as a group, or even as representative of that specific person.

Painting the Alt.Net community as “elitist” can be simply an excuse to disregard them and their practices and I think doing so is a mistake. Even if they were a bunch of elitist jerkwads (which, again, I do not consider them to be), the fact remains that their ideas and principles are valuable and ignoring them will only hurt you. I may disagree with the broad applicability of Dependency Injection, for example, but I'm glad I learned the concept from them so that I was able to use it when I ran into a situation that was pretty much tailor-made for that pattern.

Transparency

One of the reasons Alt.Net suffers the slings and arrows of their detractors is that they take pains to be transparent. Their email list is open (and public), many of them are prolific bloggers, and they aren't afraid to speak up when they disagree with each other. This transparency can lead to unfavorable comparisons to organizations that are less open (like, say, the Patterns and Practices group at Microsoft). Such comparisons are misleading at best.

Laying your disputes out for public observation can make you look weaker than you really are because people will use those disputes as an excuse to ignore your strengths.  It's easy to miss the reverse side of that coin—being open allows your ideas to be broadly tested and honed. It takes a certain degree if intellectual fortitude to operate transparently, but being able to do so leads inevitably to stronger solutions, principles, and practices.

Those who are willing to put their ideas out there with warts and all deserve respect. I, for one, am happy to give them the respect they've earned (even as I remain willing to question individual applicability in my own development).

Many of the interesting .Net bloggers are part of the Alt.Net crowd; evangelizing Dependency Injection, Design for Testability, Test Driven Development, SOLID design and other development practices that they find useful in their work. It doesn't take long reading these blogs to pick up on what looks like an unforgiving attitude towards those who don't use the latest acronyms in their software development.

This acrimony is unfortunate because most often what is at the heart of those who question the standard Alt.Net toolset isn't so much principle as it is context.

A Fundamental Assumption

Unfortunately, discussing software development at all imposes an initial assumption that too often goes unexamined—the assumption that the software development you are doing is substantially similar. I'm afraid that this assumption is violated more often than we realize.

Most prominent development bloggers are consultants, conference speakers, tool vendors, or framework providers of one sort or another. As such, they are used to considering the problems of very large applications with many cross-cutting concerns. Is it any wonder, then, that they are strident about the importance of segregation and testing? For the majority of their hands-on development, everything that increases separation and independence is a benefit because isolating fail points and mitigating the cost of touching individual elements carries a heavy multiplying effect on overall complexity. No wonder they want to use a DI framework on every project and insist that other developers practice TDD. It's all up-side in such projects because the costs disappear in the noise and the benefits stand out.

Not all of us work in those kinds of environments and/or applications, however. Many of us work on projects that are wildly dissimilar. Our applications are narrowly focused, single use, and with limited distribution. For us, implementing solution-wide DI eats up half the project configuration overhead and delivers functionally zero chance of eventual benefit.

I'm unsure how many find themselves in similar (smaller) development shops. I suspect quite a few. We might even be a majority of actual, working developers. We're pretty much absent from the development blogger population, however.

A Problem

What that means for us small-shop developers is that we end up having to do a lot of on-the-fly translation. Up-scale bloggers have shown themselves averse to “thinking down” so we end up having to analyze the methods and techniques they discuss to see where the fail points are for our projects. If your QA team is the folks in the warehouse and your architecture a straight-on table insert, is TDD really going to pay off? What about bug tracking? Or continuous integration?

These translations are interesting and part of why I like smaller development. Our problem space is largely undiscovered country. We can't really trust anything labeled “best practice” and we have to do our own experiments to find what works and what doesn't and at what point it starts to make sense to engage different methodologies.

It'd be nice, though, if some of the gee-whiz bloggers would discuss the fail points of their methods and tools. Or point out the trade offs that various techniques are likely to entail. No technique or method is so useful that it is universally applicable, so where are the borders?

Worse, though, are those who actively stifle discussion of such fail points. Mention that you'd rather not use Dependency Injection, for example, and you can find your professionalism and/or intelligence questioned. Indeed, for many the very fact that you don't <insert favorite practice here> indicates that you must never have tried it. Or if you have you must have been doing it wrong.

This friction fosters resentment and shuts down discussion. Perhaps one reason we don't see more developers willing to discuss alternatives to the popular band wagons of today is a reluctance to enter what amounts to a hostile arena. As the troubles on Wall Street filter down to Main Street (where us small-shop developers live) the last thing we need is other developers calling us unprofessional in public. Particularly when those other developers are well-known and respected in their community.

I know this from personal experience. I'm pretty secure in my current position. My company is healthy and has a marked competitive advantage that I am key to delivering. My boss respects me as do the owners of the company. Even so, my gut clenched tight and my heart sank to my knees when I came across a twitter message from a luminary in our field (someone any informed .Net developer would recognize) dismissing me as unprofessional and my points as unworthy of consideration.

Seeing something like that will make you wonder if expressing your doubts is worth the risk, even when it isn't directed at you.

The latest .Net Rocks podcast was from a panel at TechEd 2008 (that I sadly missed while there). Richard Campbell was the moderator and the topic was Software Quality. It was a good, if somewhat one-sided discussion and I recommend giving it a listen if you are so inclined.

I particularly liked Billy Hollis' perspective on quality because it was a perspective that was grounded in both reality and sound business principles. He kept bringing up quality in terms of trade-offs and the others kept trying to waltz around those comments with a more absolutist, almost dogmatic, vision of software quality. I think it was a real shame that Billy Hollis was so out numbered because his points are important if we are to gain the stature we desire as a profession.

The final comments completely got on my nerves. Which is fortunate because it helped crystallize my discomfort with the others.

Billy Hollis started off with this simple statement.

My key insight that I try to communicate is it’s all a matter of trade-offs. And getting to what the user needs is first and quality is defined within that context.

To which Neil Roodyn responded.

I’ll disagree. It’s about quality. You’ve got to get your quality as high as you possibly can. Always. The higher the quality of the software, the higher the quality the software will remain through the development life cycle. So right from iteration zero keep that quality as high as you possibly can.

And Jeffrey Palermo added.

The higher the quality the better. You can’t have too high of quality, you would just say you have too much behavior for the situation.

The thing is that Hollis is right and the other two are not only wrong, but dangerously wrong. Other members of the panel (David Platt and Neelesh Kamkolkar) didn’t support Hollis either, though they weren’t as overt as Roodyn and Palermo.

What Are You Trading?

The thing is that software quality, besides being pretty much impossible to measure, isn’t a first-order good in the first place—i.e. it isn’t valuable in and of itself and delivers what value it adds indirectly. You can make the reality of this simple business fact plain by pointing out how the deciding factor between two competing products is often not quality. Betamax anyone? Rolls-Royce? Sun Microsystems?

People choose software for a lot of different reasons and those reasons are very rarely explicitly the “quality” of the software. Price, availability, familiarity, and usability all play roles in the purchase (or use/approval if internal) of software and these factors need to be kept in mind. It could be that it is better for your company to have software that is harder to maintain but completed two months earlier. I do this all the time because I have lots of projects that are single-use and with a restricted domain.

Software development involves time, money, commitment, and concentration. Spending those valuable resources should be done carefully and be based on actual value to the company. Quality has a cost. Quality doesn’t always have a pay-off. Business people have a term for a cost that doesn’t return a pay-off. That term is usually “good bye”.

Dangerous?

So Palermo and Roodyn are wrong, but why are they dangerous? This comes back to my observations during the dot com bust. I observed the glee in putting IT back in its place. You see, during that whole tech bubble IT got arrogant. Technical folks were wagging the business dog, dictating strategy, policies, spending, and sometimes even remapping the boardroom to their liking. Which would have been fine if the business rules really had changed as some thought. Unfortunately, they hadn’t. This arrogance set the entire profession back because business leaders learned not only of our arrogance, but of our ignorance as well.

We don’t bother learning even basic business principles before we make important, even critical, business decisions. This discussion of quality is an example of just such thinking. Roodyn and Palermo are talking about dictating business resources without any concern for accommodating business realities. Worse, they don’t seem interested in even learning about or exploring the business realities they, of necessity, work under.

As much as we might like the universe to bend to our personal theories and values, the fact of the matter remains that there are a lot of reasons to compromise aesthetical purity when the rubber meets the road. Those reasons don’t simply go away just because they make us uncomfortable. Indeed, how we react to the inevitable trade-offs required will largely determine how much business leaders decide to trust us in the future.

Here’s a helpful exercise. Imagine you are a business leader evaluating which company to hire to solve some software need. As part of your vetting process, you ask two finalists their stance on software quality. The first one answers with “Getting you the things you need is our first priority and quality will be defined in that context.” The second one answers with “Quality is number one. Always. Quality cannot be too high though it could be that you want too much behavior for the situation.” Which would you hire? If you answered the second, it is because of ideological blindness. Actual business leaders will not only tend to go with the first, but will have confidence and trust in doing so.

Are You Agile?

A lot of the value in IT comes from giving businesses flexibility. Processes that used to take indoctrinating a new generation of workers can now turn on a software release. While this is both a blessing and a curse for businesses, it remains a key to our value in IT. Businesses have learned to be wary of our use of resources, however, and with some cause. It is unfortunate that they cannot trust IT, but until we let the needs of the business drive our development I’d say that their trust hasn’t yet been earned. If we really want to be masters of our own fate, we must learn to live in the messy world of business. We can only do so by proving that we will use the flexibility inherent in our profession wisely.

This post is from my new digs at The Runtime, cross-posted here for your edification. I’ll continue here for the foreseeable future, so no need to jigger your subscription if you don’t want to.

I thought I’d take a shot at introducing myself here as suggested by Jay and maybe dispel any pretense at being a thinker, heady or no. Frankly, it’s probably long past time that I put together some kind of background post about myself if only to give those who disagree with me a way to discount my arguments out of hand.

I suspect this’ll be long as I do go on sometimes. You can skip to a recent, similar post if you only want the heart of my current situation.

Street Cred

I’ve been playing with computers in one way or another since my fifth grade class acquired a then-new PET computer (I recognize the CBM Model 4032 from that link as the one we had). I think it had 8k RAM and we saved and loaded our programs using cassette tapes. I bought my own used Atari 800XL from hard-earned savings as a young teen and have owned a personal computer of some sort continually ever since (frankly, the Atari made it all the way to my Sophomore year of college when PC-envy finally became overwhelming).

It didn’t occur to me to take computers seriously as a career until after I graduated college, though. That’s right, I went through college working in the computer center and making money on the side setting up personal computers and local networks for small-companies and individuals without realizing that I could actually major in them. The one intro to programming class I took (which I remember was Pascal-based) was so easy (if only from familiarity and years of being a computer geek) that I treated it as a study hall.

After graduation, I discovered that my computer skills were worth more on the market than anything I’d actually studied so that’s what I did. This was the mid 1990s so mildly talented lighting fixtures could find programming jobs. Kind of like the mid 2000s, really.

Jacking a Career

I started my actual, salaried career working for a couple of vertical market software vendors who dominated their respective niches by using Pick (which had it all over those stuffy relational databases for the ability to map real-world information). It was with Pick Basic that I discovered the true meaning of Spaghetti Code. With great power comes great responsibility and with great responsibility comes the opportunity to make very large mistakes (none of them mine, of course).

Which means that Visual Basic 6 was an upgrade in sophistication for me. Fortunately, I learn fast—my single greatest career asset. When Visual Studio and the .Net framework came around, I was all over that in turn.

I started my own company with a very talented developer and friend in early 2000 serving the Multi-level Marketing vertical market providing independent compensation programming. If you’ve ever been in one of those presentations where they describe how you can earn money from the sales of people you recruit and they mention "levels" or "generations" you’ve pretty much seen the the design documents I was given. Ever wonder how those were programmed? I did, too. I’ll just say here that walking a tree with money on the line is an addictive form of adrenaline.

It’s a small and competitive niche, though, so it was only really viable while the market was booming. Also, it turns out that I had a lot to learn about marketing and sales.

Developer In Da House

Since then, I’ve been an in-house developer and/or development manager for various small to medium-sized companies. I’ve learned enough about how businesses operate to be a good interface to upper management (I can speak all the relevant dialects including accounting, sales, management, and operations) and I work in the trenches enough to not have completely lost the respect of actual developers.

My current position is with a small reading-glasses wholesaler where I am the development department. We’re relatively top-heavy in the IT department than you would think we need, but that’s because the company’s value-add in the marketplace is our ability to orchestrate complex vendor interactions (with up to 120 day lead-times) to provide a flexible product-line at a high-enough volume to satisfy our retailers.

I also contribute to a couple of open source and volunteer projects, the most notable being Subtext. I feel guilty about bringing that one up, though, because it’s been months since I’ve done anything helpful there.

My Hood

Since my in-house projects are small and self-contained, I tend not to use many formal development processes. My users are all within fifty feet of me and I’ve trained them not to hesitate in letting me know when something goes wrong or when they think of a better way to do something. If you’re doing things right in the first place, this isn’t really as disruptive as you’d think. My longest project in the last two years has taken less than a month to go from the start of development to deployment. Some of that is because I am just that good, but a lot of it is that we’re not doing anything that complex. It’s a sweet-spot, I admit.

All of which makes any formal development process so much overkill.

Flashing it Hardcore

I’ve been around long enough to have seen silver bullets come and go. It has gotten to the point where the only dogma that I’ll tolerate is anti-dogma. I am a Microsoft fanboi, but that doesn’t mean I don’t explore other technologies and ideas. I’m not into didacticism of any flavor.

I do have a core set of practices that I tend to use. They each have two criteria:

• I understand how they work.
• I’ve used them and verified that they work as expected.

There’s nothing like real-life to expose flawed base assumptions or principles.

As you can see, this tends to make me a process skeptic. I love iterative development because I’ve used it and seen that it works. I like Agile as an idea and implementation of iterative development, but I’ve never actually seen it in action so I try to keep my discussions of it on a theoretical level.

I’ve been known to create and maintain the occasional unit test, but I’ve found that even that can be overkill on many of my projects. Test Driven Development is interesting and all but I can’t really see myself using it. It’d be a pain to retrain my way of thinking and I’m not convinced that the payoff is there (and yeah, I’ve heard the testimonials). I’m not going to move to an experiment until I understand how the payoff is supposed to come. And with TDD, I just don’t see it.

This attitude is what keeps getting me in trouble with the Dependency Injection crowd. I’ve worked out a specific, relatively narrow set of circumstances where it can be useful, but as I found out when I actually tried to implement it, those circumstances turn out to be extremely rare in an already well-architected solution.

Busting Out

Skepticism aside, software development fascinates me and I truly cannot leave new stuff alone. I echoed a sentiment by Jay Kimble recently that says essentially that claims of universal applicability are always wrong. But there’s a flip side I want to acknowledge as well: every process, tool, or technology is useful somewhere.

What that means for me is that every popular tool or technique deserves a fair hearing and an attempt at understanding to see if there is something there that I can use. Rejecting something new merely because it is popular is no better than adopting it for the same reason. I admit that this is a weird blend of skepticism and optimism. That said it turns out that, while uncomfortable, it’s really useful for growth as a developer and maintaining the ability to jump into new situations and still find yourself in familiar territory.

Software development is fun and I can’t seem to be able to leave it alone. Fortunately, it’s a useful addiction—not unlike, say, air. And if you’ve stuck with me this far, you’re obviously interested in it yourself (or you’re my wife who reads every post if only because she knows I’ll eventually ask her what she thought of it). I hope you’ll stick around and join the conversation if so moved, even if only to tell me where you think that I am wrong.

"So when will you be done with this development project?"

I don't know about you, but I hate this question. There simply is no good answer for it. It seems like such a simple question with a simple DateTime valued answer. One of these days I swear I'll answer with, "Oh, I'll be done next Tuesday at 2:34pm." just to see what happens.

And seriously, businesses hate that we have such difficulty answering the question. It seems perfectly reasonable for them to want to know when they can plan to have the new processes that they know they desperately need. Developers demand high salaries and are ostensibly professionals, they should be able to give a professional answer, right?

The Road is Well Paved

The thing is, software development is a lot harder than people expect it to be--and this includes software professionals. Even simple software projects can run afoul of hidden complexities that can destroy well meaning estimates and make everyone unhappy. And no matter how you hedge your answers, people simply don't remember all your caveats, maybes, and what ifs that you use to indicate uncertainty.

The end result is that developers seldom make their ship-by dates and companies become disillusioned and impatient with all software development. That's not helpful for anybody, but it's pretty much the rule anymore.

And the fact of the matter is that the vast majority of developers (and development managers) never learn how to answer the estimate question. They'll move from company to company, repeating the cycle of hope, suspicion, and disappointment over and over again. Which works well enough for the developers in the boom times when the demand for development is so high that mildly talented house plants can get hired as developers.

So a lot of people are making the same mistakes over and over. Businesses can be excused for assuming that this is simply the way things are and feel confident in their distrust of software professionals. They've been there, done that, bought the t-shirt.

Paying the Toll

This environment causes developers who care about these kinds of things a lot of heartburn. Everyone pays for the ongoing cycle of disillusionment. I believe that this is what really prompts posts like the recent ones from Ted Neward talking about professional ethics. And I've been known to throw my own hat into the ring as well.

We get tired of paying for the sins of those who have gone before. And I'm not referring to the messed up legacy code we stumble into, either. Frankly, messed up code is the least of your problems coming into a situation with a client who has been burned by previous developer promises. Companies that have had deadline after deadline missed have a degree of mistrust that is very hard to overcome.

We pay for this distrust in a hundred different ways. The thing is, trust is a paying commodity in business. Working with partners you trust means a whole lot of overhead you can simply skip. An analogy: if I trust a plumber to fix my sink quickly and professionally, I can go get a burger and leave him to it. It's only when I don't have that trust that I have to pay the additional overhead of having someone I do trust watching to make sure he's not napping under the sink.

Want to see a business manager go into a dreamy fantasy? Ask them what it'd be like to be able to trust their software developers (in house or not). The more experience they've had with developers the more intense the fantasy.

The Rubber Meets the Road

We have a couple of areas of friction in businesses that exacerbate this situation. The main disconnect with business managers is that we have borrowed terminology and tools from other disciplines without understanding that our processes are fundamentally different. It's tricky because the temptation to use manufacturing terminology is immense. After all, we are creating a product of sorts. This makes so much sense on an intuitive level that it's hard to realize that the comparison is misleading and potentially dangerous.

I wish we could retrain everyone to make analogies to other business specialties. Scientific research or law come to mind as potentially useful analogies because both are similarly plagued by the impact of unique situations, changing ground rules, and unforeseen complexities. It would be interesting to investigate how managing software development like a patent application or drug research would change how we look at the problems involved. We might have stumbled onto iterative cycles and responding to altered requirements a whole lot sooner, for example.

Paying Attention

The real problem, though, is that most developers (and even most development managers) don't take the time to learn about common friction points. Nor do they take the time to build relations with their business counterparts so that you have some political capital (aka trust) to use when it is needed. It's easy to forget that much of the progress in software development practices are pretty recent in terms of business processes. After all, business managers don't move at the speed of light and changes tend to take time to penetrate those layers.

Which means that a whole lot of industry advances aren't even theory yet in the board room.

And the fact of the matter is that you cannot expect a business manager to understand what makes Agile practices work. Or the reason that strong unit testing saves time over the long run even though it takes more time up front. Learning to communicate at a level that is sufficiently detailed for smart business decisions without getting bogged down into the jargon inherent in any specialty is an invaluable skill, and one best learned earlier than later. That means thoroughly understanding those theories yourself--not just on the surface or in buzzword compliance. It also means learning to communicate that understanding from orbit, 30,000 ft, 5,000 ft, and right on the ground. This is hard to do. It takes practice. It also takes exposure to business manager types. I'm not sure which is harder...

Something to think about, though: not learning this skill leaves you at the mercy of those who do learn it.

My point, though, is that it takes both. You have to learn your profession so thoroughly that you can deconstruct its "best practices" ("design patterns", whatever) and rebuild them from basic principles on the fly. AND you have to learn to communicate that understanding comfortably to people of varying familiarity with software development in a business environment.

That's what it takes to be a true professional. It's easy to let those two skills fall out of balance. Individuals who understand both are invaluable to a company. Also rare. Companies who discover someone capable of both are often surprised at how much smoother things run with that person placed where they can do the most good--a point Jeff Atwood's latest on becoming a better programmer drives home.

So I don't have a formula for quick and accurate estimates. Just a lot of hard work. Still, here's a tip for free: anyone asking for a firm delivery date is inherently assuming BDUF. Once you know that, you know where to start your answer.

Lidor Wyssocky has some good thoughts on why it is that developers don't implement changes that they know would be helpful.

The problem is that although we know exactly what doesn’t work right and how it should be fixed, most of us will never say anything. We don’t say anything because there’s a very good chance the minute we do we will be marked as uncooperative, pessimistic, or simply detached from the business reality. (emphasis in original)

He concludes with his call to action.

If more of us say what we know in our hearts to be true, the rest won’t be able to ignore it anymore. Hopefully.

And I agree with him. What he is talking about is an aspect of professional integrity. If you are a professional that a business relies on for good decision making, then you need to act the part and provide reasonable suggestions wherever they may do good.

But there exists both a flip-side and a problem with Lidor's points that are inherent in professional integrity that need to be explored.

The Flip-side

While you avoid negative attention by remaining silent about needed process change, you can gain positive attention by supporting whatever has caught the eye of your executives. This is a major contributing factor to "The Silver Bullet Syndrome". At XanGo for example, we had one company convince management that they could cut development costs to 10% of their current level. Not 10% off, mind, that's 10% of current levels.

Despite all logic to the contrary (I mean, think about it--if a product could actually deliver cost savings of 90%, every company on the planet would be using it--they'd be incompetent not to), XanGo ended up spending millions of dollars and wasted a full year of development on this silver bullet. What was most fascinating to me were those who were willing to endorse this silver bullet. Those people were promoted to team leaders and project managers. Any who would oppose the new development project were either removed before its introduction or told (in at least one case explicitly) that negative feedback would put their jobs in jeopardy.

In this way, technical people who should have known better ended up perpetrating the problem not by staying silent but by actively promoting bad solutions. They were rewarded for doing so. Here's the thing: in most cases, there really is no price to pay for backing a losing silver-bullet solution. Oh, those in front of the board have negative exposure when it all blows up. Some of the higher management tiers are at risk as well. But for most of IT, there simply is no bill to pay for being in the pack.

The Problem

Here's the thing: having Professional Integrity carries not-insignificant risks.

1. Having integrity makes you predictable. Those with an agenda can plan their actions confident of what you will do. That means that if things get nasty, you end up following Marquess of Queensberry rules in a bar fight.
2. Your compensation is in the hands of ignorant people. They don't know what you know. Many of them may be smarter than you. Even if they aren't, though, they are still the ones signing the paychecks.
3. There is seldom a single, obvious best answer. Honest and rational people can arrive at differing conclusions based on the same information (because they weight aspects of the problem domain differently). This can easily split the message received by management and open the door for bad solutions. This also leaves you vulnerable to the manipulative because they can confuse the issue at need.
4. Costs of bad solutions are often deferred while the costs of changing minds is immediate. Whether opposing a bad proposal or proposing a better process you are asking for change and that makes people uncomfortable. If you make people uncomfortable they will often hold you responsible for their discomfort.
5. Sometimes having professional integrity means looking for a new job. Whether you are fired or are forced to resign, some positions are simply incompatible with maintaining professional integrity. These situations are rare, but they can happen.
6. People aren't rational. As a result businesses often make choices that seem irrational. Sometimes those choices are right even though they are irrational. Professional Integrity in such a situation is problematic at best.

As I see it, Lidor's call to arms makes sense but is unlikely without more support. The thing is that the problem is bigger than he has acknowledged. He is asking us to take actions that are overtly risky and he hasn't given us enough of a reason to do so. I mean, the emperor may not have any clothes, but there's no telling how he'll react to you pointing that out.

The Solution Domain

"Because it's the right thing to do" resonates with most of us, I think. But it is important to acknowledge that, at heart, Lidor's plea is a moral and not a technical one. As such, it needs to be approached from a social, not a technical standpoint. The thing is, we are equipped to make this happen, but we need to explore it in the right context if we hope to make progress.

Societies have a lot of experience coercing correct behavior. There are a lot of tools available, some better than others. My preferences tend to flow from my LDS background. As such, I'd advocate proselytizing correct procedures. We should explain as clearly as we can the benefits of acting correctly. While the payoffs of acting correctly are long-term, they exist nevertheless and are provable and measurable. Professional Integrity in IT has similar long-term value (above and beyond correct practices) to businesses and those who learn to identify and value it will gain the benefits it brings. A company that hires professionals with integrity will do better than companies that hire similarly talented people who lack that dedication.

As long as we're borrowing from religious tradition, lets hit up the Catholics and/or Jews (leaving no stereotype unturned) and apply some well-earned shame. I know who sold us out at XanGo. My reaction to them in the future can (and should) be informed by their actions there. I think we do ourselves a disservice if we know compromised IT professionals and we don't tell them our opinion of their actions or make those opinions public. Repentance (more religious terminology, but applicable) should be possible, certainly--people can change and reformation should be encouraged. That said, it's important that forgiveness be contingent on honest efforts towards reparation and change.

On a more general level, ostracizing those who behave badly is one of society's greatest tools to restrain bad behavior. The next time I review resumes to evaluate new hire candidates, I'll have an eye out for those I know who, while they may be bright and talented developers, ended up endorsing those things that were bad for the business. More generally, I'll try to craft questions that explore a candidate's professional integrity in addition to their technical expertise. Reputations in IT should be more than merely technical. Professional Integrity is important not just to businesses but to the perception of IT as a whole. The IT crash in 2001 contained a lot of payback for abuses in IT during the build-up of the bubble. While we were on top many of us made unreasonable and expensive demands on companies (some personal, some technological) and those abuses weakened us as a group.

Collective bargaining is one route that I'm sure will come up--a union or standards body that enforces "correctness". Personally, I'm not a fan of that option. Bureaucracies are inefficient and tend to be in opposition to business interests. We do not need more antagonism from the executive suite. Unions also have a tendency to groupthink that I consider inimical to technology innovation.

I'm sure there are others that I am overlooking. I'd love to hear further thoughts in the comments. Please leave 'em if ya got 'em. Don't make me beg.

I acknowledge that my thoughts here will be hard to implement. They require conscious short-term effort with no promise of reward. It's certainly beyond the scope for a single individual to affect. Still, I believe it's possible to make things better than they are right now and that the potential for reward is both real and achievable. Plus, Paladins are made for exactly this kind of idealistic fight for what's right...