The latest .Net Rocks podcast was from a panel at TechEd 2008 (that I sadly missed while there). Richard Campbell was the moderator and the topic was Software Quality. It was a good, if somewhat one-sided discussion and I recommend giving it a listen if you are so inclined.

I particularly liked Billy Hollis' perspective on quality because it was a perspective that was grounded in both reality and sound business principles. He kept bringing up quality in terms of trade-offs and the others kept trying to waltz around those comments with a more absolutist, almost dogmatic, vision of software quality. I think it was a real shame that Billy Hollis was so out numbered because his points are important if we are to gain the stature we desire as a profession.

The final comments completely got on my nerves. Which is fortunate because it helped crystallize my discomfort with the others.

Billy Hollis started off with this simple statement.

My key insight that I try to communicate is it’s all a matter of trade-offs. And getting to what the user needs is first and quality is defined within that context.

To which Neil Roodyn responded.

I’ll disagree. It’s about quality. You’ve got to get your quality as high as you possibly can. Always. The higher the quality of the software, the higher the quality the software will remain through the development life cycle. So right from iteration zero keep that quality as high as you possibly can.

And Jeffrey Palermo added.

The higher the quality the better. You can’t have too high of quality, you would just say you have too much behavior for the situation.

The thing is that Hollis is right and the other two are not only wrong, but dangerously wrong. Other members of the panel (David Platt and Neelesh Kamkolkar) didn’t support Hollis either, though they weren’t as overt as Roodyn and Palermo.

What Are You Trading?

The thing is that software quality, besides being pretty much impossible to measure, isn’t a first-order good in the first place—i.e. it isn’t valuable in and of itself and delivers what value it adds indirectly. You can make the reality of this simple business fact plain by pointing out how the deciding factor between two competing products is often not quality. Betamax anyone? Rolls-Royce? Sun Microsystems?

People choose software for a lot of different reasons and those reasons are very rarely explicitly the “quality” of the software. Price, availability, familiarity, and usability all play roles in the purchase (or use/approval if internal) of software and these factors need to be kept in mind. It could be that it is better for your company to have software that is harder to maintain but completed two months earlier. I do this all the time because I have lots of projects that are single-use and with a restricted domain.

Software development involves time, money, commitment, and concentration. Spending those valuable resources should be done carefully and be based on actual value to the company. Quality has a cost. Quality doesn’t always have a pay-off. Business people have a term for a cost that doesn’t return a pay-off. That term is usually “good bye”.

Dangerous?

So Palermo and Roodyn are wrong, but why are they dangerous? This comes back to my observations during the dot com bust. I observed the glee in putting IT back in its place. You see, during that whole tech bubble IT got arrogant. Technical folks were wagging the business dog, dictating strategy, policies, spending, and sometimes even remapping the boardroom to their liking. Which would have been fine if the business rules really had changed as some thought. Unfortunately, they hadn’t. This arrogance set the entire profession back because business leaders learned not only of our arrogance, but of our ignorance as well.

We don’t bother learning even basic business principles before we make important, even critical, business decisions. This discussion of quality is an example of just such thinking. Roodyn and Palermo are talking about dictating business resources without any concern for accommodating business realities. Worse, they don’t seem interested in even learning about or exploring the business realities they, of necessity, work under.

As much as we might like the universe to bend to our personal theories and values, the fact of the matter remains that there are a lot of reasons to compromise aesthetical purity when the rubber meets the road. Those reasons don’t simply go away just because they make us uncomfortable. Indeed, how we react to the inevitable trade-offs required will largely determine how much business leaders decide to trust us in the future.

Here’s a helpful exercise. Imagine you are a business leader evaluating which company to hire to solve some software need. As part of your vetting process, you ask two finalists their stance on software quality. The first one answers with “Getting you the things you need is our first priority and quality will be defined in that context.” The second one answers with “Quality is number one. Always. Quality cannot be too high though it could be that you want too much behavior for the situation.” Which would you hire? If you answered the second, it is because of ideological blindness. Actual business leaders will not only tend to go with the first, but will have confidence and trust in doing so.

Are You Agile?

A lot of the value in IT comes from giving businesses flexibility. Processes that used to take indoctrinating a new generation of workers can now turn on a software release. While this is both a blessing and a curse for businesses, it remains a key to our value in IT. Businesses have learned to be wary of our use of resources, however, and with some cause. It is unfortunate that they cannot trust IT, but until we let the needs of the business drive our development I’d say that their trust hasn’t yet been earned. If we really want to be masters of our own fate, we must learn to live in the messy world of business. We can only do so by proving that we will use the flexibility inherent in our profession wisely.

I found Scott Adams' (yes, that Scott Adams) blog post today about The Power of Choice interesting. Particularly at the end where he says this:

The next time your mate or co-worker is butting heads with you over a decision, recast the situation as their choice.

For example, let’s say you favor Option A, and someone else wants Option B for reasons that seem to you irrational. You are at an impasse. Change the question to this:

“Okay, do you want Option A with this risk, or do you want Option B with this other risk? It’s your call.”

When you put things in the form of a choice, sometimes it gives people the only thing they wanted in the first place.

I’ve been using this technique in software development for years with great success but it takes some care. The thing is that "with this risk" and "with this other risk" buries some real temptation to undermine the actual choice. An example of undermining the choice:

• Would you like to use Linux which is free and has broad support and experts available and eager to help where you can have access to the actual developers to fix any trouble we run into?

or

• Would you like to send hundreds of dollars per seat to a corporation that lies and cheats and whose software is insecure, buggy, and closed?

This isn’t a choice, this is merely a statement of your own prejudice. It is a dishonest representation of the alternatives. Presenting a choice like this is a sure-fire way to alienate opposition and polarize discussion. Worse, it undermines trust.

True Choice

Here’s a key I’ve found in offering a client, customer, or business manager a choice: be prepared and honestly reconciled to following either path. This includes an honest and complete evaluation of the options. This preparation manifests as sincerity and is easily sensed by those you work with. The more ego you have invested in one of the options, the harder this becomes. I find it useful to practice not tying my ego up in technical topics and decisions. Non-technical people find this refreshing. Sadly.

The key to offering sincere choice is that it invokes an effect I described earlier this year in Winning Arguments—putting everyone on the same side.

Anything is Possible

One technique I’ve found useful in exploring software development options is to preempt speculation. When a business person comes to me and starts off with "Can we..." I’ll snap off a quick "we can do anything." This statement has the benefit of being absolutely true with the added bonus of turning the discussion to resources, requirements, and trade-offs. There’s no downside there.

"Can we implement a shopping cart like Amazon.com’s so that our customers can get recommendations and save shopping-carts?"

"Absolutely. We can do anything."

Unless they’ve asked me broad questions like this before, this response is a stumper. They are expecting hemming and hawing and evasions. They are expecting opposition. This statement turns their expected opposition on its head. It gets them thinking automatically about why we can’t do this amazing thing.

And here’s a blunt truth: nine times in ten they haven’t really considered their question at all. Consciously or not, they’re trying to get you to do their work for them. Preempt that by giving them as much as they are giving you—an answer requiring as much thought as they put into their question.

Now, you can make this sincere (and less antagonistic) by following up. If they don’t follow up on their own, I’ll add "Let’s explore what it would take." Notice the "Let’s". i.e. we’re working together to explore our resources, requirements, and options.

It doesn’t take long for people I’ve done this to before to start thinking about resources and requirements before they approach me. The sooner you can get people thinking in terms of trade-offs in software development discussions, the sooner you can get to the meat of those discussions.

And really, every software development discussion is all about trade-offs. That’s because we really can do anything with software—if it can be conceived, it can be done.

I listened to a recent .Net Rocks podcast about Domain Driven Design. Now, despite reminding me of a post last week by Secret Geek about how everything is driving design these days, I think a lot of what was said makes sense to me. Eric Evans' point on the podcast seemed to revolve around learning a given domain and letting experience with the domain drive development decisions.

From the contents of his book at Amazon, it looks like Eric's idea is a modification or extension of Model Driven Design—i.e. an attempt to unify development efforts within a descriptive framework approachable by non-technical entities. In this case, letting the needs of the domain determine the focus of development. For example, he talked near the end about how it is often more valuable to leave legacy code alone (maybe wrapping it with a translation layer) in order to spend resources on the things that will provide better value than you can achieve by replacing the legacy application.

Good Enough

Eric also used a phrase that I listen for as a flag that identifies someone to take seriously: "good enough". Good enough is a phrase used by people who are actually trying to apply YAGNI as more than a way to win an argument. Good enough is a limiting term, a cut-off, a phrase that gets you from the isolation of the silicon tower to actually developing software.

Now, I have to be careful here because "good enough" is also useful for burying dangerous assumptions. I think that Eric is an example because he never elaborates on what something is good enough for. Given that he's explaining Domain-Driven Design, you could assume he's saying that it's good enough for the domain, but that doesn't map. Nobody works for a domain.

Pandering to Business Managers

Which got me thinking. Eric is talking about developers and domain experts creating a common framework for communication, but really, starting at the domain elides over the most important consideration in software development: the business. The context for "good enough" should be rooted in tangible benefit for your company.

Naturally, I thought I would be all clever-like and create another "driven". Unfortunately, IBM already mucked this one up. In what looks like naked pandering to clueless business managers, IBM proposed "Business-Driven Development" in December of 2005. I mean, can you take any paper seriously that contains this quote:

The inherent problem with the enterprise software development process is that it suffers from a lack of agility to match the pace at which the business needs to change in order to keep up with the market trends and competition.

Now, you could make a case for market trends and competition being pretty malleable—even to the point of outpacing the ability of software development to keep up—but that's kind of beside the point. The fact of the matter is that of all the things a business needs to change to adopt a new business process, software is the most adaptable.

Functionally, IBM's proposal is a way to isolate software development and assert management control. Under scrutiny, it looks to me like an attempt to force agile iterative principles into a BDUF format that IBM's Rational products can actually handle.

Dollar-Driven Development

What I'd like to see is Eric's domain drivers better focused as a responsibility towards whoever is underwriting development. This is hard to name because his principles are broadly applicable for business, government or even open source.

Still, somebody is fiscally responsible for software development and it is that responsibility that should drive development decisions--i.e. the best interests of the entity underwriting it. For open source projects, that would be the interests of the benevolent dictator in charge. For business development, that would be the best interest of the business you work for.

This is easier said than done. In order to work for something's best interest, you really have to know its interests well. I think that may be the heart of what Eric was saying about learning the concerns of a domain and letting those concerns drive development. It means developing expertise in a domain that may or may not be of your choosing. This takes time, effort, and access to domain experts. It means delving deeply into the reasons things are done now and the problems current processes were meant to solve.

Mainly, it means learning about things that aren't computers and likely have nothing to do with them. Whether it's a bank, talent agency, or bail-bondsman, it's not going to be something that you learned in school. It's yet another thing you have to learn on top of all the technology stuff that keeps changing.

Good thing you're good at learning things.  Right?

Developers hate it when someone changes the requirements in the middle of a development project. What few have realized yet is that they've gone and changed the requirements to be a developer right in the middle of our careers.

An Unnecessary, But Illustrative, Story

The summer of 1994 found me in Mesa, Arizona, new-minted diploma in hand, getting ready for my first real job. A couple days before I was to report for the new position, I received news that the job that I had pulled my wife and baby daughter into the gods-forsaken desert for had, well, been eliminated. Not a happy piece of news to receive. Since my parents lived there in Mesa we ended up taking refuge in the familial homestead. I love my parents, don't get me wrong, but this was not a celebrated event—by any of us.

This was apparently all the impetus that my dad needed to leave the largish firm he worked for as the main estate attorney. He asked me to help him automate the documents he used to create trusts and wills. For money. This was the start of my transition from an English B.A. to a computer programmer. Who knew you could make money with computers?!?

The Point

My dad found that having much of the drudge work in creating those documents automated allowed him to concentrate on larger aspects of estate law. He was able to grapple with the greater context and put things together in a way he hadn't been able to before. This change in focus helped both his practice and his clients because he was able to provide better service at a cheaper cost and still make more money himself.

So here's what this has to do with software development: advances in tools, design patterns, and technologies have created the same shift for software developers that my dad realized as an attorney. While the occasional Luddite might quail at the thought of leaving vi for an IDE with built-in debugger support, most developers are happy to move away from the drudge work that has preoccupied us for so long (as with any boost in efficiency the expected new leisure has yet to show up).

In fact, what we find now is that developers are expected to take on more and more responsibility. This is a good thing overall because increased responsibility == increased value (and hence, generally, salary). This can be a bad thing, however, if software developers neglect some of the new skills that we need in order to handle these responsibilities well.

This is what lies at the heart of recent calls for developers to learn "soft" skills and develop better communication. That's because, unless you work in some back-water in the benighted hinterlands, the days of the cave-troll coder are over. A developer can no longer count on being isolated from users, managers, or <shudder> marketing weasels.

What Everybody Seems to be Missing

All of this is well and good and recognized by those who are paying attention. But it's not enough.

I like to compare business software development with practicing business law because they are both logical disciplines that have to support the human decision-making processes that are inherent in running a business. There's one large difference between the two that I think is important—lawyers are taught the fundamentals of business as a part of their law degree. They don't have an MBA or anything, but they do attend classes explaining typical business structures and basic accounting principles. That's because doing so helps them both with their interface to the business and in their ability to deliver work that properly supports the companies they work for or with.

Well, software developers are increasingly in a similar relationship with the companies they work for. I'm fortunate in my English degree because I'm reasonably well set up in the whole soft/communication skills area. What I'm finding lately, though, is that the courses I took in Business, Economics and Accounting (I never did figure out what I wanted to major in) are becoming crucial to my ability to do my job as a software developer. Understanding double-book accounting and GAAP help me understand the needs of the finance guys. Understanding supply-chain and basic inventory control helps me understand the operations folks.

I'm not talking here about becoming an expert in business. What I'm saying is that developers need to take the time to make sure that they understand the reasons behind the weird and wacky things that other business units are doing (and requesting that we automate).

And it wouldn't hurt to take a formal class or two.

Too often, the software developer going over the current supply-chain configuration with an operations manager will jump to conclusions, or worse, offer "helpful" suggestions about how things can be done more logically. More often than not, we come across as isolated from the realities that the current configuration was created to overcome. Or worse, we come across as judgmental jerks.

It's a drag having to learn so much that was originally considered outside of our discipline. Unless you're a hardware or other highly-specialized developer, however, it's going to turn out to be the kind of thing that sets the competent apart from the competence-challenged.

Steve Harman had a post back at the beginning of the month about stuff you'd tell a young developer. It's a reaction to a similar piece by Jeremy Allison. It's an interesting topic, so I thought I'd waste a few pixels on it myself.

If it's not what you love, don't do it

I wouldn't generalize this to other fields, but for software development, I think this is a good thing to keep in mind. A lot has been made in the past by career counselors and other gurus about "finding your bliss" or similar nonsense. I think that's mostly a crock. You can be a good doctor, mechanic, or professor without liking what you do. Sure, those who love what they do will tend to excel at it and rise to the top of their field, but you have to ask yourself how much better off people at the top of a field are vs. those who are merely good enough.

That said, I think software development is somewhat unique as a career. Software development advances incredibly fast and keeping up with new technologies and ways of doing things takes a non-trivial amount of effort. You have to paddle hard just to stay even and that means experimenting and learning on your own. You could probably do fine if you have Arnold Schwarzenegger level drive, but short of that, loving what you do is about the only way to get you where you need to be.

Reputation is important

People who have been around the block know that the difference between developers can be extreme. Top developers are 5 to 20 times more productive than their peers. Given some of the complete incompetents I've known, I suspect that there's really no upper limit on that number.

Unfortunately, both Jeremy and Steve use this point to jump into the virtues of Open Source software. Open source contributions can enhance your reputation, but that has nothing to do with people being able to look at your code—it's more about working with other developers who can then form their own opinions of you as a developer. That's helpful, but limited.

A good reputation is someone talking about how well you did a job for them. It's your name coming up when someone asks "do you know any good developers?" A good reputation is more important than mere code sample availability and the wider your reputation can penetrate the better off you are.

How important is reputation? Let me put it this way: I haven't gotten a job from anyone who didn't know somebody I'd worked with since 1997. Your reputation and your skills are your only true security in this field.

Never stop asking why

The vast majority of people in any field are content to learn what they have to do to get the job done. Top-tier people are the ones who are asking pesky, even impertinent, questions all the time. Jeremy puts this under the heading of "Learn the architecture of the machine" but that's too limiting. Learn as much as you possibly can about why and how things come together to get the job done.

There's two parts to this, really. The first is that you don't stop just because you got something working. Find out why iterating backwards through an array fixed your bug. Look for the best event to use for control loading rather than just the first one you found that happens to work. Learning how to parse technical documents, navigate help files, and put a concise Google query together are key skills you'll need to master.

The second part of this is harder: if you don't know something people are talking about, ask. This becomes harder to do over time because you start believing your own hype. It's easy to believe that inner voice that's saying you can bluff your way through this conversation and look up anything important later. The thing is, the really bright people, the ones you most want to impress, aren't going to be fooled. I've also found that top technologists enjoy getting intelligent questions almost as much as they love answering them. Note the qualifier there and spend the mental effort to make your questions intelligent, though.

Your employer is more important than the community

This goes directly counter to both Jeremy and Steve and I'm going to tank any chance I have with digg or reddit, but it needs to be said.

Jeremy looks like he's coming from that staunch anti-corporation Slashdot tradition that talks in moral terms about the beauty of open source software and the depravity of Microsoft. Reality check: it is not a common goal of open source developers to create "the greatest, most beautiful work of art that all of you can create together." That's certainly not why I contribute to open source projects.

Open source developers are no more virtuous than corporate developers. While you may program for fun and the satisfaction of a job well done, the choice of project you work on is inherently a selfish one. Always. Whether you program for a paycheck, community recognition, or because you need working blog software that you can tweak when you want to your choice is based on internal needs, not virtues.

If they're both selfish, shouldn't I be saying that they're equally important? I don't think so and it's not just because a company will help support your family. The reason your company is more important is that your company has to compete in the market place. It has no choice. Because a company has to charge money for its products or services, the company is forced to provide things that other people want badly enough to part with their hard-earned cash. In practical terms, it means that a company has a reality check that can't be ignored because sooner or later they're going to have to go out there and prove themselves in the marketplace. While the tech bubble bursting was tremendously painful when it happened, imagine the crap pile we would have if it didn't burst. Ever.

In addition, companies force you, the developer, to do work that you don't want to do. Developers hate that, which is perfectly understandable. Being forced to work with unreasonable managers, clueless users, and hopelessly disconnected marketing weenies is a drag. If you're good, though, it's also a rich source of opportunities to tackle things that you might never have thought needed tackling. And it forces you to justify your dogma of choice to a group of supremely skeptical, self-confident people who have the welfare of their families on the line. Yeah, that's not fun, but it means that you'll have to learn to understand and articulate things you just know are true. With the added bonus that you'll find that they sometimes aren't so true after all.

Finally, be wary of advice from old timers

Not me, of course. My advice is nigh infallible. Just ask me, I'll tell you. If you go awry following my advice, it's probably because of errors in implementation.

Okay, maybe not. I've blogged about recent errors I've made, so I have to assume there will be others. Be careful about who you listen to. Be aware that everybody has their own hobby horses and pet peeves and it's a human trait to cram those prejudices into everything (particularly advice). Also be aware that internet communities trend towards consensus as much as any other human community does. Consensus is dangerous and will tend to mask assumptions and pitfalls. Develop a well-honed BS filter and use it liberally.

While you're at it, the sooner you learn to pass your own ideas and theories through a well-developed skepticism the better off you'll be.

Have you ever found yourself in a situation where you know what's the best thing to do, but are unable to convince anyone else that you are right? Developers know that even simple problems have more than one solution. Developers who have worked on a team of more than one have probably been in a situation where they just knew that the team was heading in the wrong direction and that they had a solution that was more elegant, easier to program, and better to maintain.

Higher profile developers often find themselves trying to explain their solutions to non-technical people as well; sometimes before development has begun, but sometimes after it is truly too late to do anything different.

So what do you do when you have to butt heads with buttheads? Here is my secret to winning in these situations

Geek Whisperer

As a developer and/or manager of developers, it is astonishingly easy to find yourself trying to explain concepts to people who, however smart, just aren't equipped to understand what the heck you are talking about. This is an uncomfortable position for all involved.

It is tempting in these situations to pull out a call to authority and tell people that you are the subject matter expert and they simply have to trust your expertise. This works fine for the little stuff, but as the scope increases, so too does the reluctance to put up with this answer.

And with good reason. Technologists can find themselves literally with the fate of the company in their hands. Business leaders find that hard to take and are going to be extremely wary of any solution that involves the fate of their company.

How to Win With Business Folk

There is really only one way to handle this situation: concentrate on what is best for the company.

Everything you say and do in this situation should start and end with what is best for the company. The magic of this orientation is that you can defeat every argument and diffuse even personal attacks by leading back to what's best for the company. No matter how irrational the objection, how confusing the question, or how deliberate the ignorance, you'll lower tension and create space for a solution if you can turn back to what is best for the company. Even asking the question, "What course of action would be best for the company?" can be sufficient to redirect an out-of-control or confrontational situation.

Talk Amongst Yourselves

Developers tend to believe that they live in a rational universe—or at least that their work is based on the application of structured reason to provide solutions to defined problems. This belief is shared, more or less, among all developers—even those who believe themselves to be the only developer who is actually rational.

This makes it both easier and harder to solve disagreements between developers. The fundamental belief in reason tends to keep disagreements from being personal, so you'll find most developers are willing to discuss alternatives and the ramifications of design decisions.

Every now and then, however, it is possible to find yourself at an impasse with another developer (or group of developers). In an impasse, developers can become more intractable than business folk if only because you lose the ability to appeal to consensus. Rational is rational just as right is right. The best rational choice remains the best rational choice no matter how many delusional fools find themselves unable to comprehend it.

How to Win With Technical Folk

There is really only one way to deal with this situation: concentrate on what is best for the company.

When two (or more) sides have presented what they feel is the most rational solution to a problem and remain at an impasse, the source of contention almost inevitably lies in unstated assumptions. In this situation, asking the question, "Why is this solution better for the company?" helps to ferret out those assumptions. Some disagreements will need to apply this question repeatedly as it is not uncommon for assumptions to be stacked on other assumptions. Ferreting out the source of your assumptions can be tiring, but in the end well worth the effort.

Open Sores

But what if you aren't developing as a part of a company? The principle remains the same as does the effect. Every open source project has a problem that it is trying to solve and people working on that project will tend to be there because they believe they can help solve that problem. Leading disagreements back to the reason of a project's existence can help straighten out priorities and direct solutions back to the original problem domain (and away from the individuals involved in the discussion).

Winning Everywhere!

Is there application for this question outside of geekdom? Darn skippy there is. In fact, asking "what is best for the company?" is a technique I learned at a leadership conference when I was more manager than developer. It turns out that any time people group up, they'll tend to have a reason for doing so. In fact, "tend to" == "always" in this case. The reason can be explicit or implicit, but people simply don't group up for no reason. Learning to find that reason and drive disagreements back to it helps keep you focused and gives you the opportunity to pick your battles.

A Warning

Asking what's best for your company/group/whatever will often show you the flaws in your own arguments, particularly as you first start applying the question. If you want to win for personal egotistical reasons then this technique isn't going to do you much good.

In fact, ego can destroy the benefits of this technique more generally. Depending on how much power the person with the ego has in the organization, their presence can significantly alter the ability of this technique to work. An example: I once worked for a company where the owners truly did not care what was best for the company. As such, arguing from the standpoint of what was best for the company had no effect whatsoever.

Fortunately, reality has a way of asserting itself in these situations. Companies that can't act for their own best interest tend to disappear, members who don't care about what's best for the group tend to leave the group, and employees who don't care about the best interest of their company tend to find themselves parted from the company.

Still, there can be significant discomfort in the meantime and heaven knows that "tends to" != "will" in this case. You'll have to decide how you approach a situation where ego is going to be a problem. It's almost always best to separate yourself from the ego getting in the way—particularly if it is your own.

"So when will you be done with this development project?"

I don't know about you, but I hate this question. There simply is no good answer for it. It seems like such a simple question with a simple DateTime valued answer. One of these days I swear I'll answer with, "Oh, I'll be done next Tuesday at 2:34pm." just to see what happens.

And seriously, businesses hate that we have such difficulty answering the question. It seems perfectly reasonable for them to want to know when they can plan to have the new processes that they know they desperately need. Developers demand high salaries and are ostensibly professionals, they should be able to give a professional answer, right?

The Road is Well Paved

The thing is, software development is a lot harder than people expect it to be--and this includes software professionals. Even simple software projects can run afoul of hidden complexities that can destroy well meaning estimates and make everyone unhappy. And no matter how you hedge your answers, people simply don't remember all your caveats, maybes, and what ifs that you use to indicate uncertainty.

The end result is that developers seldom make their ship-by dates and companies become disillusioned and impatient with all software development. That's not helpful for anybody, but it's pretty much the rule anymore.

And the fact of the matter is that the vast majority of developers (and development managers) never learn how to answer the estimate question. They'll move from company to company, repeating the cycle of hope, suspicion, and disappointment over and over again. Which works well enough for the developers in the boom times when the demand for development is so high that mildly talented house plants can get hired as developers.

So a lot of people are making the same mistakes over and over. Businesses can be excused for assuming that this is simply the way things are and feel confident in their distrust of software professionals. They've been there, done that, bought the t-shirt.

Paying the Toll

This environment causes developers who care about these kinds of things a lot of heartburn. Everyone pays for the ongoing cycle of disillusionment. I believe that this is what really prompts posts like the recent ones from Ted Neward talking about professional ethics. And I've been known to throw my own hat into the ring as well.

We get tired of paying for the sins of those who have gone before. And I'm not referring to the messed up legacy code we stumble into, either. Frankly, messed up code is the least of your problems coming into a situation with a client who has been burned by previous developer promises. Companies that have had deadline after deadline missed have a degree of mistrust that is very hard to overcome.

We pay for this distrust in a hundred different ways. The thing is, trust is a paying commodity in business. Working with partners you trust means a whole lot of overhead you can simply skip. An analogy: if I trust a plumber to fix my sink quickly and professionally, I can go get a burger and leave him to it. It's only when I don't have that trust that I have to pay the additional overhead of having someone I do trust watching to make sure he's not napping under the sink.

Want to see a business manager go into a dreamy fantasy? Ask them what it'd be like to be able to trust their software developers (in house or not). The more experience they've had with developers the more intense the fantasy.

The Rubber Meets the Road

We have a couple of areas of friction in businesses that exacerbate this situation. The main disconnect with business managers is that we have borrowed terminology and tools from other disciplines without understanding that our processes are fundamentally different. It's tricky because the temptation to use manufacturing terminology is immense. After all, we are creating a product of sorts. This makes so much sense on an intuitive level that it's hard to realize that the comparison is misleading and potentially dangerous.

I wish we could retrain everyone to make analogies to other business specialties. Scientific research or law come to mind as potentially useful analogies because both are similarly plagued by the impact of unique situations, changing ground rules, and unforeseen complexities. It would be interesting to investigate how managing software development like a patent application or drug research would change how we look at the problems involved. We might have stumbled onto iterative cycles and responding to altered requirements a whole lot sooner, for example.

Paying Attention

The real problem, though, is that most developers (and even most development managers) don't take the time to learn about common friction points. Nor do they take the time to build relations with their business counterparts so that you have some political capital (aka trust) to use when it is needed. It's easy to forget that much of the progress in software development practices are pretty recent in terms of business processes. After all, business managers don't move at the speed of light and changes tend to take time to penetrate those layers.

Which means that a whole lot of industry advances aren't even theory yet in the board room.

And the fact of the matter is that you cannot expect a business manager to understand what makes Agile practices work. Or the reason that strong unit testing saves time over the long run even though it takes more time up front. Learning to communicate at a level that is sufficiently detailed for smart business decisions without getting bogged down into the jargon inherent in any specialty is an invaluable skill, and one best learned earlier than later. That means thoroughly understanding those theories yourself--not just on the surface or in buzzword compliance. It also means learning to communicate that understanding from orbit, 30,000 ft, 5,000 ft, and right on the ground. This is hard to do. It takes practice. It also takes exposure to business manager types. I'm not sure which is harder...

Something to think about, though: not learning this skill leaves you at the mercy of those who do learn it.

My point, though, is that it takes both. You have to learn your profession so thoroughly that you can deconstruct its "best practices" ("design patterns", whatever) and rebuild them from basic principles on the fly. AND you have to learn to communicate that understanding comfortably to people of varying familiarity with software development in a business environment.

That's what it takes to be a true professional. It's easy to let those two skills fall out of balance. Individuals who understand both are invaluable to a company. Also rare. Companies who discover someone capable of both are often surprised at how much smoother things run with that person placed where they can do the most good--a point Jeff Atwood's latest on becoming a better programmer drives home.

So I don't have a formula for quick and accurate estimates. Just a lot of hard work. Still, here's a tip for free: anyone asking for a firm delivery date is inherently assuming BDUF. Once you know that, you know where to start your answer.