In many companies developer career progression is deceptively straight-forward; Jr. Programmer, Programmer, Sr. Programmer, Team Lead, Architect, Sr. Architect, Bob (Bob being the semi-mythical entity referred to in obscure comments, worshipped by now-extinct aboriginal tribes, and rumored to haunt the sub-sub-basement).

The differentiation between these positions starts off with how much you know. A Sr. Programmer is a Jr. Programmer who knows his tools inside and out and can complete assigned tasks quickly and without a lot of supervision. Around Team Lead time, however, progression stops being about what you know and starts revolving around your ability to choose wisely between competing trade-offs.

This is an easy transition to miss if you aren’t paying attention. Indeed, I’ve known "Architects" that I wouldn’t ask to design a "Hello World" application. Because being a good architect is about developing good judgement, there are two things that are essential to becoming and remaining competent.

Know Your Stuff

Yeah, I know I just said that being an architect isn’t about what you know. That’s only slightly true. What changes between Sr. Programmer and Architect is the depth of your knowledge—you have to penetrate to the reasons and complexities behind patterns, practices, and technologies. You go from knowing what an n-tier architecture is (and how to create and maintain one) to understanding why that architecture was invented, what problems it solves, and what costs are involved in creating and maintaining one.

Take, for example, the "architecture" I found in coming to a new position. A number of projects are used for this intranet application, each in its own application space. Here’s a short illustration of the structure of two of the solutions.

As you might guess, BOL stands for "Business Object Layer", DAL for "Data Access Layer", and OBJ for "Objects". You might not be surprised, given the closely related project names, to learn that each project contains classes that exist in the other. In short, despite having the structure and using the terminology of n-tier architecture, this isn’t actually anything of the kind.

The developer responsible for this design is not an architect, no matter how much experience he may have, because he has failed to understand when and why you would use an n-tier architecture. Understanding the reasons behind n-tier architecture would make it impossible to create this structure.

Understanding architecture takes hard work. It requires more than reading blogs, attending conferences or purchasing books (though each of those are certainly useful in developing understanding). True understanding requires thinking things through and asking tough questions. It requires breaking down concepts until you can identify and understand their component parts. Often, it takes asking the questions that risk confronting a room full of people glaring at you in varying degrees of shock. Most of all, it takes admitting that you might not know something and working to rectify that lack—or, harder still, being willing to re-examine assumptions (or even conclusions) when presented with new information.

The Development Twins: Cost and Benefit

It’s easy to forget that every technology, architecture, and practice has two sides; a cost and a benefit. To borrow from Heinlein, "There ain’t no such thing as a free lunch." Because making choices is hard, it’s easy for people to remember only one side of a given technology, architecture, or practice—to see only the cost or only the benefit.

This is particularly true in new processes or designs. We go through so many silver bullets at least partially because people teaching new processes tend to concentrate on benefits. This is understandable even if the presenter isn’t financially tied to your adoption of the Great New Thing™ because the benefits are the reason the Great New Thing™ exists. This means that you’ll be taught the benefits, but you’ll typically have to discover the costs on your own.

It’s also important to bear in mind that developers have a natural bias towards solving problems, not finding them which leads to a tendency to grasp benefits quickly. Because architects are grown from developers, it’s important to examine and balance this potential optimistic bias. And it is a balance. Ask an average Linux geek about Microsoft if you want to see a bias towards cost.

The Scope Multiplier

A project’s scope has a multiplicative effect on both cost and benefit. Understanding this became explicit for me in a reply I made to a recent comment by Udi Dahan.

I’d be willing to bet that the benefits of DI, SoC and other good design practices increase exponentially with the number of developers while their costs are a linear progression. Huh. That bears some thought...

For any given process, you need to understand how both benefit and cost are affected by the scope of your project. Practices that scale benefits exponentially and costs linearly are excellent when the scope is large. Indeed, the larger the scope, the better the net gain from applying the given practice.

A corollary of that evaluation is that practices designed for large scales are less likely to provide a net benefit in a small team with a small project.

It’s possible that project scope can be calculated separately for project complexity and team size, but I’d have to give that some more thought. Since high complexity projects tend to require larger team sizes, you’d seldom have one without the other (making this a moot conjecture). That said, my initial instinct is that something like n-tier is more important vs. complexity whereas the benefits of SoC scale more based on team size (though it helps mitigate complexity as well). It is possible that this difference may play an important role in how and when you’d apply each.

Time

That pesky fourth dimension can also have a dramatic effect on cost and benefit. A quick down-and-dirty project with limited scope may seem like it wouldn’t benefit from any real architecture at all. That evaluation can change drastically if the project is likely to change a lot over time. Most developers have been ambushed by application maintenance at some point because they hadn’t planned adequately for change.

Predicting the future is a dubious pursuit at best, however, so be careful here. There’s a reason YAGNI became a well-recognized acronym. Don’t forget that a project can be refactored into a useful pattern after the fact if circumstances change to warrant it—particularly if you have prepared for easy refactoring (unit testing, anyone?).

The biggest problem with evaluating the effect of time on your project is that it is literally impossible to measure the impact of the choice not taken. As a result, I’m always very careful when I hear something like "If only we had..." or "Good thing we...". Personal, anecdotal experience is error prone at best—something to keep in mind during project postmortems.

The Courage to Choose

The most important thing an architect does is decide. The architect chooses the technologies, the patterns, and the practices to apply to a given software development problem knowing that success and failure can hang on the effects of those choices. That’s a lot of responsibility. The impulse to push your choice off to a vendor, or some guru’s public statements, or industry consensus is a seductive one that can work for a time. Reality being the harsh b**tch she is, however, it isn’t a safe impulse to indulge. Personally, I prefer to hold my fate in my own hands.

I found Scott Adams' (yes, that Scott Adams) blog post today about The Power of Choice interesting. Particularly at the end where he says this:

The next time your mate or co-worker is butting heads with you over a decision, recast the situation as their choice.

For example, let’s say you favor Option A, and someone else wants Option B for reasons that seem to you irrational. You are at an impasse. Change the question to this:

“Okay, do you want Option A with this risk, or do you want Option B with this other risk? It’s your call.”

When you put things in the form of a choice, sometimes it gives people the only thing they wanted in the first place.

I’ve been using this technique in software development for years with great success but it takes some care. The thing is that "with this risk" and "with this other risk" buries some real temptation to undermine the actual choice. An example of undermining the choice:

• Would you like to use Linux which is free and has broad support and experts available and eager to help where you can have access to the actual developers to fix any trouble we run into?

or

• Would you like to send hundreds of dollars per seat to a corporation that lies and cheats and whose software is insecure, buggy, and closed?

This isn’t a choice, this is merely a statement of your own prejudice. It is a dishonest representation of the alternatives. Presenting a choice like this is a sure-fire way to alienate opposition and polarize discussion. Worse, it undermines trust.

True Choice

Here’s a key I’ve found in offering a client, customer, or business manager a choice: be prepared and honestly reconciled to following either path. This includes an honest and complete evaluation of the options. This preparation manifests as sincerity and is easily sensed by those you work with. The more ego you have invested in one of the options, the harder this becomes. I find it useful to practice not tying my ego up in technical topics and decisions. Non-technical people find this refreshing. Sadly.

The key to offering sincere choice is that it invokes an effect I described earlier this year in Winning Arguments—putting everyone on the same side.

Anything is Possible

One technique I’ve found useful in exploring software development options is to preempt speculation. When a business person comes to me and starts off with "Can we..." I’ll snap off a quick "we can do anything." This statement has the benefit of being absolutely true with the added bonus of turning the discussion to resources, requirements, and trade-offs. There’s no downside there.

"Can we implement a shopping cart like Amazon.com’s so that our customers can get recommendations and save shopping-carts?"

"Absolutely. We can do anything."

Unless they’ve asked me broad questions like this before, this response is a stumper. They are expecting hemming and hawing and evasions. They are expecting opposition. This statement turns their expected opposition on its head. It gets them thinking automatically about why we can’t do this amazing thing.

And here’s a blunt truth: nine times in ten they haven’t really considered their question at all. Consciously or not, they’re trying to get you to do their work for them. Preempt that by giving them as much as they are giving you—an answer requiring as much thought as they put into their question.

Now, you can make this sincere (and less antagonistic) by following up. If they don’t follow up on their own, I’ll add "Let’s explore what it would take." Notice the "Let’s". i.e. we’re working together to explore our resources, requirements, and options.

It doesn’t take long for people I’ve done this to before to start thinking about resources and requirements before they approach me. The sooner you can get people thinking in terms of trade-offs in software development discussions, the sooner you can get to the meat of those discussions.

And really, every software development discussion is all about trade-offs. That’s because we really can do anything with software—if it can be conceived, it can be done.

I wrote last month about winning arguments in IT. Earlier this week, Phil Haack asked a question (through Twitter) about things he could do to help convince a company to create an in-house QA department. Well, it turns out that I did exactly that at XanGo—successfully pushed for and oversaw the installation of an in-house QA department. I thought it might be a useful follow-up to the previous post to use this as an example of how I "won" that argument.

Concentrate on What is Best for the Company

This is the key, the whole key and nothing but the key to winning corporate arguments. You should approach any problem from this perspective. In this case, I have seen what software development looks like with a competent QA team and I wanted that again. Simple developer testing was leaving holes and we didn't have a consistent deployment model. I was sure that having an in-house QA team would benefit the company.

Now, since I was the Software Development Manager, instantiating a QA team was likely out of the scope of my responsibilities. Fortunately, I had a good relationship with my boss and he was the functional IT Director. It was well within his scope. This gave me my first temptation: the desire to expand my personal empire by creating a "QA Team" within my department. After all, it was my idea, it would take a lot of effort on my part to bring about, and QA is associated with development, right?

Tempting as it is, though, it would definitely not be to the company's benefit. A QA team should have as much power as they can get because they are the last rational review before stuff goes into production. What that means in practical terms is that they should be a first-class IT member. If other departments can override QA, you are headed for a lot of pain.

Which is when I realized that what I had to do first is marshal all the reasons I could think of that an in-house QA team would benefit XanGo. This is probably really what Phil was looking for. It's easy to come up with reasons it'd be good from your own perspective. It's a lot harder to broaden that perspective to the company as a whole.

Here's what a list of personal benefits might look like:

1. Backstop development so that buggy code doesn't hit production.
2. Help train developers by pointing out errors early enough that it is still in the mental cache of the developer who created it.
3. Give me a blame-shield if something broke.

Here's some things that are more useful from the company's perspective:

1. Backstop development so that buggy code doesn't hit production.
2. Help train developers by providing timely feedback.
3. Ensure that customers don't encounter broken processes.
4. Ensure that employee bug reports and feature requests are tracked, prioritized, and scheduled.
5. Develop and own a formal build/deploy process that is repeatable and automated.
6. Increase executive confidence that software releases do the things they are promising people it does.
7. Help identify training opportunities for development.
8. Help identify training opportunities for non-development staff.
9. Create a manager who can concentrate on identifying people with QA skills and provide a career path for them.
10. We have a Software Development Manager who knows what a good QA team looks like and how to create one and we can easily leverage his expertise without bringing in expensive consultants.

This list was my external list of arguments. Which ones I'd present depended heavily on a given audience. With executives, for example, I'd start with #3, glide through #4, #1 and #7 and then slam into #6. I'd pause there and ask if there was interest. If I had their interest (and I guarantee that I did), I could cover the other points in more detail--generally in a follow-up, formal meeting.

Show Me the Money

Reason is all well and good, but without something concrete and measurable, we're still in the realm of faith. Yeah, it makes sense, but it's still just a framework of belief. Now, there's nothing wrong with belief and sometimes you have to have a holy war in order to keep everyone pulling the same direction. But proof is better so find it whenever you can.

The only proof that counts in business is money. The closer the money is to the company, the better. In other words, find examples that people can relate to. Real examples. I hate that I have to stipulate "Real", but unfortunately, I do. Too often process advocates go looking for best (or worst) case examples or even pull them out of their hat. I've gotten to the point where if it didn't happen internally, I'll check sources. I've learned to derive a certain glee from demolishing someone who was, uh, less than careful. Be aware that people like me are out there and nothing will tank you faster than being shown to be faking your proof. You stand to lose more than simply the argument when that happens.

The best source for examples are things that happened in your company. In the case of XanGo, I was able to track back the cost of some bugs that happened there. A bug doesn't have to tank the company to impose a cost. It is always good policy to do a complete autopsy of significant bugs that hit production. Don't be satisfied with simply fixing the error. Dig as deeply as you can. That's just good practice. In this case, it also let me attach a dollar figure to bugs that actually had hit production despite our best vigilance in development. Adding a reasonable cost-benefit is almost like magic--it clears barriers away like they're made of tissue paper.

Don't mistake, though, a cost-benefit list isn't going to do you any good if you cannot articulate the reasons it will work the way you predict. A money-based analysis that isn't accompanied by solid reasons that you can articulate clearly isn't going to work any better than having reasons without a money-based analysis. Which is to say that it might work, but you're essentially rolling the dice. I prefer not to dice with my career.

More Than Winning

In addition to the above, I made a couple resolutions to myself. These are things that I knew from experience were important in helping a QA department work but that are easy to overlook from the development side of the fence.

1. The eventual QA team should be at least as high up on the IT totem pole as my team. They would need all the power they could get when crunch time comes and they're still finding bugs. It is the unfortunate reality in business that the ones who had the code last tend to be seen as the barrier to making a deadline.
2. Educate as many people as possible that QA is a wholly different skill set from development. A skill set, by the way, that is rarer (and hence more valuable). This includes making sure that the eventual QA manager (who I would have a part in interviewing) believed the same way. Too often, companies use their QA department as a back-door to a development career. While this is flattering to developers because it means they are the final product (i.e. the pinnacle, end-point, and better than), it is also a recipe for tragedy because the skill sets aren't simply unrelated--they are, to an extent, opposed.
3. Resolve to give the QA team my support in any future dust-ups. Dust-ups that are, by the way, inevitable. Developers are going to resent having bugs found by someone not them. They aren't going to like that they didn't actually reach a deadline they busted hump to hit. Further, too often when things go wrong, people start whacking around with the blame stick and they'll start in the QA department. It's really easy to keep a low profile when that happens because you want to avoid getting whacked yourself. It is, nonetheless, a bad idea to give in to this cowardice.

Now, all these things share a common feature: they're all things that require developer pain for the benefit of the QA team. Each one is also for the benefit of the company. My biggest point in this and the previous post is that concentrating on what is best for the company is most effective when it is sincere. We shouldn't be merely willing to do what is best for the company when it means pain for us (and possibly benefit for someone else), we should actively hunt down that pain with the intent of embracing it.

Having a QA department actually makes my job as Software Development Manager harder. It means imposing discipline, slowing down production, formalizing development processes, and giving some of my current corporate power to someone not me. It meant inviting another manager into my department who is at least as high on the corporate ladder as I am and who won't always agree with me.

That's a lot of downside to implement something that would be a lot of extra work and take some of my personal corporate political capital to make happen. The meager personal benefit list above doesn't really stack up well to the short term downside. The only reason I took it on is because I knew that doing so was in the best interest of the company and I knew that I could make that case in terms that others would understand.

 I said yesterday that

The skill sets [of developers and QA people] aren't simply unrelated—they are, to an extent, opposed.

 I think that deserves some explanation because for some reason this is not obvious even to people in IT.

The reason that developers and testers require fundamentally different skill sets is that they have fundamentally different responsibilities. What it comes down to is that it is a developer's job to make software work and it is a testers job to make "working" software break. In fact, I knew I had found the right QA Manager for XanGo when one of the first things he said to me was "hey, you wanna see something cool?", opened Excel, and demonstrated how to bring his computer to its knees in three easy steps. I know I'm not much of a tester because while I did think it was cool, I had forgotten those steps practically before he had finished his demonstration.

Separate is Better

When you are creating software, you want to take the quickest path to a functional application. Your emphasis is, correctly, on delivering an end product. You don't want to spend time thinking of every possible situation that can go wrong. You make your program as robust as it needs to be given its use and the importance it has to those using it.

When you are testing software, you want to take the quickest path you can find to break the software. You start with common uses and expand from that point to ensure that the finished software actually does all the things it is intended to do.

Is the skill set used for testing software useful for developers in creating it (and vice versa)? I get this question sometimes from people hoping to save the budget for a QA department. Their real question is "can't we just tell the programmers to do a better job testing their software?" The answer to that is "you save money, but only in the short term."

The thing is, we've found as an industry that testing works best if you have a group of people whose sole responsibility is to test software. This makes sense if you look at it as a case of separation of interests leading to a better combined result. This is why I consider the two skill sets ultimately opposed. When looking at a piece of software, is your intent to make it work or to break it? You really cannot do both at the same time and do them both well.

I'll take it one step further: developers are better off cultivating a perspective of getting their software out the door and testers are better off cultivating a perspective of preventing software from getting out the door until they cannot break it. Could someone, technically, develop both skill sets? Absolutely. I even know one developer who seems to have both built-in. I just don't see why anybody would want to cultivate both when companies are much better served having those functions separated.

Rarity and Value

I also stated yesterday that

QA is a wholly different skill set from development. A skill set, by the way, that is rarer (and hence more valuable).

I think this may be the more controversial statement because testers tend to be paid less than developers of comparable experience. I think this is unfortunate and based on being able to get away with it more than it represents true market value. Testers are often seen as Jr. Developers and that's unfortunate. I think this is possible because those who are truly good at testing are extremely rare. If you've ever had the chance to work with a top tester, though, you'll know what I'm talking about.

The best testers are meticulous to a degree that developers simply find unnatural. They test every edge condition and look for new ones wherever they can. They are almost obsessively detail oriented, even when they don't have to be. And they record everything they can think of.

Top testers improve the developers they work with. That's because a top tester will document fail points and develop hypotheses about why the errors occur in addition to where and under what conditions.

In addition, top testers analyze more than just the software they are testing. They'll assemble things like common fail points and patterns of error--mainly for their own benefit. Indeed, I've found that I sometimes have to dig to get their analysis because they sometimes don't realize the value of their findings to developers.

More subtly, they'll review common bug reports and feature requests and identify areas of improvement that users simply haven't thought of. This comes from their familiarity with the software they are testing. It isn't uncommon for the testers to know the software better than the developers who created it. This makes sense if you consider that a developer often visits any given piece of functionality in mostly unrelated chunks and a tester reviews it as a comprehensive whole.

How Much Testing?

My first experience with the benefits of a strong QA team came at Jenkon in the 90s. I was evaluating our software estimates because they tended to be off by a relatively constant margin. When I started tracking project estimates against actual delivery over time (I first had to start keeping that data. You'd be surprised how few people actually keep it beyond project completion), I found that thorough QA takes almost exactly as long as creating the software in the first place. For some reason, our estimators had it in their head that QA only took a quarter of the development time. Correcting this misperception was instrumental in reining in constant project overruns.

I haven't seen this ratio change since then. Every time someone thinks it has changed, it turns out that they are either skimping on QA or they're not counting all the actual QA time spent. It's been a couple of years since I had access to a decent QA team, though, so I could be off.

Oh my. Steve McConnell has started a blog. If you do not know who he is, you are either young in the ways of software development or aren't paying attention.

via Haacked

I listened to a recent .Net Rocks podcast about Domain Driven Design. Now, despite reminding me of a post last week by Secret Geek about how everything is driving design these days, I think a lot of what was said makes sense to me. Eric Evans' point on the podcast seemed to revolve around learning a given domain and letting experience with the domain drive development decisions.

From the contents of his book at Amazon, it looks like Eric's idea is a modification or extension of Model Driven Design—i.e. an attempt to unify development efforts within a descriptive framework approachable by non-technical entities. In this case, letting the needs of the domain determine the focus of development. For example, he talked near the end about how it is often more valuable to leave legacy code alone (maybe wrapping it with a translation layer) in order to spend resources on the things that will provide better value than you can achieve by replacing the legacy application.

Good Enough

Eric also used a phrase that I listen for as a flag that identifies someone to take seriously: "good enough". Good enough is a phrase used by people who are actually trying to apply YAGNI as more than a way to win an argument. Good enough is a limiting term, a cut-off, a phrase that gets you from the isolation of the silicon tower to actually developing software.

Now, I have to be careful here because "good enough" is also useful for burying dangerous assumptions. I think that Eric is an example because he never elaborates on what something is good enough for. Given that he's explaining Domain-Driven Design, you could assume he's saying that it's good enough for the domain, but that doesn't map. Nobody works for a domain.

Pandering to Business Managers

Which got me thinking. Eric is talking about developers and domain experts creating a common framework for communication, but really, starting at the domain elides over the most important consideration in software development: the business. The context for "good enough" should be rooted in tangible benefit for your company.

Naturally, I thought I would be all clever-like and create another "driven". Unfortunately, IBM already mucked this one up. In what looks like naked pandering to clueless business managers, IBM proposed "Business-Driven Development" in December of 2005. I mean, can you take any paper seriously that contains this quote:

The inherent problem with the enterprise software development process is that it suffers from a lack of agility to match the pace at which the business needs to change in order to keep up with the market trends and competition.

Now, you could make a case for market trends and competition being pretty malleable—even to the point of outpacing the ability of software development to keep up—but that's kind of beside the point. The fact of the matter is that of all the things a business needs to change to adopt a new business process, software is the most adaptable.

Functionally, IBM's proposal is a way to isolate software development and assert management control. Under scrutiny, it looks to me like an attempt to force agile iterative principles into a BDUF format that IBM's Rational products can actually handle.

Dollar-Driven Development

What I'd like to see is Eric's domain drivers better focused as a responsibility towards whoever is underwriting development. This is hard to name because his principles are broadly applicable for business, government or even open source.

Still, somebody is fiscally responsible for software development and it is that responsibility that should drive development decisions--i.e. the best interests of the entity underwriting it. For open source projects, that would be the interests of the benevolent dictator in charge. For business development, that would be the best interest of the business you work for.

This is easier said than done. In order to work for something's best interest, you really have to know its interests well. I think that may be the heart of what Eric was saying about learning the concerns of a domain and letting those concerns drive development. It means developing expertise in a domain that may or may not be of your choosing. This takes time, effort, and access to domain experts. It means delving deeply into the reasons things are done now and the problems current processes were meant to solve.

Mainly, it means learning about things that aren't computers and likely have nothing to do with them. Whether it's a bank, talent agency, or bail-bondsman, it's not going to be something that you learned in school. It's yet another thing you have to learn on top of all the technology stuff that keeps changing.

Good thing you're good at learning things.  Right?

Have you ever found yourself in a situation where you know what's the best thing to do, but are unable to convince anyone else that you are right? Developers know that even simple problems have more than one solution. Developers who have worked on a team of more than one have probably been in a situation where they just knew that the team was heading in the wrong direction and that they had a solution that was more elegant, easier to program, and better to maintain.

Higher profile developers often find themselves trying to explain their solutions to non-technical people as well; sometimes before development has begun, but sometimes after it is truly too late to do anything different.

So what do you do when you have to butt heads with buttheads? Here is my secret to winning in these situations

Geek Whisperer

As a developer and/or manager of developers, it is astonishingly easy to find yourself trying to explain concepts to people who, however smart, just aren't equipped to understand what the heck you are talking about. This is an uncomfortable position for all involved.

It is tempting in these situations to pull out a call to authority and tell people that you are the subject matter expert and they simply have to trust your expertise. This works fine for the little stuff, but as the scope increases, so too does the reluctance to put up with this answer.

And with good reason. Technologists can find themselves literally with the fate of the company in their hands. Business leaders find that hard to take and are going to be extremely wary of any solution that involves the fate of their company.

How to Win With Business Folk

There is really only one way to handle this situation: concentrate on what is best for the company.

Everything you say and do in this situation should start and end with what is best for the company. The magic of this orientation is that you can defeat every argument and diffuse even personal attacks by leading back to what's best for the company. No matter how irrational the objection, how confusing the question, or how deliberate the ignorance, you'll lower tension and create space for a solution if you can turn back to what is best for the company. Even asking the question, "What course of action would be best for the company?" can be sufficient to redirect an out-of-control or confrontational situation.

Talk Amongst Yourselves

Developers tend to believe that they live in a rational universe—or at least that their work is based on the application of structured reason to provide solutions to defined problems. This belief is shared, more or less, among all developers—even those who believe themselves to be the only developer who is actually rational.

This makes it both easier and harder to solve disagreements between developers. The fundamental belief in reason tends to keep disagreements from being personal, so you'll find most developers are willing to discuss alternatives and the ramifications of design decisions.

Every now and then, however, it is possible to find yourself at an impasse with another developer (or group of developers). In an impasse, developers can become more intractable than business folk if only because you lose the ability to appeal to consensus. Rational is rational just as right is right. The best rational choice remains the best rational choice no matter how many delusional fools find themselves unable to comprehend it.

How to Win With Technical Folk

There is really only one way to deal with this situation: concentrate on what is best for the company.

When two (or more) sides have presented what they feel is the most rational solution to a problem and remain at an impasse, the source of contention almost inevitably lies in unstated assumptions. In this situation, asking the question, "Why is this solution better for the company?" helps to ferret out those assumptions. Some disagreements will need to apply this question repeatedly as it is not uncommon for assumptions to be stacked on other assumptions. Ferreting out the source of your assumptions can be tiring, but in the end well worth the effort.

Open Sores

But what if you aren't developing as a part of a company? The principle remains the same as does the effect. Every open source project has a problem that it is trying to solve and people working on that project will tend to be there because they believe they can help solve that problem. Leading disagreements back to the reason of a project's existence can help straighten out priorities and direct solutions back to the original problem domain (and away from the individuals involved in the discussion).

Winning Everywhere!

Is there application for this question outside of geekdom? Darn skippy there is. In fact, asking "what is best for the company?" is a technique I learned at a leadership conference when I was more manager than developer. It turns out that any time people group up, they'll tend to have a reason for doing so. In fact, "tend to" == "always" in this case. The reason can be explicit or implicit, but people simply don't group up for no reason. Learning to find that reason and drive disagreements back to it helps keep you focused and gives you the opportunity to pick your battles.

A Warning

Asking what's best for your company/group/whatever will often show you the flaws in your own arguments, particularly as you first start applying the question. If you want to win for personal egotistical reasons then this technique isn't going to do you much good.

In fact, ego can destroy the benefits of this technique more generally. Depending on how much power the person with the ego has in the organization, their presence can significantly alter the ability of this technique to work. An example: I once worked for a company where the owners truly did not care what was best for the company. As such, arguing from the standpoint of what was best for the company had no effect whatsoever.

Fortunately, reality has a way of asserting itself in these situations. Companies that can't act for their own best interest tend to disappear, members who don't care about what's best for the group tend to leave the group, and employees who don't care about the best interest of their company tend to find themselves parted from the company.

Still, there can be significant discomfort in the meantime and heaven knows that "tends to" != "will" in this case. You'll have to decide how you approach a situation where ego is going to be a problem. It's almost always best to separate yourself from the ego getting in the way—particularly if it is your own.

"So when will you be done with this development project?"

I don't know about you, but I hate this question. There simply is no good answer for it. It seems like such a simple question with a simple DateTime valued answer. One of these days I swear I'll answer with, "Oh, I'll be done next Tuesday at 2:34pm." just to see what happens.

And seriously, businesses hate that we have such difficulty answering the question. It seems perfectly reasonable for them to want to know when they can plan to have the new processes that they know they desperately need. Developers demand high salaries and are ostensibly professionals, they should be able to give a professional answer, right?

The Road is Well Paved

The thing is, software development is a lot harder than people expect it to be--and this includes software professionals. Even simple software projects can run afoul of hidden complexities that can destroy well meaning estimates and make everyone unhappy. And no matter how you hedge your answers, people simply don't remember all your caveats, maybes, and what ifs that you use to indicate uncertainty.

The end result is that developers seldom make their ship-by dates and companies become disillusioned and impatient with all software development. That's not helpful for anybody, but it's pretty much the rule anymore.

And the fact of the matter is that the vast majority of developers (and development managers) never learn how to answer the estimate question. They'll move from company to company, repeating the cycle of hope, suspicion, and disappointment over and over again. Which works well enough for the developers in the boom times when the demand for development is so high that mildly talented house plants can get hired as developers.

So a lot of people are making the same mistakes over and over. Businesses can be excused for assuming that this is simply the way things are and feel confident in their distrust of software professionals. They've been there, done that, bought the t-shirt.

Paying the Toll

This environment causes developers who care about these kinds of things a lot of heartburn. Everyone pays for the ongoing cycle of disillusionment. I believe that this is what really prompts posts like the recent ones from Ted Neward talking about professional ethics. And I've been known to throw my own hat into the ring as well.

We get tired of paying for the sins of those who have gone before. And I'm not referring to the messed up legacy code we stumble into, either. Frankly, messed up code is the least of your problems coming into a situation with a client who has been burned by previous developer promises. Companies that have had deadline after deadline missed have a degree of mistrust that is very hard to overcome.

We pay for this distrust in a hundred different ways. The thing is, trust is a paying commodity in business. Working with partners you trust means a whole lot of overhead you can simply skip. An analogy: if I trust a plumber to fix my sink quickly and professionally, I can go get a burger and leave him to it. It's only when I don't have that trust that I have to pay the additional overhead of having someone I do trust watching to make sure he's not napping under the sink.

Want to see a business manager go into a dreamy fantasy? Ask them what it'd be like to be able to trust their software developers (in house or not). The more experience they've had with developers the more intense the fantasy.

The Rubber Meets the Road

We have a couple of areas of friction in businesses that exacerbate this situation. The main disconnect with business managers is that we have borrowed terminology and tools from other disciplines without understanding that our processes are fundamentally different. It's tricky because the temptation to use manufacturing terminology is immense. After all, we are creating a product of sorts. This makes so much sense on an intuitive level that it's hard to realize that the comparison is misleading and potentially dangerous.

I wish we could retrain everyone to make analogies to other business specialties. Scientific research or law come to mind as potentially useful analogies because both are similarly plagued by the impact of unique situations, changing ground rules, and unforeseen complexities. It would be interesting to investigate how managing software development like a patent application or drug research would change how we look at the problems involved. We might have stumbled onto iterative cycles and responding to altered requirements a whole lot sooner, for example.

Paying Attention

The real problem, though, is that most developers (and even most development managers) don't take the time to learn about common friction points. Nor do they take the time to build relations with their business counterparts so that you have some political capital (aka trust) to use when it is needed. It's easy to forget that much of the progress in software development practices are pretty recent in terms of business processes. After all, business managers don't move at the speed of light and changes tend to take time to penetrate those layers.

Which means that a whole lot of industry advances aren't even theory yet in the board room.

And the fact of the matter is that you cannot expect a business manager to understand what makes Agile practices work. Or the reason that strong unit testing saves time over the long run even though it takes more time up front. Learning to communicate at a level that is sufficiently detailed for smart business decisions without getting bogged down into the jargon inherent in any specialty is an invaluable skill, and one best learned earlier than later. That means thoroughly understanding those theories yourself--not just on the surface or in buzzword compliance. It also means learning to communicate that understanding from orbit, 30,000 ft, 5,000 ft, and right on the ground. This is hard to do. It takes practice. It also takes exposure to business manager types. I'm not sure which is harder...

Something to think about, though: not learning this skill leaves you at the mercy of those who do learn it.

My point, though, is that it takes both. You have to learn your profession so thoroughly that you can deconstruct its "best practices" ("design patterns", whatever) and rebuild them from basic principles on the fly. AND you have to learn to communicate that understanding comfortably to people of varying familiarity with software development in a business environment.

That's what it takes to be a true professional. It's easy to let those two skills fall out of balance. Individuals who understand both are invaluable to a company. Also rare. Companies who discover someone capable of both are often surprised at how much smoother things run with that person placed where they can do the most good--a point Jeff Atwood's latest on becoming a better programmer drives home.

So I don't have a formula for quick and accurate estimates. Just a lot of hard work. Still, here's a tip for free: anyone asking for a firm delivery date is inherently assuming BDUF. Once you know that, you know where to start your answer.

Have you ever been in an interview and gotten some variation on the question "What do you regret most about your last position?" Everyone hates questions like that. They're a huge risk with little upside for you. You're caught between the Scylla of honesty and the Charybdis of revealing unflattering things about yourself.

Still, such questions can be very valuable if used personally for analysis and improvement. In that light, I'll share with you two things I regret about my stay at XanGo. Since I've ripped on the environment there in the past, it's only fair if I elaborate on things that were under my control at the time--things I could have done better.

Neglecting the User

Tim was the Senior IT Manager (the position that became IT Director once XanGo had grown up a bit). He was the best boss I ever had. His tech skills were top-notch (if somewhat "old school"). In addition, he knew his executives and how to communicate with them on a level they understood. It was a refreshing experience to have someone good at both technology and management (and since he's no longer my boss, you can take that to the bank :)).

After a little break-in time as the new Software Development Manager, Tim and I discussed what we needed to do for the organization. Tim's advice was to establish a pattern of delivering one new "toy" for our Distributors each month. He said that the executive board are very attached to the Distributors and that keeping things fresh and delivering new functionality and tools to them each month would make sure that we had enough of the right kind of visibility. Goodwill in the bank, so to speak.

This sounded like a great idea, and frankly, "toy" was loosely defined enough that it shouldn't have been a hard thing to do. It turned out to be a lot harder than expected, however. In my defense, I'll point out that we were experiencing between 15% and 20% growth per month and that we had done so since the company had started a year and a half before. That growth continued my entire tenure there. Now, if you've never experienced that kind of growth, let me point out some of what that means.

First off, using the Rule of 72 (the coolest numeric rule I know) will tell you that we were doubling every 4 to 5 months (in every significant measure--sales, revenue, Distributors, traffic, shipping, everything).

In case you've never experienced that kind of growth, it feels like ice-skating with a jet engine fired up on your back. Even good architecture will strain with that kind of relentless growth. When this happens, you become hyper-vigilant for signs of strain. This vigilance has sufficient reality to be important to maintain. Unfortunately, it also makes it easy to forget your users.

Developers like to live in a pristine world of logic and procedure. Unfortunately, life, and users, aren't like that. If they were, there'd be less need for developers. Users don't see all the bullets you dodged. They take for granted the fact that a system originally designed for a small start up is now pulling off enterprise-level stunts. They don't see it, so it doesn't exist. It is very easy to get caught up in the technology and forget that often it is the little touches that make your product meaningful. Sometimes the new report you spent an hour hacking together means more than the three weeks of sweating out communication with a new bank transaction processor. And by means more, I mean "is more valuable than".

Not that you can afford to neglect your architecture or needed improvements to sustain the needs of the company and prepare for foreseeable events. If you ignore that little glitch in the payments processing this month, you have really no excuse when it decides to spew chunks spectacularly next month.

What I'm saying here is that you have to balance functionality with perceived value. You have to know your users and their expectations because if you aren't meeting those expectations, no amount of technical expertise or developer-fu is going to help you when things get rough. In the case of XanGo, I could have afforded to ease up on the architecture enough to kick out monthly toys for the users. Yeah, some things would have been a touch rockier, but looking back there was room for a better balance.

Premature Deprecation

When I arrived at XanGo, our original product (a customized vertical market app written in VB6 on MS SQL Server) was serving way beyond its original specifications. We'd made some customizations, many of them penetrating deep into the core of the product. Our primary concern, however, was the Internet application used by our Distributors in managing their sales. We spent a month or two moving it from ASP to ASP.NET and ironing out bugs brought on by the number of concurrent users we had to maintain. We also removed the dependence on a couple of VB6 modules that were spitting out raw HTML (yeah, I know. All I can say is that I didn't design the monster).

Anyway, after that was well enough in hand, we gave a serious look to that VB6 vertical market app. Since VB6 wasn't all that hot at concurrent data access and couldn't handle some of the functionality we were delivering over the new web app, we decided that it should be phased out. Adding to this decision was the fact that we had lost control of the customizations to that app and what we had wouldn't compile in its present state.

Now developers (and for any management-fu I may have acquired, I remain a developer at heart) tend to be optimistic souls, so we figured "no big", we'll be replacing this app anyway. And we set to work. Bad choice. In a high growth environment, the inability to fix bugs now takes on a magnified importance. Replacing an application always takes longer than you expect if only because it's so easy to take the current functionality for granted. Any replacement has to be at least as good as the current application, and should preferably provide significant, visible improvements.

The result of this decision was that we limped along for quite some time before we finally came to the conclusion that we absolutely must have the ability to fix the current app. We paid a lot of political capital for that lack. In the end, it took a top developer out of circulation for a while but once it was done, it was astonishing how much pressure was lifted from Development.

It's the Users, Stupid

No, I did not mean to say "It's the stupid users." When it comes right down to it, software exists to serve users, not the other way around. As developers, it is easy to acquire a casual (or even vehement) dislike of our users. They are never satisfied, they do crazy stuff that makes no sense, and they're always asking for more. It's tempting to think that things would be so much better without them.*

I got into computers because I like making computers do cool stuff. Whatever gets developers into computers, though, it's a good idea to poke your head up periodically and see what your users are doing. Get to know who they are. Find out what they think about what you've provided for them. Losing that focus can cost you. Sometimes dearly.

*I think one of the draws of Open Source is that the developer is the user. It's also the primary drawback. But that's a post for another day.

Lidor Wyssocky has some good thoughts on why it is that developers don't implement changes that they know would be helpful.

The problem is that although we know exactly what doesn’t work right and how it should be fixed, most of us will never say anything. We don’t say anything because there’s a very good chance the minute we do we will be marked as uncooperative, pessimistic, or simply detached from the business reality. (emphasis in original)

He concludes with his call to action.

If more of us say what we know in our hearts to be true, the rest won’t be able to ignore it anymore. Hopefully.

And I agree with him. What he is talking about is an aspect of professional integrity. If you are a professional that a business relies on for good decision making, then you need to act the part and provide reasonable suggestions wherever they may do good.

But there exists both a flip-side and a problem with Lidor's points that are inherent in professional integrity that need to be explored.

The Flip-side

While you avoid negative attention by remaining silent about needed process change, you can gain positive attention by supporting whatever has caught the eye of your executives. This is a major contributing factor to "The Silver Bullet Syndrome". At XanGo for example, we had one company convince management that they could cut development costs to 10% of their current level. Not 10% off, mind, that's 10% of current levels.

Despite all logic to the contrary (I mean, think about it--if a product could actually deliver cost savings of 90%, every company on the planet would be using it--they'd be incompetent not to), XanGo ended up spending millions of dollars and wasted a full year of development on this silver bullet. What was most fascinating to me were those who were willing to endorse this silver bullet. Those people were promoted to team leaders and project managers. Any who would oppose the new development project were either removed before its introduction or told (in at least one case explicitly) that negative feedback would put their jobs in jeopardy.

In this way, technical people who should have known better ended up perpetrating the problem not by staying silent but by actively promoting bad solutions. They were rewarded for doing so. Here's the thing: in most cases, there really is no price to pay for backing a losing silver-bullet solution. Oh, those in front of the board have negative exposure when it all blows up. Some of the higher management tiers are at risk as well. But for most of IT, there simply is no bill to pay for being in the pack.

The Problem

Here's the thing: having Professional Integrity carries not-insignificant risks.

1. Having integrity makes you predictable. Those with an agenda can plan their actions confident of what you will do. That means that if things get nasty, you end up following Marquess of Queensberry rules in a bar fight.
2. Your compensation is in the hands of ignorant people. They don't know what you know. Many of them may be smarter than you. Even if they aren't, though, they are still the ones signing the paychecks.
3. There is seldom a single, obvious best answer. Honest and rational people can arrive at differing conclusions based on the same information (because they weight aspects of the problem domain differently). This can easily split the message received by management and open the door for bad solutions. This also leaves you vulnerable to the manipulative because they can confuse the issue at need.
4. Costs of bad solutions are often deferred while the costs of changing minds is immediate. Whether opposing a bad proposal or proposing a better process you are asking for change and that makes people uncomfortable. If you make people uncomfortable they will often hold you responsible for their discomfort.
5. Sometimes having professional integrity means looking for a new job. Whether you are fired or are forced to resign, some positions are simply incompatible with maintaining professional integrity. These situations are rare, but they can happen.
6. People aren't rational. As a result businesses often make choices that seem irrational. Sometimes those choices are right even though they are irrational. Professional Integrity in such a situation is problematic at best.

As I see it, Lidor's call to arms makes sense but is unlikely without more support. The thing is that the problem is bigger than he has acknowledged. He is asking us to take actions that are overtly risky and he hasn't given us enough of a reason to do so. I mean, the emperor may not have any clothes, but there's no telling how he'll react to you pointing that out.

The Solution Domain

"Because it's the right thing to do" resonates with most of us, I think. But it is important to acknowledge that, at heart, Lidor's plea is a moral and not a technical one. As such, it needs to be approached from a social, not a technical standpoint. The thing is, we are equipped to make this happen, but we need to explore it in the right context if we hope to make progress.

Societies have a lot of experience coercing correct behavior. There are a lot of tools available, some better than others. My preferences tend to flow from my LDS background. As such, I'd advocate proselytizing correct procedures. We should explain as clearly as we can the benefits of acting correctly. While the payoffs of acting correctly are long-term, they exist nevertheless and are provable and measurable. Professional Integrity in IT has similar long-term value (above and beyond correct practices) to businesses and those who learn to identify and value it will gain the benefits it brings. A company that hires professionals with integrity will do better than companies that hire similarly talented people who lack that dedication.

As long as we're borrowing from religious tradition, lets hit up the Catholics and/or Jews (leaving no stereotype unturned) and apply some well-earned shame. I know who sold us out at XanGo. My reaction to them in the future can (and should) be informed by their actions there. I think we do ourselves a disservice if we know compromised IT professionals and we don't tell them our opinion of their actions or make those opinions public. Repentance (more religious terminology, but applicable) should be possible, certainly--people can change and reformation should be encouraged. That said, it's important that forgiveness be contingent on honest efforts towards reparation and change.

On a more general level, ostracizing those who behave badly is one of society's greatest tools to restrain bad behavior. The next time I review resumes to evaluate new hire candidates, I'll have an eye out for those I know who, while they may be bright and talented developers, ended up endorsing those things that were bad for the business. More generally, I'll try to craft questions that explore a candidate's professional integrity in addition to their technical expertise. Reputations in IT should be more than merely technical. Professional Integrity is important not just to businesses but to the perception of IT as a whole. The IT crash in 2001 contained a lot of payback for abuses in IT during the build-up of the bubble. While we were on top many of us made unreasonable and expensive demands on companies (some personal, some technological) and those abuses weakened us as a group.

Collective bargaining is one route that I'm sure will come up--a union or standards body that enforces "correctness". Personally, I'm not a fan of that option. Bureaucracies are inefficient and tend to be in opposition to business interests. We do not need more antagonism from the executive suite. Unions also have a tendency to groupthink that I consider inimical to technology innovation.

I'm sure there are others that I am overlooking. I'd love to hear further thoughts in the comments. Please leave 'em if ya got 'em. Don't make me beg.

I acknowledge that my thoughts here will be hard to implement. They require conscious short-term effort with no promise of reward. It's certainly beyond the scope for a single individual to affect. Still, I believe it's possible to make things better than they are right now and that the potential for reward is both real and achievable. Plus, Paladins are made for exactly this kind of idealistic fight for what's right...