Aaron Lerch talks a bit about what it takes to be a technical manager. He enumerates five characteristics that he feels are essential for a technical manager (personally, I think that they are better classified as skills because they’re learnable and improvable). Here’s his five:

1. Communication
2. Technical Savvy
3. Organizational Skills
4. Priorities
5. Humility

It’s a good list for the good times, I think. As long as things run more or less smoothly, a manager with those traits will be in good shape. As you can probably tell by my qualifiers, I think that he’s missing something.

Tough Choices

Unfortunately, managers (technical or not) often find themselves in a position where a decision has to be made and where the choices are all distasteful. What do you do when it turns out that someone doesn’t live up to your expectations or, more importantly, their job description? Or they simply don’t get along with the other developers on the team? Or, heaven forbid, the company has decided that less overhead in the development department is needed?

Firing someone is, by far, the hardest thing I’ve ever done as a manager. Firing someone is both an admission of failure and one of the harshest things a person can legally do to another. So a manager who fires somebody is pretty much a big, mean failure.

It’s a real shame that sometimes, people simply need firing. If you’ve ever been on a team with someone who couldn’t (or wouldn’t) pull their own weight, you’ll understand what I mean. Not firing someone who needs it can have a larger impact on team productivity than just about anything else I’ve encountered. I know because I’ve been a team member when someone needed to be let go and the manager couldn’t bring himself to pull the trigger. The hit to both productivity and morale is huge and grows geometrically over time.

Firing someone is the extreme case of an important skill. Management is all about the hard decisions. Yes, communication is important. And making irrevocable technical decisions without team input is suicide. But consensus is the one sure path to mediocrity and, frankly, mediocrity doesn’t do that well in the marketplace. It’d be nice if everything would shake out into rational lines with obvious choices, but real life (and real development) doesn’t work that way.

Decisive Humility

To make things harder, Aaron’s point about humility is still a good one. Technical managers should do their very best to surround themselves with people who are smarter than they are. Being surrounded by smart folks can be hard on you if you wear your ego on your sleeve, though. Also, smart people are incredibly difficult to control.

Being able to make compromises and solicit (and listen to) advice from your team is important for both you and them. But eventually, you will find yourself in a situation where the alternatives are as known as they are going to be and it is time to make that tough call. Making that call (and sticking to it) is your job. Managers who waffle or leave it up to a vote (whether formal or not) or who weasel out of responsibility later compromise the integrity of their team.

Balancing humility with the power and ability to decide is tough. It is, nonetheless, necessary.

Top Skills

So how would I re-order Aaron’s top five?

1. Communication
2. Technical Savvy
3. Responsible Decision Making
4. Humility
5. Priorities
6. Organizational Skills

While this is a general ranking, situations will arise where any one can become paramount—meaning that none are optional. All of them should be as strong as you can make them and the better you do, the better manager you will be.

In many companies developer career progression is deceptively straight-forward; Jr. Programmer, Programmer, Sr. Programmer, Team Lead, Architect, Sr. Architect, Bob (Bob being the semi-mythical entity referred to in obscure comments, worshipped by now-extinct aboriginal tribes, and rumored to haunt the sub-sub-basement).

The differentiation between these positions starts off with how much you know. A Sr. Programmer is a Jr. Programmer who knows his tools inside and out and can complete assigned tasks quickly and without a lot of supervision. Around Team Lead time, however, progression stops being about what you know and starts revolving around your ability to choose wisely between competing trade-offs.

This is an easy transition to miss if you aren’t paying attention. Indeed, I’ve known "Architects" that I wouldn’t ask to design a "Hello World" application. Because being a good architect is about developing good judgement, there are two things that are essential to becoming and remaining competent.

Know Your Stuff

Yeah, I know I just said that being an architect isn’t about what you know. That’s only slightly true. What changes between Sr. Programmer and Architect is the depth of your knowledge—you have to penetrate to the reasons and complexities behind patterns, practices, and technologies. You go from knowing what an n-tier architecture is (and how to create and maintain one) to understanding why that architecture was invented, what problems it solves, and what costs are involved in creating and maintaining one.

Take, for example, the "architecture" I found in coming to a new position. A number of projects are used for this intranet application, each in its own application space. Here’s a short illustration of the structure of two of the solutions.

As you might guess, BOL stands for "Business Object Layer", DAL for "Data Access Layer", and OBJ for "Objects". You might not be surprised, given the closely related project names, to learn that each project contains classes that exist in the other. In short, despite having the structure and using the terminology of n-tier architecture, this isn’t actually anything of the kind.

The developer responsible for this design is not an architect, no matter how much experience he may have, because he has failed to understand when and why you would use an n-tier architecture. Understanding the reasons behind n-tier architecture would make it impossible to create this structure.

Understanding architecture takes hard work. It requires more than reading blogs, attending conferences or purchasing books (though each of those are certainly useful in developing understanding). True understanding requires thinking things through and asking tough questions. It requires breaking down concepts until you can identify and understand their component parts. Often, it takes asking the questions that risk confronting a room full of people glaring at you in varying degrees of shock. Most of all, it takes admitting that you might not know something and working to rectify that lack—or, harder still, being willing to re-examine assumptions (or even conclusions) when presented with new information.

The Development Twins: Cost and Benefit

It’s easy to forget that every technology, architecture, and practice has two sides; a cost and a benefit. To borrow from Heinlein, "There ain’t no such thing as a free lunch." Because making choices is hard, it’s easy for people to remember only one side of a given technology, architecture, or practice—to see only the cost or only the benefit.

This is particularly true in new processes or designs. We go through so many silver bullets at least partially because people teaching new processes tend to concentrate on benefits. This is understandable even if the presenter isn’t financially tied to your adoption of the Great New Thing™ because the benefits are the reason the Great New Thing™ exists. This means that you’ll be taught the benefits, but you’ll typically have to discover the costs on your own.

It’s also important to bear in mind that developers have a natural bias towards solving problems, not finding them which leads to a tendency to grasp benefits quickly. Because architects are grown from developers, it’s important to examine and balance this potential optimistic bias. And it is a balance. Ask an average Linux geek about Microsoft if you want to see a bias towards cost.

The Scope Multiplier

A project’s scope has a multiplicative effect on both cost and benefit. Understanding this became explicit for me in a reply I made to a recent comment by Udi Dahan.

I’d be willing to bet that the benefits of DI, SoC and other good design practices increase exponentially with the number of developers while their costs are a linear progression. Huh. That bears some thought...

For any given process, you need to understand how both benefit and cost are affected by the scope of your project. Practices that scale benefits exponentially and costs linearly are excellent when the scope is large. Indeed, the larger the scope, the better the net gain from applying the given practice.

A corollary of that evaluation is that practices designed for large scales are less likely to provide a net benefit in a small team with a small project.

It’s possible that project scope can be calculated separately for project complexity and team size, but I’d have to give that some more thought. Since high complexity projects tend to require larger team sizes, you’d seldom have one without the other (making this a moot conjecture). That said, my initial instinct is that something like n-tier is more important vs. complexity whereas the benefits of SoC scale more based on team size (though it helps mitigate complexity as well). It is possible that this difference may play an important role in how and when you’d apply each.

Time

That pesky fourth dimension can also have a dramatic effect on cost and benefit. A quick down-and-dirty project with limited scope may seem like it wouldn’t benefit from any real architecture at all. That evaluation can change drastically if the project is likely to change a lot over time. Most developers have been ambushed by application maintenance at some point because they hadn’t planned adequately for change.

Predicting the future is a dubious pursuit at best, however, so be careful here. There’s a reason YAGNI became a well-recognized acronym. Don’t forget that a project can be refactored into a useful pattern after the fact if circumstances change to warrant it—particularly if you have prepared for easy refactoring (unit testing, anyone?).

The biggest problem with evaluating the effect of time on your project is that it is literally impossible to measure the impact of the choice not taken. As a result, I’m always very careful when I hear something like "If only we had..." or "Good thing we...". Personal, anecdotal experience is error prone at best—something to keep in mind during project postmortems.

The Courage to Choose

The most important thing an architect does is decide. The architect chooses the technologies, the patterns, and the practices to apply to a given software development problem knowing that success and failure can hang on the effects of those choices. That’s a lot of responsibility. The impulse to push your choice off to a vendor, or some guru’s public statements, or industry consensus is a seductive one that can work for a time. Reality being the harsh b**tch she is, however, it isn’t a safe impulse to indulge. Personally, I prefer to hold my fate in my own hands.

I wrote last month about winning arguments in IT. Earlier this week, Phil Haack asked a question (through Twitter) about things he could do to help convince a company to create an in-house QA department. Well, it turns out that I did exactly that at XanGo—successfully pushed for and oversaw the installation of an in-house QA department. I thought it might be a useful follow-up to the previous post to use this as an example of how I "won" that argument.

Concentrate on What is Best for the Company

This is the key, the whole key and nothing but the key to winning corporate arguments. You should approach any problem from this perspective. In this case, I have seen what software development looks like with a competent QA team and I wanted that again. Simple developer testing was leaving holes and we didn't have a consistent deployment model. I was sure that having an in-house QA team would benefit the company.

Now, since I was the Software Development Manager, instantiating a QA team was likely out of the scope of my responsibilities. Fortunately, I had a good relationship with my boss and he was the functional IT Director. It was well within his scope. This gave me my first temptation: the desire to expand my personal empire by creating a "QA Team" within my department. After all, it was my idea, it would take a lot of effort on my part to bring about, and QA is associated with development, right?

Tempting as it is, though, it would definitely not be to the company's benefit. A QA team should have as much power as they can get because they are the last rational review before stuff goes into production. What that means in practical terms is that they should be a first-class IT member. If other departments can override QA, you are headed for a lot of pain.

Which is when I realized that what I had to do first is marshal all the reasons I could think of that an in-house QA team would benefit XanGo. This is probably really what Phil was looking for. It's easy to come up with reasons it'd be good from your own perspective. It's a lot harder to broaden that perspective to the company as a whole.

Here's what a list of personal benefits might look like:

1. Backstop development so that buggy code doesn't hit production.
2. Help train developers by pointing out errors early enough that it is still in the mental cache of the developer who created it.
3. Give me a blame-shield if something broke.

Here's some things that are more useful from the company's perspective:

1. Backstop development so that buggy code doesn't hit production.
2. Help train developers by providing timely feedback.
3. Ensure that customers don't encounter broken processes.
4. Ensure that employee bug reports and feature requests are tracked, prioritized, and scheduled.
5. Develop and own a formal build/deploy process that is repeatable and automated.
6. Increase executive confidence that software releases do the things they are promising people it does.
7. Help identify training opportunities for development.
8. Help identify training opportunities for non-development staff.
9. Create a manager who can concentrate on identifying people with QA skills and provide a career path for them.
10. We have a Software Development Manager who knows what a good QA team looks like and how to create one and we can easily leverage his expertise without bringing in expensive consultants.

This list was my external list of arguments. Which ones I'd present depended heavily on a given audience. With executives, for example, I'd start with #3, glide through #4, #1 and #7 and then slam into #6. I'd pause there and ask if there was interest. If I had their interest (and I guarantee that I did), I could cover the other points in more detail--generally in a follow-up, formal meeting.

Show Me the Money

Reason is all well and good, but without something concrete and measurable, we're still in the realm of faith. Yeah, it makes sense, but it's still just a framework of belief. Now, there's nothing wrong with belief and sometimes you have to have a holy war in order to keep everyone pulling the same direction. But proof is better so find it whenever you can.

The only proof that counts in business is money. The closer the money is to the company, the better. In other words, find examples that people can relate to. Real examples. I hate that I have to stipulate "Real", but unfortunately, I do. Too often process advocates go looking for best (or worst) case examples or even pull them out of their hat. I've gotten to the point where if it didn't happen internally, I'll check sources. I've learned to derive a certain glee from demolishing someone who was, uh, less than careful. Be aware that people like me are out there and nothing will tank you faster than being shown to be faking your proof. You stand to lose more than simply the argument when that happens.

The best source for examples are things that happened in your company. In the case of XanGo, I was able to track back the cost of some bugs that happened there. A bug doesn't have to tank the company to impose a cost. It is always good policy to do a complete autopsy of significant bugs that hit production. Don't be satisfied with simply fixing the error. Dig as deeply as you can. That's just good practice. In this case, it also let me attach a dollar figure to bugs that actually had hit production despite our best vigilance in development. Adding a reasonable cost-benefit is almost like magic--it clears barriers away like they're made of tissue paper.

Don't mistake, though, a cost-benefit list isn't going to do you any good if you cannot articulate the reasons it will work the way you predict. A money-based analysis that isn't accompanied by solid reasons that you can articulate clearly isn't going to work any better than having reasons without a money-based analysis. Which is to say that it might work, but you're essentially rolling the dice. I prefer not to dice with my career.

More Than Winning

In addition to the above, I made a couple resolutions to myself. These are things that I knew from experience were important in helping a QA department work but that are easy to overlook from the development side of the fence.

1. The eventual QA team should be at least as high up on the IT totem pole as my team. They would need all the power they could get when crunch time comes and they're still finding bugs. It is the unfortunate reality in business that the ones who had the code last tend to be seen as the barrier to making a deadline.
2. Educate as many people as possible that QA is a wholly different skill set from development. A skill set, by the way, that is rarer (and hence more valuable). This includes making sure that the eventual QA manager (who I would have a part in interviewing) believed the same way. Too often, companies use their QA department as a back-door to a development career. While this is flattering to developers because it means they are the final product (i.e. the pinnacle, end-point, and better than), it is also a recipe for tragedy because the skill sets aren't simply unrelated--they are, to an extent, opposed.
3. Resolve to give the QA team my support in any future dust-ups. Dust-ups that are, by the way, inevitable. Developers are going to resent having bugs found by someone not them. They aren't going to like that they didn't actually reach a deadline they busted hump to hit. Further, too often when things go wrong, people start whacking around with the blame stick and they'll start in the QA department. It's really easy to keep a low profile when that happens because you want to avoid getting whacked yourself. It is, nonetheless, a bad idea to give in to this cowardice.

Now, all these things share a common feature: they're all things that require developer pain for the benefit of the QA team. Each one is also for the benefit of the company. My biggest point in this and the previous post is that concentrating on what is best for the company is most effective when it is sincere. We shouldn't be merely willing to do what is best for the company when it means pain for us (and possibly benefit for someone else), we should actively hunt down that pain with the intent of embracing it.

Having a QA department actually makes my job as Software Development Manager harder. It means imposing discipline, slowing down production, formalizing development processes, and giving some of my current corporate power to someone not me. It meant inviting another manager into my department who is at least as high on the corporate ladder as I am and who won't always agree with me.

That's a lot of downside to implement something that would be a lot of extra work and take some of my personal corporate political capital to make happen. The meager personal benefit list above doesn't really stack up well to the short term downside. The only reason I took it on is because I knew that doing so was in the best interest of the company and I knew that I could make that case in terms that others would understand.

 I said yesterday that

The skill sets [of developers and QA people] aren't simply unrelated—they are, to an extent, opposed.

 I think that deserves some explanation because for some reason this is not obvious even to people in IT.

The reason that developers and testers require fundamentally different skill sets is that they have fundamentally different responsibilities. What it comes down to is that it is a developer's job to make software work and it is a testers job to make "working" software break. In fact, I knew I had found the right QA Manager for XanGo when one of the first things he said to me was "hey, you wanna see something cool?", opened Excel, and demonstrated how to bring his computer to its knees in three easy steps. I know I'm not much of a tester because while I did think it was cool, I had forgotten those steps practically before he had finished his demonstration.

Separate is Better

When you are creating software, you want to take the quickest path to a functional application. Your emphasis is, correctly, on delivering an end product. You don't want to spend time thinking of every possible situation that can go wrong. You make your program as robust as it needs to be given its use and the importance it has to those using it.

When you are testing software, you want to take the quickest path you can find to break the software. You start with common uses and expand from that point to ensure that the finished software actually does all the things it is intended to do.

Is the skill set used for testing software useful for developers in creating it (and vice versa)? I get this question sometimes from people hoping to save the budget for a QA department. Their real question is "can't we just tell the programmers to do a better job testing their software?" The answer to that is "you save money, but only in the short term."

The thing is, we've found as an industry that testing works best if you have a group of people whose sole responsibility is to test software. This makes sense if you look at it as a case of separation of interests leading to a better combined result. This is why I consider the two skill sets ultimately opposed. When looking at a piece of software, is your intent to make it work or to break it? You really cannot do both at the same time and do them both well.

I'll take it one step further: developers are better off cultivating a perspective of getting their software out the door and testers are better off cultivating a perspective of preventing software from getting out the door until they cannot break it. Could someone, technically, develop both skill sets? Absolutely. I even know one developer who seems to have both built-in. I just don't see why anybody would want to cultivate both when companies are much better served having those functions separated.

Rarity and Value

I also stated yesterday that

QA is a wholly different skill set from development. A skill set, by the way, that is rarer (and hence more valuable).

I think this may be the more controversial statement because testers tend to be paid less than developers of comparable experience. I think this is unfortunate and based on being able to get away with it more than it represents true market value. Testers are often seen as Jr. Developers and that's unfortunate. I think this is possible because those who are truly good at testing are extremely rare. If you've ever had the chance to work with a top tester, though, you'll know what I'm talking about.

The best testers are meticulous to a degree that developers simply find unnatural. They test every edge condition and look for new ones wherever they can. They are almost obsessively detail oriented, even when they don't have to be. And they record everything they can think of.

Top testers improve the developers they work with. That's because a top tester will document fail points and develop hypotheses about why the errors occur in addition to where and under what conditions.

In addition, top testers analyze more than just the software they are testing. They'll assemble things like common fail points and patterns of error--mainly for their own benefit. Indeed, I've found that I sometimes have to dig to get their analysis because they sometimes don't realize the value of their findings to developers.

More subtly, they'll review common bug reports and feature requests and identify areas of improvement that users simply haven't thought of. This comes from their familiarity with the software they are testing. It isn't uncommon for the testers to know the software better than the developers who created it. This makes sense if you consider that a developer often visits any given piece of functionality in mostly unrelated chunks and a tester reviews it as a comprehensive whole.

How Much Testing?

My first experience with the benefits of a strong QA team came at Jenkon in the 90s. I was evaluating our software estimates because they tended to be off by a relatively constant margin. When I started tracking project estimates against actual delivery over time (I first had to start keeping that data. You'd be surprised how few people actually keep it beyond project completion), I found that thorough QA takes almost exactly as long as creating the software in the first place. For some reason, our estimators had it in their head that QA only took a quarter of the development time. Correcting this misperception was instrumental in reining in constant project overruns.

I haven't seen this ratio change since then. Every time someone thinks it has changed, it turns out that they are either skimping on QA or they're not counting all the actual QA time spent. It's been a couple of years since I had access to a decent QA team, though, so I could be off.

I listened to a recent .Net Rocks podcast about Domain Driven Design. Now, despite reminding me of a post last week by Secret Geek about how everything is driving design these days, I think a lot of what was said makes sense to me. Eric Evans' point on the podcast seemed to revolve around learning a given domain and letting experience with the domain drive development decisions.

From the contents of his book at Amazon, it looks like Eric's idea is a modification or extension of Model Driven Design—i.e. an attempt to unify development efforts within a descriptive framework approachable by non-technical entities. In this case, letting the needs of the domain determine the focus of development. For example, he talked near the end about how it is often more valuable to leave legacy code alone (maybe wrapping it with a translation layer) in order to spend resources on the things that will provide better value than you can achieve by replacing the legacy application.

Good Enough

Eric also used a phrase that I listen for as a flag that identifies someone to take seriously: "good enough". Good enough is a phrase used by people who are actually trying to apply YAGNI as more than a way to win an argument. Good enough is a limiting term, a cut-off, a phrase that gets you from the isolation of the silicon tower to actually developing software.

Now, I have to be careful here because "good enough" is also useful for burying dangerous assumptions. I think that Eric is an example because he never elaborates on what something is good enough for. Given that he's explaining Domain-Driven Design, you could assume he's saying that it's good enough for the domain, but that doesn't map. Nobody works for a domain.

Pandering to Business Managers

Which got me thinking. Eric is talking about developers and domain experts creating a common framework for communication, but really, starting at the domain elides over the most important consideration in software development: the business. The context for "good enough" should be rooted in tangible benefit for your company.

Naturally, I thought I would be all clever-like and create another "driven". Unfortunately, IBM already mucked this one up. In what looks like naked pandering to clueless business managers, IBM proposed "Business-Driven Development" in December of 2005. I mean, can you take any paper seriously that contains this quote:

The inherent problem with the enterprise software development process is that it suffers from a lack of agility to match the pace at which the business needs to change in order to keep up with the market trends and competition.

Now, you could make a case for market trends and competition being pretty malleable—even to the point of outpacing the ability of software development to keep up—but that's kind of beside the point. The fact of the matter is that of all the things a business needs to change to adopt a new business process, software is the most adaptable.

Functionally, IBM's proposal is a way to isolate software development and assert management control. Under scrutiny, it looks to me like an attempt to force agile iterative principles into a BDUF format that IBM's Rational products can actually handle.

Dollar-Driven Development

What I'd like to see is Eric's domain drivers better focused as a responsibility towards whoever is underwriting development. This is hard to name because his principles are broadly applicable for business, government or even open source.

Still, somebody is fiscally responsible for software development and it is that responsibility that should drive development decisions--i.e. the best interests of the entity underwriting it. For open source projects, that would be the interests of the benevolent dictator in charge. For business development, that would be the best interest of the business you work for.

This is easier said than done. In order to work for something's best interest, you really have to know its interests well. I think that may be the heart of what Eric was saying about learning the concerns of a domain and letting those concerns drive development. It means developing expertise in a domain that may or may not be of your choosing. This takes time, effort, and access to domain experts. It means delving deeply into the reasons things are done now and the problems current processes were meant to solve.

Mainly, it means learning about things that aren't computers and likely have nothing to do with them. Whether it's a bank, talent agency, or bail-bondsman, it's not going to be something that you learned in school. It's yet another thing you have to learn on top of all the technology stuff that keeps changing.

Good thing you're good at learning things.  Right?

Have you ever found yourself in a situation where you know what's the best thing to do, but are unable to convince anyone else that you are right? Developers know that even simple problems have more than one solution. Developers who have worked on a team of more than one have probably been in a situation where they just knew that the team was heading in the wrong direction and that they had a solution that was more elegant, easier to program, and better to maintain.

Higher profile developers often find themselves trying to explain their solutions to non-technical people as well; sometimes before development has begun, but sometimes after it is truly too late to do anything different.

So what do you do when you have to butt heads with buttheads? Here is my secret to winning in these situations

Geek Whisperer

As a developer and/or manager of developers, it is astonishingly easy to find yourself trying to explain concepts to people who, however smart, just aren't equipped to understand what the heck you are talking about. This is an uncomfortable position for all involved.

It is tempting in these situations to pull out a call to authority and tell people that you are the subject matter expert and they simply have to trust your expertise. This works fine for the little stuff, but as the scope increases, so too does the reluctance to put up with this answer.

And with good reason. Technologists can find themselves literally with the fate of the company in their hands. Business leaders find that hard to take and are going to be extremely wary of any solution that involves the fate of their company.

How to Win With Business Folk

There is really only one way to handle this situation: concentrate on what is best for the company.

Everything you say and do in this situation should start and end with what is best for the company. The magic of this orientation is that you can defeat every argument and diffuse even personal attacks by leading back to what's best for the company. No matter how irrational the objection, how confusing the question, or how deliberate the ignorance, you'll lower tension and create space for a solution if you can turn back to what is best for the company. Even asking the question, "What course of action would be best for the company?" can be sufficient to redirect an out-of-control or confrontational situation.

Talk Amongst Yourselves

Developers tend to believe that they live in a rational universe—or at least that their work is based on the application of structured reason to provide solutions to defined problems. This belief is shared, more or less, among all developers—even those who believe themselves to be the only developer who is actually rational.

This makes it both easier and harder to solve disagreements between developers. The fundamental belief in reason tends to keep disagreements from being personal, so you'll find most developers are willing to discuss alternatives and the ramifications of design decisions.

Every now and then, however, it is possible to find yourself at an impasse with another developer (or group of developers). In an impasse, developers can become more intractable than business folk if only because you lose the ability to appeal to consensus. Rational is rational just as right is right. The best rational choice remains the best rational choice no matter how many delusional fools find themselves unable to comprehend it.

How to Win With Technical Folk

There is really only one way to deal with this situation: concentrate on what is best for the company.

When two (or more) sides have presented what they feel is the most rational solution to a problem and remain at an impasse, the source of contention almost inevitably lies in unstated assumptions. In this situation, asking the question, "Why is this solution better for the company?" helps to ferret out those assumptions. Some disagreements will need to apply this question repeatedly as it is not uncommon for assumptions to be stacked on other assumptions. Ferreting out the source of your assumptions can be tiring, but in the end well worth the effort.

Open Sores

But what if you aren't developing as a part of a company? The principle remains the same as does the effect. Every open source project has a problem that it is trying to solve and people working on that project will tend to be there because they believe they can help solve that problem. Leading disagreements back to the reason of a project's existence can help straighten out priorities and direct solutions back to the original problem domain (and away from the individuals involved in the discussion).

Winning Everywhere!

Is there application for this question outside of geekdom? Darn skippy there is. In fact, asking "what is best for the company?" is a technique I learned at a leadership conference when I was more manager than developer. It turns out that any time people group up, they'll tend to have a reason for doing so. In fact, "tend to" == "always" in this case. The reason can be explicit or implicit, but people simply don't group up for no reason. Learning to find that reason and drive disagreements back to it helps keep you focused and gives you the opportunity to pick your battles.

A Warning

Asking what's best for your company/group/whatever will often show you the flaws in your own arguments, particularly as you first start applying the question. If you want to win for personal egotistical reasons then this technique isn't going to do you much good.

In fact, ego can destroy the benefits of this technique more generally. Depending on how much power the person with the ego has in the organization, their presence can significantly alter the ability of this technique to work. An example: I once worked for a company where the owners truly did not care what was best for the company. As such, arguing from the standpoint of what was best for the company had no effect whatsoever.

Fortunately, reality has a way of asserting itself in these situations. Companies that can't act for their own best interest tend to disappear, members who don't care about what's best for the group tend to leave the group, and employees who don't care about the best interest of their company tend to find themselves parted from the company.

Still, there can be significant discomfort in the meantime and heaven knows that "tends to" != "will" in this case. You'll have to decide how you approach a situation where ego is going to be a problem. It's almost always best to separate yourself from the ego getting in the way—particularly if it is your own.

"So when will you be done with this development project?"

I don't know about you, but I hate this question. There simply is no good answer for it. It seems like such a simple question with a simple DateTime valued answer. One of these days I swear I'll answer with, "Oh, I'll be done next Tuesday at 2:34pm." just to see what happens.

And seriously, businesses hate that we have such difficulty answering the question. It seems perfectly reasonable for them to want to know when they can plan to have the new processes that they know they desperately need. Developers demand high salaries and are ostensibly professionals, they should be able to give a professional answer, right?

The Road is Well Paved

The thing is, software development is a lot harder than people expect it to be--and this includes software professionals. Even simple software projects can run afoul of hidden complexities that can destroy well meaning estimates and make everyone unhappy. And no matter how you hedge your answers, people simply don't remember all your caveats, maybes, and what ifs that you use to indicate uncertainty.

The end result is that developers seldom make their ship-by dates and companies become disillusioned and impatient with all software development. That's not helpful for anybody, but it's pretty much the rule anymore.

And the fact of the matter is that the vast majority of developers (and development managers) never learn how to answer the estimate question. They'll move from company to company, repeating the cycle of hope, suspicion, and disappointment over and over again. Which works well enough for the developers in the boom times when the demand for development is so high that mildly talented house plants can get hired as developers.

So a lot of people are making the same mistakes over and over. Businesses can be excused for assuming that this is simply the way things are and feel confident in their distrust of software professionals. They've been there, done that, bought the t-shirt.

Paying the Toll

This environment causes developers who care about these kinds of things a lot of heartburn. Everyone pays for the ongoing cycle of disillusionment. I believe that this is what really prompts posts like the recent ones from Ted Neward talking about professional ethics. And I've been known to throw my own hat into the ring as well.

We get tired of paying for the sins of those who have gone before. And I'm not referring to the messed up legacy code we stumble into, either. Frankly, messed up code is the least of your problems coming into a situation with a client who has been burned by previous developer promises. Companies that have had deadline after deadline missed have a degree of mistrust that is very hard to overcome.

We pay for this distrust in a hundred different ways. The thing is, trust is a paying commodity in business. Working with partners you trust means a whole lot of overhead you can simply skip. An analogy: if I trust a plumber to fix my sink quickly and professionally, I can go get a burger and leave him to it. It's only when I don't have that trust that I have to pay the additional overhead of having someone I do trust watching to make sure he's not napping under the sink.

Want to see a business manager go into a dreamy fantasy? Ask them what it'd be like to be able to trust their software developers (in house or not). The more experience they've had with developers the more intense the fantasy.

The Rubber Meets the Road

We have a couple of areas of friction in businesses that exacerbate this situation. The main disconnect with business managers is that we have borrowed terminology and tools from other disciplines without understanding that our processes are fundamentally different. It's tricky because the temptation to use manufacturing terminology is immense. After all, we are creating a product of sorts. This makes so much sense on an intuitive level that it's hard to realize that the comparison is misleading and potentially dangerous.

I wish we could retrain everyone to make analogies to other business specialties. Scientific research or law come to mind as potentially useful analogies because both are similarly plagued by the impact of unique situations, changing ground rules, and unforeseen complexities. It would be interesting to investigate how managing software development like a patent application or drug research would change how we look at the problems involved. We might have stumbled onto iterative cycles and responding to altered requirements a whole lot sooner, for example.

Paying Attention

The real problem, though, is that most developers (and even most development managers) don't take the time to learn about common friction points. Nor do they take the time to build relations with their business counterparts so that you have some political capital (aka trust) to use when it is needed. It's easy to forget that much of the progress in software development practices are pretty recent in terms of business processes. After all, business managers don't move at the speed of light and changes tend to take time to penetrate those layers.

Which means that a whole lot of industry advances aren't even theory yet in the board room.

And the fact of the matter is that you cannot expect a business manager to understand what makes Agile practices work. Or the reason that strong unit testing saves time over the long run even though it takes more time up front. Learning to communicate at a level that is sufficiently detailed for smart business decisions without getting bogged down into the jargon inherent in any specialty is an invaluable skill, and one best learned earlier than later. That means thoroughly understanding those theories yourself--not just on the surface or in buzzword compliance. It also means learning to communicate that understanding from orbit, 30,000 ft, 5,000 ft, and right on the ground. This is hard to do. It takes practice. It also takes exposure to business manager types. I'm not sure which is harder...

Something to think about, though: not learning this skill leaves you at the mercy of those who do learn it.

My point, though, is that it takes both. You have to learn your profession so thoroughly that you can deconstruct its "best practices" ("design patterns", whatever) and rebuild them from basic principles on the fly. AND you have to learn to communicate that understanding comfortably to people of varying familiarity with software development in a business environment.

That's what it takes to be a true professional. It's easy to let those two skills fall out of balance. Individuals who understand both are invaluable to a company. Also rare. Companies who discover someone capable of both are often surprised at how much smoother things run with that person placed where they can do the most good--a point Jeff Atwood's latest on becoming a better programmer drives home.

So I don't have a formula for quick and accurate estimates. Just a lot of hard work. Still, here's a tip for free: anyone asking for a firm delivery date is inherently assuming BDUF. Once you know that, you know where to start your answer.

Have you ever been in an interview and gotten some variation on the question "What do you regret most about your last position?" Everyone hates questions like that. They're a huge risk with little upside for you. You're caught between the Scylla of honesty and the Charybdis of revealing unflattering things about yourself.

Still, such questions can be very valuable if used personally for analysis and improvement. In that light, I'll share with you two things I regret about my stay at XanGo. Since I've ripped on the environment there in the past, it's only fair if I elaborate on things that were under my control at the time--things I could have done better.

Neglecting the User

Tim was the Senior IT Manager (the position that became IT Director once XanGo had grown up a bit). He was the best boss I ever had. His tech skills were top-notch (if somewhat "old school"). In addition, he knew his executives and how to communicate with them on a level they understood. It was a refreshing experience to have someone good at both technology and management (and since he's no longer my boss, you can take that to the bank :)).

After a little break-in time as the new Software Development Manager, Tim and I discussed what we needed to do for the organization. Tim's advice was to establish a pattern of delivering one new "toy" for our Distributors each month. He said that the executive board are very attached to the Distributors and that keeping things fresh and delivering new functionality and tools to them each month would make sure that we had enough of the right kind of visibility. Goodwill in the bank, so to speak.

This sounded like a great idea, and frankly, "toy" was loosely defined enough that it shouldn't have been a hard thing to do. It turned out to be a lot harder than expected, however. In my defense, I'll point out that we were experiencing between 15% and 20% growth per month and that we had done so since the company had started a year and a half before. That growth continued my entire tenure there. Now, if you've never experienced that kind of growth, let me point out some of what that means.

First off, using the Rule of 72 (the coolest numeric rule I know) will tell you that we were doubling every 4 to 5 months (in every significant measure--sales, revenue, Distributors, traffic, shipping, everything).

In case you've never experienced that kind of growth, it feels like ice-skating with a jet engine fired up on your back. Even good architecture will strain with that kind of relentless growth. When this happens, you become hyper-vigilant for signs of strain. This vigilance has sufficient reality to be important to maintain. Unfortunately, it also makes it easy to forget your users.

Developers like to live in a pristine world of logic and procedure. Unfortunately, life, and users, aren't like that. If they were, there'd be less need for developers. Users don't see all the bullets you dodged. They take for granted the fact that a system originally designed for a small start up is now pulling off enterprise-level stunts. They don't see it, so it doesn't exist. It is very easy to get caught up in the technology and forget that often it is the little touches that make your product meaningful. Sometimes the new report you spent an hour hacking together means more than the three weeks of sweating out communication with a new bank transaction processor. And by means more, I mean "is more valuable than".

Not that you can afford to neglect your architecture or needed improvements to sustain the needs of the company and prepare for foreseeable events. If you ignore that little glitch in the payments processing this month, you have really no excuse when it decides to spew chunks spectacularly next month.

What I'm saying here is that you have to balance functionality with perceived value. You have to know your users and their expectations because if you aren't meeting those expectations, no amount of technical expertise or developer-fu is going to help you when things get rough. In the case of XanGo, I could have afforded to ease up on the architecture enough to kick out monthly toys for the users. Yeah, some things would have been a touch rockier, but looking back there was room for a better balance.

Premature Deprecation

When I arrived at XanGo, our original product (a customized vertical market app written in VB6 on MS SQL Server) was serving way beyond its original specifications. We'd made some customizations, many of them penetrating deep into the core of the product. Our primary concern, however, was the Internet application used by our Distributors in managing their sales. We spent a month or two moving it from ASP to ASP.NET and ironing out bugs brought on by the number of concurrent users we had to maintain. We also removed the dependence on a couple of VB6 modules that were spitting out raw HTML (yeah, I know. All I can say is that I didn't design the monster).

Anyway, after that was well enough in hand, we gave a serious look to that VB6 vertical market app. Since VB6 wasn't all that hot at concurrent data access and couldn't handle some of the functionality we were delivering over the new web app, we decided that it should be phased out. Adding to this decision was the fact that we had lost control of the customizations to that app and what we had wouldn't compile in its present state.

Now developers (and for any management-fu I may have acquired, I remain a developer at heart) tend to be optimistic souls, so we figured "no big", we'll be replacing this app anyway. And we set to work. Bad choice. In a high growth environment, the inability to fix bugs now takes on a magnified importance. Replacing an application always takes longer than you expect if only because it's so easy to take the current functionality for granted. Any replacement has to be at least as good as the current application, and should preferably provide significant, visible improvements.

The result of this decision was that we limped along for quite some time before we finally came to the conclusion that we absolutely must have the ability to fix the current app. We paid a lot of political capital for that lack. In the end, it took a top developer out of circulation for a while but once it was done, it was astonishing how much pressure was lifted from Development.

It's the Users, Stupid

No, I did not mean to say "It's the stupid users." When it comes right down to it, software exists to serve users, not the other way around. As developers, it is easy to acquire a casual (or even vehement) dislike of our users. They are never satisfied, they do crazy stuff that makes no sense, and they're always asking for more. It's tempting to think that things would be so much better without them.*

I got into computers because I like making computers do cool stuff. Whatever gets developers into computers, though, it's a good idea to poke your head up periodically and see what your users are doing. Get to know who they are. Find out what they think about what you've provided for them. Losing that focus can cost you. Sometimes dearly.

*I think one of the draws of Open Source is that the developer is the user. It's also the primary drawback. But that's a post for another day.

It's always nice to have your opinions confirmed by someone you respect. Joel Spolsky's latest series on recruiting developers has a final section that includes essentially the same point I made a couple months ago: that specific technologies aren't as important in hiring developers as general savvy is. He even issues the same caveat that you do need domain experts for leavening. Nice. As is usual with Joel, he includes a thorough analysis of the issue he explores so there's a lot of good rumination there.

The following applies mainly to in-house business software development. It might or might not apply to ISV or other product development houses. I think that there's room for broad application, but you can hit my list of software blogs if you want some quality sources for more generalized ISV or product development exploration.

Back when I was looking to hire developers a couple years ago, I knew some programmers that I wanted on my team. I had worked with them before and knew what they were capable of. Unfortunately, they didn't have much experience with .NET--our platform at the time. I made the case then (to my boss and HR) that a good developer is a good developer and the language and framework are more or less immaterial for people with a broad set of experience. The syntax and capabilities of the language and environment are teachable and a developer's broad experience helps them pick it up quickly.

I was fortunate enough to bring one of those developers (who is currently working in indie games) onto my team. The experience was great. I was right that he was able to come up to speed and pick up the nuances of the environment very quickly. Further, he was able to solve some issues outside of the framework in ways that saved us a lot of time (using Python for text file manipulation for example).

So does this mean that you can toss out environment-specific skill sets when evaluating who to hire? Not in my opinion. You see at that time we already had a core group that was intimately familiar with the capabilities of .NET. Domain-specific knowledge is crucial in producing software applications that actually do what they are supposed to do in a reasonable timeframe. People who have already digested the learning curve are invaluable in blazing the way for others who haven't yet done so. The reason specialists are crucial is simple--somebody who has already walked the trail is able to point out the pitfalls and shortcuts and help the whole team move quickly. The new guy was able to come up to speed so fast because he had people he could ask when he had questions and who could explain why things worked the way they did.

I came out of this experience with respect for having a diverse programming team. You have to have at least some specialists who know the environment inside and out. These specialists are going to be the limit of what you can accomplish in a lot of ways so they need certain skills--chief among them, the ability to communicate with other developers (because they'll do more of that than they probably want to and because a specialist who can't communicate isn't able to influence the project nearly as well as one who can).

But I personally don't think it is a good idea to have a team or department composed solely of specialists. Any software development that requires more than two people is going to run into problems in a wide array of domains and having a broad skill set to draw on can be very useful. What proportion of specialists to generally kick-ass developers do you need? Well, that's a good question. I welcome ideas in the comments, but I suspect this is one of those things that is highly contextual and thus at the heart of the "art" of software development. I'm thinking that a narrowly defined ISV benefits with a higher ratio of specialists than, say, a multi-level marketing company looking to manage their necessarily custom business needs.