The Rabid Paladin

Developers hate it when someone changes the requirements in the middle of a development project. What few have realized yet is that they've gone and changed the requirements to be a developer right in the middle of our careers.

An Unnecessary, But Illustrative, Story

The summer of 1994 found me in Mesa, Arizona, new-minted diploma in hand, getting ready for my first real job. A couple days before I was to report for the new position, I received news that the job that I had pulled my wife and baby daughter into the gods-forsaken desert for had, well, been eliminated. Not a happy piece of news to receive. Since my parents lived there in Mesa we ended up taking refuge in the familial homestead. I love my parents, don't get me wrong, but this was not a celebrated event—by any of us.

This was apparently all the impetus that my dad needed to leave the largish firm he worked for as the main estate attorney. He asked me to help him automate the documents he used to create trusts and wills. For money. This was the start of my transition from an English B.A. to a computer programmer. Who knew you could make money with computers?!?

The Point

My dad found that having much of the drudge work in creating those documents automated allowed him to concentrate on larger aspects of estate law. He was able to grapple with the greater context and put things together in a way he hadn't been able to before. This change in focus helped both his practice and his clients because he was able to provide better service at a cheaper cost and still make more money himself.

So here's what this has to do with software development: advances in tools, design patterns, and technologies have created the same shift for software developers that my dad realized as an attorney. While the occasional Luddite might quail at the thought of leaving vi for an IDE with built-in debugger support, most developers are happy to move away from the drudge work that has preoccupied us for so long (as with any boost in efficiency the expected new leisure has yet to show up).

In fact, what we find now is that developers are expected to take on more and more responsibility. This is a good thing overall because increased responsibility == increased value (and hence, generally, salary). This can be a bad thing, however, if software developers neglect some of the new skills that we need in order to handle these responsibilities well.

This is what lies at the heart of recent calls for developers to learn "soft" skills and develop better communication. That's because, unless you work in some back-water in the benighted hinterlands, the days of the cave-troll coder are over. A developer can no longer count on being isolated from users, managers, or <shudder> marketing weasels.

What Everybody Seems to be Missing

All of this is well and good and recognized by those who are paying attention. But it's not enough.

I like to compare business software development with practicing business law because they are both logical disciplines that have to support the human decision-making processes that are inherent in running a business. There's one large difference between the two that I think is important—lawyers are taught the fundamentals of business as a part of their law degree. They don't have an MBA or anything, but they do attend classes explaining typical business structures and basic accounting principles. That's because doing so helps them both with their interface to the business and in their ability to deliver work that properly supports the companies they work for or with.

Well, software developers are increasingly in a similar relationship with the companies they work for. I'm fortunate in my English degree because I'm reasonably well set up in the whole soft/communication skills area. What I'm finding lately, though, is that the courses I took in Business, Economics and Accounting (I never did figure out what I wanted to major in) are becoming crucial to my ability to do my job as a software developer. Understanding double-book accounting and GAAP help me understand the needs of the finance guys. Understanding supply-chain and basic inventory control helps me understand the operations folks.

I'm not talking here about becoming an expert in business. What I'm saying is that developers need to take the time to make sure that they understand the reasons behind the weird and wacky things that other business units are doing (and requesting that we automate).

And it wouldn't hurt to take a formal class or two.

Too often, the software developer going over the current supply-chain configuration with an operations manager will jump to conclusions, or worse, offer "helpful" suggestions about how things can be done more logically. More often than not, we come across as isolated from the realities that the current configuration was created to overcome. Or worse, we come across as judgmental jerks.

It's a drag having to learn so much that was originally considered outside of our discipline. Unless you're a hardware or other highly-specialized developer, however, it's going to turn out to be the kind of thing that sets the competent apart from the competence-challenged.

Steve Harman had a post back at the beginning of the month about stuff you'd tell a young developer. It's a reaction to a similar piece by Jeremy Allison. It's an interesting topic, so I thought I'd waste a few pixels on it myself.

If it's not what you love, don't do it

I wouldn't generalize this to other fields, but for software development, I think this is a good thing to keep in mind. A lot has been made in the past by career counselors and other gurus about "finding your bliss" or similar nonsense. I think that's mostly a crock. You can be a good doctor, mechanic, or professor without liking what you do. Sure, those who love what they do will tend to excel at it and rise to the top of their field, but you have to ask yourself how much better off people at the top of a field are vs. those who are merely good enough.

That said, I think software development is somewhat unique as a career. Software development advances incredibly fast and keeping up with new technologies and ways of doing things takes a non-trivial amount of effort. You have to paddle hard just to stay even and that means experimenting and learning on your own. You could probably do fine if you have Arnold Schwarzenegger level drive, but short of that, loving what you do is about the only way to get you where you need to be.

Reputation is important

People who have been around the block know that the difference between developers can be extreme. Top developers are 5 to 20 times more productive than their peers. Given some of the complete incompetents I've known, I suspect that there's really no upper limit on that number.

Unfortunately, both Jeremy and Steve use this point to jump into the virtues of Open Source software. Open source contributions can enhance your reputation, but that has nothing to do with people being able to look at your code—it's more about working with other developers who can then form their own opinions of you as a developer. That's helpful, but limited.

A good reputation is someone talking about how well you did a job for them. It's your name coming up when someone asks "do you know any good developers?" A good reputation is more important than mere code sample availability and the wider your reputation can penetrate the better off you are.

How important is reputation? Let me put it this way: I haven't gotten a job from anyone who didn't know somebody I'd worked with since 1997. Your reputation and your skills are your only true security in this field.

Never stop asking why

The vast majority of people in any field are content to learn what they have to do to get the job done. Top-tier people are the ones who are asking pesky, even impertinent, questions all the time. Jeremy puts this under the heading of "Learn the architecture of the machine" but that's too limiting. Learn as much as you possibly can about why and how things come together to get the job done.

There's two parts to this, really. The first is that you don't stop just because you got something working. Find out why iterating backwards through an array fixed your bug. Look for the best event to use for control loading rather than just the first one you found that happens to work. Learning how to parse technical documents, navigate help files, and put a concise Google query together are key skills you'll need to master.

The second part of this is harder: if you don't know something people are talking about, ask. This becomes harder to do over time because you start believing your own hype. It's easy to believe that inner voice that's saying you can bluff your way through this conversation and look up anything important later. The thing is, the really bright people, the ones you most want to impress, aren't going to be fooled. I've also found that top technologists enjoy getting intelligent questions almost as much as they love answering them. Note the qualifier there and spend the mental effort to make your questions intelligent, though.

Your employer is more important than the community

This goes directly counter to both Jeremy and Steve and I'm going to tank any chance I have with digg or reddit, but it needs to be said.

Jeremy looks like he's coming from that staunch anti-corporation Slashdot tradition that talks in moral terms about the beauty of open source software and the depravity of Microsoft. Reality check: it is not a common goal of open source developers to create "the greatest, most beautiful work of art that all of you can create together." That's certainly not why I contribute to open source projects.

Open source developers are no more virtuous than corporate developers. While you may program for fun and the satisfaction of a job well done, the choice of project you work on is inherently a selfish one. Always. Whether you program for a paycheck, community recognition, or because you need working blog software that you can tweak when you want to your choice is based on internal needs, not virtues.

If they're both selfish, shouldn't I be saying that they're equally important? I don't think so and it's not just because a company will help support your family. The reason your company is more important is that your company has to compete in the market place. It has no choice. Because a company has to charge money for its products or services, the company is forced to provide things that other people want badly enough to part with their hard-earned cash. In practical terms, it means that a company has a reality check that can't be ignored because sooner or later they're going to have to go out there and prove themselves in the marketplace. While the tech bubble bursting was tremendously painful when it happened, imagine the crap pile we would have if it didn't burst. Ever.

In addition, companies force you, the developer, to do work that you don't want to do. Developers hate that, which is perfectly understandable. Being forced to work with unreasonable managers, clueless users, and hopelessly disconnected marketing weenies is a drag. If you're good, though, it's also a rich source of opportunities to tackle things that you might never have thought needed tackling. And it forces you to justify your dogma of choice to a group of supremely skeptical, self-confident people who have the welfare of their families on the line. Yeah, that's not fun, but it means that you'll have to learn to understand and articulate things you just know are true. With the added bonus that you'll find that they sometimes aren't so true after all.

Finally, be wary of advice from old timers

Not me, of course. My advice is nigh infallible. Just ask me, I'll tell you. If you go awry following my advice, it's probably because of errors in implementation.

Okay, maybe not. I've blogged about recent errors I've made, so I have to assume there will be others. Be careful about who you listen to. Be aware that everybody has their own hobby horses and pet peeves and it's a human trait to cram those prejudices into everything (particularly advice). Also be aware that internet communities trend towards consensus as much as any other human community does. Consensus is dangerous and will tend to mask assumptions and pitfalls. Develop a well-honed BS filter and use it liberally.

While you're at it, the sooner you learn to pass your own ideas and theories through a well-developed skepticism the better off you'll be.

Have you ever found yourself in a situation where you know what's the best thing to do, but are unable to convince anyone else that you are right? Developers know that even simple problems have more than one solution. Developers who have worked on a team of more than one have probably been in a situation where they just knew that the team was heading in the wrong direction and that they had a solution that was more elegant, easier to program, and better to maintain.

Higher profile developers often find themselves trying to explain their solutions to non-technical people as well; sometimes before development has begun, but sometimes after it is truly too late to do anything different.

So what do you do when you have to butt heads with buttheads? Here is my secret to winning in these situations

Geek Whisperer

As a developer and/or manager of developers, it is astonishingly easy to find yourself trying to explain concepts to people who, however smart, just aren't equipped to understand what the heck you are talking about. This is an uncomfortable position for all involved.

It is tempting in these situations to pull out a call to authority and tell people that you are the subject matter expert and they simply have to trust your expertise. This works fine for the little stuff, but as the scope increases, so too does the reluctance to put up with this answer.

And with good reason. Technologists can find themselves literally with the fate of the company in their hands. Business leaders find that hard to take and are going to be extremely wary of any solution that involves the fate of their company.

How to Win With Business Folk

There is really only one way to handle this situation: concentrate on what is best for the company.

Everything you say and do in this situation should start and end with what is best for the company. The magic of this orientation is that you can defeat every argument and diffuse even personal attacks by leading back to what's best for the company. No matter how irrational the objection, how confusing the question, or how deliberate the ignorance, you'll lower tension and create space for a solution if you can turn back to what is best for the company. Even asking the question, "What course of action would be best for the company?" can be sufficient to redirect an out-of-control or confrontational situation.

Talk Amongst Yourselves

Developers tend to believe that they live in a rational universe—or at least that their work is based on the application of structured reason to provide solutions to defined problems. This belief is shared, more or less, among all developers—even those who believe themselves to be the only developer who is actually rational.

This makes it both easier and harder to solve disagreements between developers. The fundamental belief in reason tends to keep disagreements from being personal, so you'll find most developers are willing to discuss alternatives and the ramifications of design decisions.

Every now and then, however, it is possible to find yourself at an impasse with another developer (or group of developers). In an impasse, developers can become more intractable than business folk if only because you lose the ability to appeal to consensus. Rational is rational just as right is right. The best rational choice remains the best rational choice no matter how many delusional fools find themselves unable to comprehend it.

How to Win With Technical Folk

There is really only one way to deal with this situation: concentrate on what is best for the company.

When two (or more) sides have presented what they feel is the most rational solution to a problem and remain at an impasse, the source of contention almost inevitably lies in unstated assumptions. In this situation, asking the question, "Why is this solution better for the company?" helps to ferret out those assumptions. Some disagreements will need to apply this question repeatedly as it is not uncommon for assumptions to be stacked on other assumptions. Ferreting out the source of your assumptions can be tiring, but in the end well worth the effort.

Open Sores

But what if you aren't developing as a part of a company? The principle remains the same as does the effect. Every open source project has a problem that it is trying to solve and people working on that project will tend to be there because they believe they can help solve that problem. Leading disagreements back to the reason of a project's existence can help straighten out priorities and direct solutions back to the original problem domain (and away from the individuals involved in the discussion).

Winning Everywhere!

Is there application for this question outside of geekdom? Darn skippy there is. In fact, asking "what is best for the company?" is a technique I learned at a leadership conference when I was more manager than developer. It turns out that any time people group up, they'll tend to have a reason for doing so. In fact, "tend to" == "always" in this case. The reason can be explicit or implicit, but people simply don't group up for no reason. Learning to find that reason and drive disagreements back to it helps keep you focused and gives you the opportunity to pick your battles.

A Warning

Asking what's best for your company/group/whatever will often show you the flaws in your own arguments, particularly as you first start applying the question. If you want to win for personal egotistical reasons then this technique isn't going to do you much good.

In fact, ego can destroy the benefits of this technique more generally. Depending on how much power the person with the ego has in the organization, their presence can significantly alter the ability of this technique to work. An example: I once worked for a company where the owners truly did not care what was best for the company. As such, arguing from the standpoint of what was best for the company had no effect whatsoever.

Fortunately, reality has a way of asserting itself in these situations. Companies that can't act for their own best interest tend to disappear, members who don't care about what's best for the group tend to leave the group, and employees who don't care about the best interest of their company tend to find themselves parted from the company.

Still, there can be significant discomfort in the meantime and heaven knows that "tends to" != "will" in this case. You'll have to decide how you approach a situation where ego is going to be a problem. It's almost always best to separate yourself from the ego getting in the way—particularly if it is your own.

"So when will you be done with this development project?"

I don't know about you, but I hate this question. There simply is no good answer for it. It seems like such a simple question with a simple DateTime valued answer. One of these days I swear I'll answer with, "Oh, I'll be done next Tuesday at 2:34pm." just to see what happens.

And seriously, businesses hate that we have such difficulty answering the question. It seems perfectly reasonable for them to want to know when they can plan to have the new processes that they know they desperately need. Developers demand high salaries and are ostensibly professionals, they should be able to give a professional answer, right?

The Road is Well Paved

The thing is, software development is a lot harder than people expect it to be--and this includes software professionals. Even simple software projects can run afoul of hidden complexities that can destroy well meaning estimates and make everyone unhappy. And no matter how you hedge your answers, people simply don't remember all your caveats, maybes, and what ifs that you use to indicate uncertainty.

The end result is that developers seldom make their ship-by dates and companies become disillusioned and impatient with all software development. That's not helpful for anybody, but it's pretty much the rule anymore.

And the fact of the matter is that the vast majority of developers (and development managers) never learn how to answer the estimate question. They'll move from company to company, repeating the cycle of hope, suspicion, and disappointment over and over again. Which works well enough for the developers in the boom times when the demand for development is so high that mildly talented house plants can get hired as developers.

So a lot of people are making the same mistakes over and over. Businesses can be excused for assuming that this is simply the way things are and feel confident in their distrust of software professionals. They've been there, done that, bought the t-shirt.

Paying the Toll

This environment causes developers who care about these kinds of things a lot of heartburn. Everyone pays for the ongoing cycle of disillusionment. I believe that this is what really prompts posts like the recent ones from Ted Neward talking about professional ethics. And I've been known to throw my own hat into the ring as well.

We get tired of paying for the sins of those who have gone before. And I'm not referring to the messed up legacy code we stumble into, either. Frankly, messed up code is the least of your problems coming into a situation with a client who has been burned by previous developer promises. Companies that have had deadline after deadline missed have a degree of mistrust that is very hard to overcome.

We pay for this distrust in a hundred different ways. The thing is, trust is a paying commodity in business. Working with partners you trust means a whole lot of overhead you can simply skip. An analogy: if I trust a plumber to fix my sink quickly and professionally, I can go get a burger and leave him to it. It's only when I don't have that trust that I have to pay the additional overhead of having someone I do trust watching to make sure he's not napping under the sink.

Want to see a business manager go into a dreamy fantasy? Ask them what it'd be like to be able to trust their software developers (in house or not). The more experience they've had with developers the more intense the fantasy.

The Rubber Meets the Road

We have a couple of areas of friction in businesses that exacerbate this situation. The main disconnect with business managers is that we have borrowed terminology and tools from other disciplines without understanding that our processes are fundamentally different. It's tricky because the temptation to use manufacturing terminology is immense. After all, we are creating a product of sorts. This makes so much sense on an intuitive level that it's hard to realize that the comparison is misleading and potentially dangerous.

I wish we could retrain everyone to make analogies to other business specialties. Scientific research or law come to mind as potentially useful analogies because both are similarly plagued by the impact of unique situations, changing ground rules, and unforeseen complexities. It would be interesting to investigate how managing software development like a patent application or drug research would change how we look at the problems involved. We might have stumbled onto iterative cycles and responding to altered requirements a whole lot sooner, for example.

Paying Attention

The real problem, though, is that most developers (and even most development managers) don't take the time to learn about common friction points. Nor do they take the time to build relations with their business counterparts so that you have some political capital (aka trust) to use when it is needed. It's easy to forget that much of the progress in software development practices are pretty recent in terms of business processes. After all, business managers don't move at the speed of light and changes tend to take time to penetrate those layers.

Which means that a whole lot of industry advances aren't even theory yet in the board room.

And the fact of the matter is that you cannot expect a business manager to understand what makes Agile practices work. Or the reason that strong unit testing saves time over the long run even though it takes more time up front. Learning to communicate at a level that is sufficiently detailed for smart business decisions without getting bogged down into the jargon inherent in any specialty is an invaluable skill, and one best learned earlier than later. That means thoroughly understanding those theories yourself--not just on the surface or in buzzword compliance. It also means learning to communicate that understanding from orbit, 30,000 ft, 5,000 ft, and right on the ground. This is hard to do. It takes practice. It also takes exposure to business manager types. I'm not sure which is harder...

Something to think about, though: not learning this skill leaves you at the mercy of those who do learn it.

My point, though, is that it takes both. You have to learn your profession so thoroughly that you can deconstruct its "best practices" ("design patterns", whatever) and rebuild them from basic principles on the fly. AND you have to learn to communicate that understanding comfortably to people of varying familiarity with software development in a business environment.

That's what it takes to be a true professional. It's easy to let those two skills fall out of balance. Individuals who understand both are invaluable to a company. Also rare. Companies who discover someone capable of both are often surprised at how much smoother things run with that person placed where they can do the most good--a point Jeff Atwood's latest on becoming a better programmer drives home.

So I don't have a formula for quick and accurate estimates. Just a lot of hard work. Still, here's a tip for free: anyone asking for a firm delivery date is inherently assuming BDUF. Once you know that, you know where to start your answer.

Renter beware. I've heard a number of technology companies lately extol the great virtues of renting—both hardware and software. They sell the concept to their investors and shareholders and talk about the wonderful revenue stream they will build. And I'm afraid that is all that they see. As such, they are doomed to failure. There will be a backlash, mark my words. The thing is, as pointed out in Infoworld,

But we don't see any vendor propaganda promising that we'll save money by renting.

In fact, companies moving to rental are missing a fundamental rule of business—the customer is king. The whole point of capitalism is that the absence of compulsion means that you have to win customer dollars by providing something people want. And the bottom line here is that nobody wants to rent. At the very most fundamental, nobody wants to rent unless doing so is significantly cheaper than buying. Not just a little cheaper, significantly cheaper. Renting means that someone else has control over your destiny. It means you do not own the tools that make your business run. In something like IT spending for businesses in particular, where change represents significant cost, you do not want to be dependent on another company for the continuing good function of your computer systems. It is suicide. And Ed Foster at the same magazine says similar things about "maintenance" contracts which is rent on the back side.

Companies aren't going to agree to draconian rental policies just because tech companies want them to. Even when they want them to really badly and they prattle on about bug fixes and free upgrades.

• Fact, free upgrades won't happen—companies will invent new names and split upgrade paths to, well, generate more revenues.
• Fact, bug fixes aren't going to happen any faster with rental agreements—you can guarantee that companies won't bump spending on support and programming just because they have more revenues coming in.
• Fact, new revenues will go to new programs, new initiatives, and new products to generate new revenues.

Consumers are not stupid. Companies who see their customers as walking revenue streams have lost the focus that made them successful in the first place. You build revenues by accurately identifying the wants and needs of your customers—not by accurately identifying their budgets. My advice? Avoid tech companies with great plans for rental revenues like the plague—not only their products, but I'd stay away from their stock, too. The resentment of their customers will rebound and they will end up the worse for it.

Attack! One of the fundamental rules of business I have learned is that to succeed, you must attack the leader. If your efforts aren't geared to become better than the current #1, then you don't really have any justification for existence. Find the weaknesses, become more efficient, discover hidden customer wants.

Attacking yourself is a problem with Sun Microsystems. They just don't seem able to do it. Oh, they do okay when they’re in the pack somewhere competing for profits, but whenever they find themselves ahead, they have a tendency to sit back and enjoy the scenery. But even worse is when Sun only thinks that they’re number 1. Their release of Java as a competitive development language was brilliant and they managed to put it out there with enough oomph to attract every anti-Microsoft developer on the planet. And they achieved enough momentum and enthusiasm that they thought that they would be able to coast into number 1 position in development languages. Coasting along for the last year, they haven’t been very responsive to the development community. They’ve delayed standards reviews. They’ve stalled on effective IDE issues. And, perhaps worst of all, they stopped pushing Java in broad marketing initiatives.

The result is disaster. Java is steadily losing ground to Microsoft’s .NET and there’s no end to the slippage in sight. By not maintaining their momentum, Sun has allowed Microsoft to overcome their advantages and leverage the millions of existing Microsoft developers into the Internet development arena. As a result, Sun faces more than just the direct challenge of VBScript vs. JavaScript (or even VB.NET vs. Java). Now they face whole architectures they let get out of the bag with what amounts to no answer from Sun—XML, ADO and SOAP to name the most obvious, and groundbreaking, examples.

So have they learned from their mistakes? Unfortunately, the answer seems to be “no”. Scott McNealy seems to have decided that his ego alone can overcome any obstacle and he has failed to give any answers with substance about how Sun will manage to overcome these threats to Java. This is making a lot of developers who have banked on the popularity and utility of Java very nervous. New third-party tools are coming out that help extend the usefulness of Java immensely, but these tools can be expensive and make a poor comparison for a development house that is looking at the ease of VB.NET and comparing that to the hoary, expensive, patch-work monster that Java has become.

Good thing Sun has that lovely server business to fall back on. Um, or maybe not. While Sun was playing around doing whatever they were doing back there, their server market is being eaten alive from the bottom. Sure, Sun makes incredibly reliable servers, but they’re pricey suckers and Sun hasn’t implemented any substantive improvements recently. In other words, they didn’t attack themselves. Their prices didn’t come down until they started losing business and that is far, far too late. Further, consumers have found that if reliable costs too much, they can often achieve the same results with less reliable, but redundant. Thank Michael Dell for that little epiphany. Dell pushed server prices so low that you can afford three of his servers for the price of one Sun. And that’s after a major price-slash on the part of Sun.

My prediction? Sun is in for a tough time. They’re clearly in decline and the pit seems to be bottomless. McNealy and co. don’t seem to have fully realized their danger. Upon hearing that Sun needs to recalibrate, Scott McNealy’s response is simply “We couldn’t be better positioned.” He’s known as a tough guy, but even tough guys get knocked out if they’re not careful. Particularly when they walk around with their eyes closed.

What do you do once you're #1? Same thing. Attacking yourself is a tough thing for companies to do and it isn’t something they typically do well. Those who do, however, will tend to reach number 1 and stay there. This principle is the single biggest factor in the continuing success of Microsoft. Bill Gates is the most paranoid man on Earth and he is convinced that if he rests on his laurels for even a moment, some upstart will come around and nail him. He’s right.