In many companies developer career progression is deceptively straight-forward; Jr. Programmer, Programmer, Sr. Programmer, Team Lead, Architect, Sr. Architect, Bob (Bob being the semi-mythical entity referred to in obscure comments, worshipped by now-extinct aboriginal tribes, and rumored to haunt the sub-sub-basement).
The differentiation between these positions starts off with how much you know. A Sr. Programmer is a Jr. Programmer who knows his tools inside and out and can complete assigned tasks quickly and without a lot of supervision. Around Team Lead time, however, progression stops being about what you know and starts revolving around your ability to choose wisely between competing trade-offs.
This is an easy transition to miss if you aren’t paying attention. Indeed, I’ve known "Architects" that I wouldn’t ask to design a "Hello World" application. Because being a good architect is about developing good judgement, there are two things that are essential to becoming and remaining competent.
Know Your Stuff
Yeah, I know I just said that being an architect isn’t about what you know. That’s only slightly true. What changes between Sr. Programmer and Architect is the depth of your knowledge—you have to penetrate to the reasons and complexities behind patterns, practices, and technologies. You go from knowing what an n-tier architecture is (and how to create and maintain one) to understanding why that architecture was invented, what problems it solves, and what costs are involved in creating and maintaining one.
Take, for example, the "architecture" I found in coming to a new position. A number of projects are used for this intranet application, each in its own application space. Here’s a short illustration of the structure of two of the solutions.
As you might guess, BOL stands for "Business Object Layer", DAL for "Data Access Layer", and OBJ for "Objects". You might not be surprised, given the closely related project names, to learn that each project contains classes that exist in the other. In short, despite having the structure and using the terminology of n-tier architecture, this isn’t actually anything of the kind.
The developer responsible for this design is not an architect, no matter how much experience he may have, because he has failed to understand when and why you would use an n-tier architecture. Understanding the reasons behind n-tier architecture would make it impossible to create this structure.
Understanding architecture takes hard work. It requires more than reading blogs, attending conferences or purchasing books (though each of those are certainly useful in developing understanding). True understanding requires thinking things through and asking tough questions. It requires breaking down concepts until you can identify and understand their component parts. Often, it takes asking the questions that risk confronting a room full of people glaring at you in varying degrees of shock. Most of all, it takes admitting that you might not know something and working to rectify that lack—or, harder still, being willing to re-examine assumptions (or even conclusions) when presented with new information.
The Development Twins: Cost and Benefit
It’s easy to forget that every technology, architecture, and practice has two sides; a cost and a benefit. To borrow from Heinlein, "There ain’t no such thing as a free lunch." Because making choices is hard, it’s easy for people to remember only one side of a given technology, architecture, or practice—to see only the cost or only the benefit.
This is particularly true in new processes or designs. We go through so many silver bullets at least partially because people teaching new processes tend to concentrate on benefits. This is understandable even if the presenter isn’t financially tied to your adoption of the Great New Thing™ because the benefits are the reason the Great New Thing™ exists. This means that you’ll be taught the benefits, but you’ll typically have to discover the costs on your own.
It’s also important to bear in mind that developers have a natural bias towards solving problems, not finding them which leads to a tendency to grasp benefits quickly. Because architects are grown from developers, it’s important to examine and balance this potential optimistic bias. And it is a balance. Ask an average Linux geek about Microsoft if you want to see a bias towards cost.
The Scope Multiplier
A project’s scope has a multiplicative effect on both cost and benefit. Understanding this became explicit for me in a reply I made to a recent comment by Udi Dahan.
I’d be willing to bet that the benefits of DI, SoC and other good design practices increase exponentially with the number of developers while their costs are a linear progression. Huh. That bears some thought...
For any given process, you need to understand how both benefit and cost are affected by the scope of your project. Practices that scale benefits exponentially and costs linearly are excellent when the scope is large. Indeed, the larger the scope, the better the net gain from applying the given practice.
A corollary of that evaluation is that practices designed for large scales are less likely to provide a net benefit in a small team with a small project.
It’s possible that project scope can be calculated separately for project complexity and team size, but I’d have to give that some more thought. Since high complexity projects tend to require larger team sizes, you’d seldom have one without the other (making this a moot conjecture). That said, my initial instinct is that something like n-tier is more important vs. complexity whereas the benefits of SoC scale more based on team size (though it helps mitigate complexity as well). It is possible that this difference may play an important role in how and when you’d apply each.
Time
That pesky fourth dimension can also have a dramatic effect on cost and benefit. A quick down-and-dirty project with limited scope may seem like it wouldn’t benefit from any real architecture at all. That evaluation can change drastically if the project is likely to change a lot over time. Most developers have been ambushed by application maintenance at some point because they hadn’t planned adequately for change.
Predicting the future is a dubious pursuit at best, however, so be careful here. There’s a reason YAGNI became a well-recognized acronym. Don’t forget that a project can be refactored into a useful pattern after the fact if circumstances change to warrant it—particularly if you have prepared for easy refactoring (unit testing, anyone?).
The biggest problem with evaluating the effect of time on your project is that it is literally impossible to measure the impact of the choice not taken. As a result, I’m always very careful when I hear something like "If only we had..." or "Good thing we...". Personal, anecdotal experience is error prone at best—something to keep in mind during project postmortems.
The Courage to Choose
The most important thing an architect does is decide. The architect chooses the technologies, the patterns, and the practices to apply to a given software development problem knowing that success and failure can hang on the effects of those choices. That’s a lot of responsibility. The impulse to push your choice off to a vendor, or some guru’s public statements, or industry consensus is a seductive one that can work for a time. Reality being the harsh b**tch she is, however, it isn’t a safe impulse to indulge. Personally, I prefer to hold my fate in my own hands.