I’ve been musing about software architecture lately and trying to come up with a framework to help choose when to go with more as opposed to less—something that’ll help me feel less arbitrary in my choices. I mean, software design is something of a dark art, but how much of that is inherent and how much is simply being too lazy to formulate good internal guidelines?

My latest ruminations have revolved specifically around Inversion of Control in general and Dependency Injection in specific. Here’s the thing: for the development I do right now at a small reading glasses company, I’m reluctant to implement any of the stronger separation of concerns architectural tools like IoC and a DI framework. The DI zealots would draw from that the conclusion that I am a sub-standard designer and a compromised developer—at least, judging from explicit claims that every project should start out with formal DI (please note both the "every" and the "should").

Pain is an Efficient Teacher

Perhaps too efficient. Most developers have had the experience of working on a project where decisions made in the expedience of the moment came back to hurt them. In a recent DotNetRocks episode, Scott Cate gives an example of hard-coding to a specific search engine and coming to find that they need to generalize in order to allow alternatives. He wishes that they had used a DI framework from the start (in this case, Castle Windsor) to make that an easy change. Scott tells about how they were lucky this need was discovered in time for a major release upgrade so they could take the time to go the full DI route.

There’s two problems with this example when translating to my own situation. First, Scott’s is a commercial product that they sell to large clients and thus the scope is far beyond anything I work on here. Second, I wonder how much pain was really involved in making the change.

You see, pain magnifies itself in memory. You consider all the might-have-beens and how much the problem could penetrate even as you work to ameliorate it. And you consider things you can do in the future to avoid this pain—both the immediate pain and the imagined pain of all those might-have-beens.

The YAGNI Nazi

This tendency to artificially inflate potential scope is the driving force behind YAGNI. YAGNI is the reminder that you can go too far in countering the mere possibility of future pain and that you can plan too far in advance of actual need. I’m a huge fan of YAGNI simply because it has allowed me to realize significant time savings and produce useful software for clients quickly and efficiently.

In many software design discussions, however, I feel like a lone YAGNI voice in an over-designed wilderness. I’m accused of negligence or stupidity (though typically through implication as opposed to outright accusation) because I look for the justification of larger design patterns before blindly implementing them. People make the unearned assumption that understanding a given principle must mean that I would adopt it. This comes through loud and clear in the discussions with DI proponents. It seems entirely foreign to them that I could both understand it and decide to give it a skip on some of my projects.

Sometimes this judgement comes in the form of a logic chain. This happens when designers use the strengths of one principle to piggy-back in a second. Unit testing is most often the piggy-back victim here. That’s because the benefits of unit testing are well-understood and apparent in projects of even minimal scope. Scott Cate admits in that DotNetRocks episode, for example, that a large part of his current push for the Model View Presenter pattern as well as DI is to facilitate unit testing (when he also took pains to point out that implementing Castle Windsor is not without cost he went on my list of DI heroes).

The problem with a logic chain, however, is that you’re right back where you started if a link breaks. What if I can achieve strong unit testing without IoC and DI?

So here is my YAGNI wisdom (for what it is worth): you don’t protect yourself from future change by throwing every possible technique or pattern at every project. You protect yourself from future change by refactoring ruthlessly to keep your code as change-ready as possible.

The Bottom Line on Dependency Injection

So here’s where I think I’ll come down as a rule on implementing DI: I’ll use it as soon as I actually need to support multiple services in a given object. Since I don’t need DI in order to mock dependencies in unit testing (yay TypeMock), this means that I’ll look to implement DI when I have to support multiple services in a given object in production. I have two examples of related projects here that will serve as useful test cases.

The first is a need to integrate customer orders into Dynamics GP. We have many customers who send us orders in a variety of ways and with a variety of processes for approving and tweaking them before we let them hit our system. It’d be nice to have a single module integrated into Dynamics GP that can accept orders in the variety of formats we end up with and go to town. I’ve already got this project up and running for a single customer using a generalized interface and I think DI can help extend that further faster than I can without it.

The second is the initial processing when we receive EDI documents from our clients. Since EDI X12 isn’t really a standard so much as a soup of possible ways to do things, each customer tends to put their own unique spin on what fields are significant and how they should be interpreted. I’ve hated our current EDI software for months (and not just because it uses FoxPro at its base) and I’d dearly love to pull that processing into an internal framework I can use and extend on my own.

Both projects will take me a while, which is appropriate because I’ve been saying all along that DI seems like something suited to larger-scoped projects. I’ll post my experiences as I go, assuming I actually get the time to tackle either one.

Both of these examples are also an inversion of typical DI examples because in both I technically have a service that needs to act on multiple objects. That said, I’ve been an architect long enough to understand that while services and objects might differ conceptually (though defining either is an exercise in nit-pickery), it’s trivially easy to turn each into the other (if you bother differentiating them at all—see nit-pickery above). Sometimes it is quite useful to invert them as I plan to do here.

In many companies developer career progression is deceptively straight-forward; Jr. Programmer, Programmer, Sr. Programmer, Team Lead, Architect, Sr. Architect, Bob (Bob being the semi-mythical entity referred to in obscure comments, worshipped by now-extinct aboriginal tribes, and rumored to haunt the sub-sub-basement).

The differentiation between these positions starts off with how much you know. A Sr. Programmer is a Jr. Programmer who knows his tools inside and out and can complete assigned tasks quickly and without a lot of supervision. Around Team Lead time, however, progression stops being about what you know and starts revolving around your ability to choose wisely between competing trade-offs.

This is an easy transition to miss if you aren’t paying attention. Indeed, I’ve known "Architects" that I wouldn’t ask to design a "Hello World" application. Because being a good architect is about developing good judgement, there are two things that are essential to becoming and remaining competent.

Know Your Stuff

Yeah, I know I just said that being an architect isn’t about what you know. That’s only slightly true. What changes between Sr. Programmer and Architect is the depth of your knowledge—you have to penetrate to the reasons and complexities behind patterns, practices, and technologies. You go from knowing what an n-tier architecture is (and how to create and maintain one) to understanding why that architecture was invented, what problems it solves, and what costs are involved in creating and maintaining one.

Take, for example, the "architecture" I found in coming to a new position. A number of projects are used for this intranet application, each in its own application space. Here’s a short illustration of the structure of two of the solutions.

As you might guess, BOL stands for "Business Object Layer", DAL for "Data Access Layer", and OBJ for "Objects". You might not be surprised, given the closely related project names, to learn that each project contains classes that exist in the other. In short, despite having the structure and using the terminology of n-tier architecture, this isn’t actually anything of the kind.

The developer responsible for this design is not an architect, no matter how much experience he may have, because he has failed to understand when and why you would use an n-tier architecture. Understanding the reasons behind n-tier architecture would make it impossible to create this structure.

Understanding architecture takes hard work. It requires more than reading blogs, attending conferences or purchasing books (though each of those are certainly useful in developing understanding). True understanding requires thinking things through and asking tough questions. It requires breaking down concepts until you can identify and understand their component parts. Often, it takes asking the questions that risk confronting a room full of people glaring at you in varying degrees of shock. Most of all, it takes admitting that you might not know something and working to rectify that lack—or, harder still, being willing to re-examine assumptions (or even conclusions) when presented with new information.

The Development Twins: Cost and Benefit

It’s easy to forget that every technology, architecture, and practice has two sides; a cost and a benefit. To borrow from Heinlein, "There ain’t no such thing as a free lunch." Because making choices is hard, it’s easy for people to remember only one side of a given technology, architecture, or practice—to see only the cost or only the benefit.

This is particularly true in new processes or designs. We go through so many silver bullets at least partially because people teaching new processes tend to concentrate on benefits. This is understandable even if the presenter isn’t financially tied to your adoption of the Great New Thing™ because the benefits are the reason the Great New Thing™ exists. This means that you’ll be taught the benefits, but you’ll typically have to discover the costs on your own.

It’s also important to bear in mind that developers have a natural bias towards solving problems, not finding them which leads to a tendency to grasp benefits quickly. Because architects are grown from developers, it’s important to examine and balance this potential optimistic bias. And it is a balance. Ask an average Linux geek about Microsoft if you want to see a bias towards cost.

The Scope Multiplier

A project’s scope has a multiplicative effect on both cost and benefit. Understanding this became explicit for me in a reply I made to a recent comment by Udi Dahan.

I’d be willing to bet that the benefits of DI, SoC and other good design practices increase exponentially with the number of developers while their costs are a linear progression. Huh. That bears some thought...

For any given process, you need to understand how both benefit and cost are affected by the scope of your project. Practices that scale benefits exponentially and costs linearly are excellent when the scope is large. Indeed, the larger the scope, the better the net gain from applying the given practice.

A corollary of that evaluation is that practices designed for large scales are less likely to provide a net benefit in a small team with a small project.

It’s possible that project scope can be calculated separately for project complexity and team size, but I’d have to give that some more thought. Since high complexity projects tend to require larger team sizes, you’d seldom have one without the other (making this a moot conjecture). That said, my initial instinct is that something like n-tier is more important vs. complexity whereas the benefits of SoC scale more based on team size (though it helps mitigate complexity as well). It is possible that this difference may play an important role in how and when you’d apply each.

Time

That pesky fourth dimension can also have a dramatic effect on cost and benefit. A quick down-and-dirty project with limited scope may seem like it wouldn’t benefit from any real architecture at all. That evaluation can change drastically if the project is likely to change a lot over time. Most developers have been ambushed by application maintenance at some point because they hadn’t planned adequately for change.

Predicting the future is a dubious pursuit at best, however, so be careful here. There’s a reason YAGNI became a well-recognized acronym. Don’t forget that a project can be refactored into a useful pattern after the fact if circumstances change to warrant it—particularly if you have prepared for easy refactoring (unit testing, anyone?).

The biggest problem with evaluating the effect of time on your project is that it is literally impossible to measure the impact of the choice not taken. As a result, I’m always very careful when I hear something like "If only we had..." or "Good thing we...". Personal, anecdotal experience is error prone at best—something to keep in mind during project postmortems.

The Courage to Choose

The most important thing an architect does is decide. The architect chooses the technologies, the patterns, and the practices to apply to a given software development problem knowing that success and failure can hang on the effects of those choices. That’s a lot of responsibility. The impulse to push your choice off to a vendor, or some guru’s public statements, or industry consensus is a seductive one that can work for a time. Reality being the harsh b**tch she is, however, it isn’t a safe impulse to indulge. Personally, I prefer to hold my fate in my own hands.

I found Scott Adams' (yes, that Scott Adams) blog post today about The Power of Choice interesting. Particularly at the end where he says this:

The next time your mate or co-worker is butting heads with you over a decision, recast the situation as their choice.

For example, let’s say you favor Option A, and someone else wants Option B for reasons that seem to you irrational. You are at an impasse. Change the question to this:

“Okay, do you want Option A with this risk, or do you want Option B with this other risk? It’s your call.”

When you put things in the form of a choice, sometimes it gives people the only thing they wanted in the first place.

I’ve been using this technique in software development for years with great success but it takes some care. The thing is that "with this risk" and "with this other risk" buries some real temptation to undermine the actual choice. An example of undermining the choice:

• Would you like to use Linux which is free and has broad support and experts available and eager to help where you can have access to the actual developers to fix any trouble we run into?

or

• Would you like to send hundreds of dollars per seat to a corporation that lies and cheats and whose software is insecure, buggy, and closed?

This isn’t a choice, this is merely a statement of your own prejudice. It is a dishonest representation of the alternatives. Presenting a choice like this is a sure-fire way to alienate opposition and polarize discussion. Worse, it undermines trust.

True Choice

Here’s a key I’ve found in offering a client, customer, or business manager a choice: be prepared and honestly reconciled to following either path. This includes an honest and complete evaluation of the options. This preparation manifests as sincerity and is easily sensed by those you work with. The more ego you have invested in one of the options, the harder this becomes. I find it useful to practice not tying my ego up in technical topics and decisions. Non-technical people find this refreshing. Sadly.

The key to offering sincere choice is that it invokes an effect I described earlier this year in Winning Arguments—putting everyone on the same side.

Anything is Possible

One technique I’ve found useful in exploring software development options is to preempt speculation. When a business person comes to me and starts off with "Can we..." I’ll snap off a quick "we can do anything." This statement has the benefit of being absolutely true with the added bonus of turning the discussion to resources, requirements, and trade-offs. There’s no downside there.

"Can we implement a shopping cart like Amazon.com’s so that our customers can get recommendations and save shopping-carts?"

"Absolutely. We can do anything."

Unless they’ve asked me broad questions like this before, this response is a stumper. They are expecting hemming and hawing and evasions. They are expecting opposition. This statement turns their expected opposition on its head. It gets them thinking automatically about why we can’t do this amazing thing.

And here’s a blunt truth: nine times in ten they haven’t really considered their question at all. Consciously or not, they’re trying to get you to do their work for them. Preempt that by giving them as much as they are giving you—an answer requiring as much thought as they put into their question.

Now, you can make this sincere (and less antagonistic) by following up. If they don’t follow up on their own, I’ll add "Let’s explore what it would take." Notice the "Let’s". i.e. we’re working together to explore our resources, requirements, and options.

It doesn’t take long for people I’ve done this to before to start thinking about resources and requirements before they approach me. The sooner you can get people thinking in terms of trade-offs in software development discussions, the sooner you can get to the meat of those discussions.

And really, every software development discussion is all about trade-offs. That’s because we really can do anything with software—if it can be conceived, it can be done.