Your environment can have a profound effect on how you develop software. The details of what I discuss here have zero practical meaning outside of the .Net world (though you can probably find parallels in other environments). That's because .Net developers have access to tools that invalidate rules of software design that are fundamentally important elsewhere (before you question whether an environment can effect what is good design, consider the difference between good design in C and, say, Prolog). For .Net, the free availability of a tool like Typemock makes a major design consideration simply disappear—namely, testability. Typemock literally robs the term “testability” of meaning in .Net design considerations. That's a freedom that should leave other developers gasping in envy.

Typemock's Magic

I don't want to be a running commercial for Typemock here (they're certainly not paying me to say any of this), but it is a truly revolutionary tool and its effect on development in .Net needs to be examined and understood. The thing is that using Typemock means that you can unit test literally any public method of any public class, regardless of any and all internal dependencies that class might have. And you can do so without changing the design and/or architecture of your software at all.

In other words, using Typemock means that everything is testable. Unit testable. Seriously. Everything.

Profound Effect

Stepping into a world where literally everything is testable is like stepping into a world where you wake up at a designated spawn point instead of dying—fundamental considerations about what is risky and what isn't need to be remapped. Which makes listening to .Net developers discuss testability considerations of various designs a little like listening to World of Warcraft players who are afraid to get too attached to characters that might die—it seems like a lot of effort for something that has little meaning.

Which is why I had such a hard time reading Jeffrey Palermo's latest blog post entitled “Inversion of Control is NOT about testability”. Since I know Jeffrey Palermo is a .Net developer, my initial response is a big fat “Duh. He must be using Typemock.” Sadly, this is not the case.

Freeeeeeedom!

That's too bad because there are some very interesting things in that post that get obscured by continually dragging in testability. For one, he gives a good explanation of Inversion of Control accompanied by a discussion of why you might want to use it outside of unit testing. His statements about coupling in particular piqued my interest. He said:

NOTE: There is no such thing as LOOSE COUPLING.  There is coupled, and not coupled.  Either a type is coupled to a type or it is not.  More accurately.  Loose coupling exists much like the concept of "cold".  When we say "close the door, you'll let the cold out" on a hot day.  Cold doesn't exist.  Either heat is there or it isn't.  We use cold to communicate, but it serves as a shortcut for the lack of heat.

Now, he got hold of a bad analogy (because there are degrees of heat—it's not a binary value—so there is nothing wrong with describing something as “less hot” just as we say "less coupled") and his point is technically wrong (because whether you call it loose, deferred, proxied, or something else, when you are coupled to an interface that a class implements, you have a relationship to that implementing class that is similar enough to coupling that an adjectival modifier is an acceptable descriptor) but the core of his point is still a good one. Tying your class to an interface is a profound change and it is probably a good idea to examine what “loose coupling” means (i.e. how “loose” modifies a coupled relationship).

He later said something I found profound:

In order to do anything useful in software, you must couple. You must couple to something. (emphasis his)

and he later concluded with

You must decide where coupling is appropriate and where it is not.

Importing Designs

Here's why it's a tragedy that Jeffrey didn't make use of his Typemock-granted freedom from testability concerns: that's all he said about coupling. He wasted so much unnecessary time with things that have no value or meaning in a .Net environment that he didn't better explore what coupling means and when to mask coupled relationships.

And here's my wider point: concerns about testability are imported from other environments. .Net folks discuss testability as if it has value in and of itself (because in other environments it does). We forget that testability is no more of a first-order good than quality is. “Testable design” only has value in the things it allows us to do—namely, unit test our classes. If we can unit test our classes as easily no matter what design patterns we choose, then that frees us up to explore other aspects of design choices. It isn't that <le design du jour> ceases to have value, it's just that testability is no longer a factor in evaluating its utility.

Those that realize this will have a competitive advantage over those who do not. If I can test no matter my project structure, then I can choose a design that more exactly fits other aspects of my development needs. Which means I can allocate resources more efficiently than someone who has design concerns that don't fit the actual environment. I can choose <Der Entwurf des Tages> when and where it is actually, immediately useful instead of doing so in order to achieve testability. And efficiency kicks butt in the marketplace.

So, uh, forget all I just said. Spend lots of time making sure your .Net projects are testable. Also: Typemock sucks. Don't bother going there...

You might say that I’ve had issues with Dependency Injection in the past. Well, with all the things I’ve learned not to do, I thought I’d go into a case where I’m considering dependency injection because it looks like it might be a good fit.

Project Background

Like many (most?) developers, I’ve been involved for many years in a particular vertical market. Like most, my involvement in that vertical market includes acquiring a substantial amount of very specific domain knowledge. I’m sometimes reluctant to admit it, but I spent a good decade and more of my career writing software designed for Multi-level Marketing companies. Specifically, I wrote programs that calculate the commission payments to their distributors. If you are familiar with that domain, my condolences. If you are considering entering that domain, talk to me (it’s not all bad).

Now, many accounting software packages deal with paying commissions, but compared to even the simplest MLM, their capabilities are pretty primitive. I won’t go into the full complexity of walking a tree with money on the line, but here’s what’s important for this post:

• The data you have to process is well contained, finite, and known in advance.
• Data concurrency tends not to be an issue (you read distributor information at the beginning and you write payout and promotion information as you go or at the end).
• Every company has their own, unique twist for qualifications, promotions, and payouts—indeed, commission plan differentiation is a key business differentiation.
• The only thing absolutely common to every commission plan is that you walk a tree of distributor nodes, processing orders at those nodes, and calculating payouts and/or promotions on them.

Most programmers working in that domain spend time designing their dream "bonus engine". One that can encapsulate the common elements so that their programming only has to deal with the payout and promotion logic. This turns out to be harder than you’d think to do right (if only due to the temptation to put more in the engine than really belongs there).

My Bonus Engine

Since I’m still getting occasional contract work programming commission payouts (and because I’ve seen a bonus engine or two and know what works and what doesn’t), I went and created an OSS bonus engine. Those of you dying to critique my code-fu to bring me up for well-deserved mockery, here’s your chance. So far I’m the only dev on the project, but that’s only because I don’t know anybody else masochistic enough to join me (hey, if you’re that nuts, let me know).

Right now, this engine is still pretty raw, but it works. Mostly. There are INode and IVolume interfaces for distributors and orders respectively. There’s a NodeBase object that implements INode with nifty events already implemented for easy inheritance and feedback and there’s a BonusManager object used to kick off a bonus run.

The tricky bit for this post is the data access. You see, when programming commission payouts, I seriously don’t care where the data comes from or where it is going. My clients all have their own tricky little data schemas and I’d just as soon not deal with the ins and outs of their systems. Abstracting that part out is beneficial all round.

In my initial design, I figured that something like that is a classic call for an Asp.Net-like provider model. This works well because the client can implement a provider (or have me create one for him) that we can drop in and we’re golden. That means that my eventual Distributor objects can be ignorant of all data implementation details and make a call to MLMBonus.DataProvider.BonusData.Promote() to notify of a promotion (for example).

Making the method references static seems like a good call because the distributor object has no other contextual information available. The engine is walking a tree and calling interface methods so the distributor and volume objects don’t really know much more about the outside world than their relationships to each other.

Because this is my first ever implementation of a "Provider Framework", I went digging for some sense of what kinds of things to look out for. While I initially went with an interface structure, I eventually decided to use Joel Ross' well written guideline for creating your own .Net providers.

There’s Always One

Unfortunately, I got some push-back when talking with my favorite client. You see, he’s chafing at the static nature of my provider model. He wants to make a grab for the holy grail of bonus commissions—executing mini-commission runs on-demand (most likely for partial distributor trees)—and the static provider just isn’t cutting it for him. mini-commission runs allow you to provide some interesting features to your distributors and that’s never a bad thing.

To support his functionality, he would need to be able to alter the data provider "on the fly" and/or be able to initiate multiple runs in the same application space without allowing them to step on each other. There’s just no way that static providers are going to work for that functionality. Naturally, I’ve been thinking about how best to work out meeting his need (mainly because I can see this feature will be something others will like but also because he’s my favorite client). Fortunately, my engine isn’t widely disseminated so it’s a good time for wholesale architectural alterations if we want to make them.

I’ve considered a number of ways to tackle the problem, playing with things like anonymous methods and context managers. They all turned ugly fast so it’s time that I face the fact that this functionality is pretty much tailor made for Dependency Injection.

Where To Inject

So where’s the best place to inject this little dependency? There are four options as far as I can tell.

• constructor injection
• property (or setter method) injection
• method parameter injection (since the number of methods affected will be very small).
• framework injection (Spring.Net or Castle Windsor for example)

Constructor and property injection are both object level references carried by the dependant object and have little to distinguish them in effect. Conceptually, I’ve heard that constructor injection is best for dependencies that are crucial to an object—something the object simply cannot do without. I think I agree with that guideline but I also like the convenience of being able to do grunt-work things in constructors, so if I were in charge of the objects, I’d include both a base constructor and one that includes the dependency as well as a setter method of some kind.

Method parameter injection is tempting, though. With method parameter injection, a client can set the dependency on BonusManager and rely on BonusManager to send the dependency to the Pay and Promote methods on INode. That way the reference to the data provider is short lived inside INode and IVolume objects. Thinking it over, though, I’m not sure why I’m so worried about carrying a bunch of references when garbage collection isn’t going to happen mid-run anyway. Yeah, there’s some overhead involved in tracking the reference on each INode object, but it’s negligible (are pointers in a 64 bit environment 64 bits long? They’d pretty much have to be right? I’ve never bothered to find that out—it’s never been significant before).

Framework injection simply isn’t going to be helpful in this case, I don’t think. For one, I’m not familiar enough with any DI Frameworks to make it worthwhile. For another, I don’t want to introduce a framework dependency on the project. And finally, I don’t want to commit future clients to a DI Framework that may be an issue in their environment.

Injecting

So my next task for the engine will be to muck around with the EngineInterfaces library. I’m thinking I’ll leave the provider object as-is (the BonusDataProvider abstract class) and simply add a property to INode to inject that sucker. In addition, I’ll update NodeBase to have an appropriate property and a new constructor that allows you to set the provider property on object creation.

Since I have the provider stuff already in place and no changes are needed to make this work, I’ll likely leave it alone. That way I won’t break anything currently working.

The only thing left is to decide if I want the IVolume dependency injected via a field or a method parameter on Pay. I’m pretty sure I don’t need the fine-grained control of allowing different orders for a single node to pay down a different path, but do I really want to commit to that? My arguments above against method parameter injection seem pretty solid for the IVolume object as well, but I haven’t completely convinced myself that they apply as strongly.

A couple of months ago, I mentioned a project I had coming up that might benefit from using Dependency Injection. The use-case on this is simple enough. We receive all of our EDI text files to a specific directory. Those files need to be processed into standard, internal temporary tables. Since each of our vendors uses the EDIFACT X12 fields differently, we need to customize parsing the files according to vendor.

This is actually one of the larger projects I undertake at our small in-house development shop so it needs more up-front architecture than most and it seemed like a good time to test my understanding of Dependency Injection and maybe develop some familiarity with a DI framework. For one, I know in advance that I’m going to want an interface for my EDI processors and I know for certain sure that I’m going to add file processors in the future.

Those of you familiar with DI might be able to see the ripples around the reefs I eventually cracked up on from here.

The Basic Solution

The solution isn’t that complex in theory.

• I’ll want a service that monitors a given directory for new files with a given file mask. Both the directory and file mask are in a config file that reloads when altered (all handled by built-in .Net processes).
• I’ll want a simple interface that the service can share with implementing classes.
• I’ll want classes to process the files that are found.

See, simple.

Failure

As much as this looked like a DI shoo-in at the time, it turns out to be a bad fit. I figured this out after digging through the Castle Windsor documentation trying to find a way to iterate through all the registered objects that implement a specific interface. I was frustrated at not being able to find one. It took me a while to realize that I was trying to bend the framework to do something it hadn’t been intended to do—invoke methods on arbitrary registered objects. A DI framework is supposed to handle injecting a class that implements a given interface into the class that depends on that interface.

Indeed, DI frameworks in general are meant to abstract and centralize the complexities of implementation object selection. As such, they are built around the assumption that the run-time already knows what class it wants to use for a given interface (either in a configuration file or determined programmatically at run-time).

It took me a while to figure out that my use of the pattern itself is at fault in this mismatch. I’m not injecting a dependency at all. My service isn’t dependent on classes that implement IEDIProcess, it just determines if there are any available that apply to a newly created file and invoke it correctly if there are. In other words, I’ve already architected around the conceptual dependency here and Dependency Injection is an additional complexity with no purpose.

If DI should be used anywhere in this solution, it’s in the processes themselves to separate out the data updates. And, as I’ve mentioned before, there’s no payoff for me in worrying about my data dependencies.

Busted

So it turns out that shoe-horning a design pattern just to play with it didn’t work out too well. Who’d have thought that could happen?

The broader take-away, at least for me, is that you look for a solution/pattern/tool based on the attributes of your problem. In other words the problem comes first and the solution flows from it. Having a pattern looking for a problem to solve is like having a hammer looking for something to fix—both methods are hit or miss.

Reading the beginning of Joel’s second section of his talk at Yale clarified one reason I find myself so at odds with much of the hard-core Dependency Injection crowd (has Joel really achieved the level of fame that we can dispense with using his last name as Phil Haack suggests? Did you know who I meant right off?). Anyway, I am an in-house developer in a small company and that has a huge effect on my architectural decisions.

In-house Development

I described it a couple of months ago as simply "small company development", but Joel’s right that the more significant aspect of it is that it is in-house—technically, my company probably has close to the same number of employees as Joel’s Fog Creek does, but our development environments couldn’t be more different. Now, Joel describes a bleak picture of in-house development and to be honest, it’s mostly true (particularly on the consulting side). I’ve been fortunate enough that my full-time gigs have been very high quality (alleviating much of the down-side that Joel dislikes so much). His first point about not being able to explore new technologies and never being able to do things "the right way" hasn’t applied to me, for example.

Return on Investment

What he got right in a universal way, no matter the quality of your in-house programmer job, is this bit.

Once the core functionality is there, the main problem is solved, there is absolutely no return-on-investment, no business reason to make the software any better.

It is this feature of in-house development that elevates the status of YAGNI to near-absolute authority. With in-house development, you won’t win more customers with new bells and whistles, extra flexibility, or optimization of pretty much any kind.

Dancing While Juggling

It’s not just return on investment that hampers the in-house developer, though, it’s also the environment in a broader sense. While the environment is relatively stable (the next section explores how that plays out), it is also dominated by large external forces. Whatever market niche your company occupies, the chances are that there is a 500 lb. gorilla creating software for your business. Barring an actual niche vendor, business software like SAP, Dynamics, or Solomon are happy to come in and run your business. The 500 lb. gorilla’s software likely handles 80% of the software needs for your company. Unfortunately, it’s the extra 20% that provides the true market value for what your company does (good ole 80/20 rule. Where would we be without it?).

In-house developers find themselves dancing around this 500 lb. gorilla finding (or creating) integration points to make it do what the company needs. If the business software is good, you develop a love/hate relationship with it as you poke and prod looking for the openings you’ll need to do your job (if it’s bad, it’s a simple hate relationship—this can get bad enough that it becomes a good idea to look for change, either of software vendor or job).

Since management is generally clueless about computer stuff and couldn’t care less as long as they can do what they want, they’ll spend their time thinking up new things to do. Phrases like "wouldn’t it be cool if" are nearly as dangerous as "hey, ya’ll, watch this". These requests and changes come flying in with irregular frequency and require deft handling by the development manager (whatever their official title) if you want to have anything like a chance to get necessary changes implemented.

So you end up dancing around your business software vendor while management continually tosses new shiny objects your way. You need a specialized kind of Agile here if you’re going to function.

Programming for Change

Institutional ADD aside, the environment itself tends to be relatively stable and, best of all, predictable. This makes planning for the future relatively simple. Your degree of certainty about environment and usage is way higher than it would be in a computer software company. I know that there is literally zero chance that I’ll be asked to make my programs run against MySQL or Oracle, for example (about the same chance I have of needing to run on Linux or Mac).

This environment makes issues like Separation of Concerns and its twin offspring Inversion of Control and Dependency Injection much less... pressing. At the very least, it means that The Last Responsible Moment can come much later than it does in "true" software development companies.

A Shipping Example

I’ll use programming a shipping interface using vendor-provided online web services to show what I mean.

A Shipping Interface in a Software Company

Now, if I were at a software company looking to hit against a vendor-provided shipping web service, I can pretty much guarantee that whatever anybody says, the scope of this project is going to include multiple vendors and frequent changes. I would be best off planning early to keep the query mechanism for the service general and make sure that the internal and external expectations are separated. I’d probably want an IShippingService interface (at a minimum) and I’d probably better look at services from several different shipping companies to see what kind of commonalities and features I want to ensure are supported.

The extra time up front spent on architecting a generalized structure will pay off in spades both as the services themselves change and as new shipping vendors come and go with the flexing of the marketplace. The chance of change approaches certainty.

An In-house Shipping Interface

Now consider the same need in an in-house environment. The biggest obvious difference is that your requirements are likely to be as specific (and short) as "tie our shipping rates to the FedEx web service for pricing". Nine times in ten, your shipping department has a detailed contract with a specific shipper complete with daily pickup schedules and a computer in the warehouse that actually belongs to the shipper printing out labels and such like. The chances of using a different shipper, let alone multiple shippers, is remote (if you’ve never worked in wholesale, this homogeneity can be a surprise).

In this environment, spending a lot of time generalizing your solution up front is unlikely to ever pay off. Your best bet in this case is to isolate your shipping module in its own class library to encapsulate it somewhat and to allow you to reference it in multiple places if necessary. You can crib much of the data structure right from your shipping vendor without huge concerns that you are backing yourself into an untenable corner you’ll later regret.

Running Blind with Scissors

Now, could it happen that you need to make substantial changes to your in-house shipping module? Absolutely. And it may be that your shipping needs are as complex as it would be for a computer software company (particularly if you have to ship wholesale into Canada <shudder>). All that I am saying is that the Last Responsible Moment can come after the fact. The conclusion from my small company development post would probably be better if an in-house development modifier were added.

The best "lean" skill [an in-house] programmer can develop is to program with an eye on refactoring. Moving to something better when it becomes clear that it is needed isn’t a lot harder than if you had implemented it in the beginning (and has the added benefit that you know it is needed) if you have kept refactoring in mind throughout. Leave "quick and dirty" programming for the amateurs. Solid refactor-ready code is the best legacy you can create and one that doesn’t age as people get over themselves or discover some new Great New Thing™.

Being ready and willing to refactor ruthlessly is the best lean practice you can adopt as an in-house developer.

Environmental Architecture

I’ll call architectural choices that have a pervasive impact on a project "environmental". Things that change how or where you instantiate objects (acting as a replacement for "new" for example), or that require that all your classes are tied to interfaces, or that force isolation of your UI layer all have a global impact. Certain patterns and techniques penetrate deeply and fundamentally and as such, they need to be used with care and, more importantly, understanding.

The thing about environmental architecture is that it requires a familiarity with the specific technique or pattern in order to understand anything that is happening in the project. As such, its cost piles up in a small in-house development shop. Before too long, the only person who can make any changes to the code is the poor sap who put it there in the first place. Environmental architecture has inertia and weight that needs to be considered specifically and individually on a project by project basis.

Note that this has more to do with the project size and reach than it does with the architecture itself. If you’re doing a lot of small, well-isolated projects (as most of my development projects are), environmental architecture doesn’t have the time or scope to amortize the skill dependency cost let alone implementation and support costs involved.

It’s this dynamic that makes YAGNI king in small-company in-house development.

Dependency Injection Cross-talk

All of this came together for me while reading Joel’s Yale talk. Much of my discussion with DI advocates has these buried assumptions that our development needs and environments are substantially similar. I’m not sure this is the case. I deal mostly with small, discreet projects that take days (in rare cases weeks) to finish. My deployment can be as simple as a one-click "publish" from the developer’s machine.

Various fanboys and architect nazis want to dismiss me or make me feel inferior because I don’t adhere closely to their Great New Thing™. Whip me, beat me, call me "Mort", but the fact of the matter is that I don’t think it is wise to cost my company the time and effort let alone commit them to spending that time and effort in perpetuity just so that I can join the in crowd. The warnings of doom and gloom simply have not panned out in the reality I occupy.

I’ve been putting off a follow-up on Dependency Injection for a couple of months now. The amount of heat I anticipate receiving is so disproportional to the probable light gained that it makes me hesitate. This weekend, I picked up on a stream of referrals from a post at InfoQ that mentions my Dependency Injection post (though not the follow-ups). It does a reasonable job of spelling out the conversation that happened, though I was feeling picked on until it brought in Eli Lopian’s contribution to the discussion. In whole, it’s a good summary. The real pile-on happens in the comments and it illustrates so much of what I dislike in software development sloganeering that it has jolted me out of my reluctance to respond (I’d have responded there, but you have to register to comment and I hate that).

There are two things that contribute to the silver-bullet, band-wagon boosterism that sends up red flags for me.

Good for What Ails You

This is characterized best by a comment by Steven Devijver when he says

It’s an interesting discussion. Just as interesting as asking oneself: shall I wear underwear or shall I try something really nifty today?

It’s the same kind of thing pointed out by Jay Kimble about a month ago as he expressed his concern with Dependency Injection.

I have been asked "Why don’t you like this pattern or that pattern? I mean [Some famous blogger] says s/he always uses them.  I think they are smart or at least they seem to be.  You must be a 'Mort' or a pretender..."  At least that’s the way it has always been stated to me.

The claim made by these individuals is that The Pattern (it can be any pattern, but this is increasingly frequent when referring to Dependency Injection) is universally applicable and should be used in all cases, preferably by default. I’m sorry, but this line of argument only shows the inexperience or narrow focus of those making the claim.

Claims of universal applicability for any pattern or development principle are always wrong.

Study any pattern or development principle in enough depth and you’ll find the edge cases and counter-indicators applicable to it. This is true of even basic principles like data normalization and object oriented design.

Variations on the universal applicability include the pseudo cost-benefit comments like that from Gabriel Lozano-Moran.

If you anticipate [no] potential future problems using a DI framework like Spring.NET and it takes you a couple of minutes to implement this hardly violates the YAGNI principle.

I call this "pseudo cost-benefit" because it seriously under-values initial implementation, training, and the burden of long-term skill set dependency created by implementing so invasive a pattern—a point I made at the end of my post about small company line-of-business software development a bit back. Yeah, you can probably add Spring.Net to a given project in just a couple of minutes, but that treats Spring.Net familiarity as a sunk cost (or worse, null cost) and commits that project indelibly to Spring.Net familiarity. You may feel that familiarity with Spring.Net is a given for any competent developer, but that proves my point that this is merely a variation of treating DI as a given for all projects in all situations.

Save Me From Myself

This argument is made in a comment by Ole Friis.

When I use DI, I see it as an advantage that I am forced into splitting up my application in well-defined services with a nice interface.

The term "forced" here indicates that he doesn’t trust himself to split his applications into "well-defined services with a nice interface" on his own. In this, Ole Friis is echoing the response I got from the intimidating Oren Eini when I suggested that TypeMock makes arguments that couple DI with unit testing go away.

The main weakness of Type Mock is its power, it allow me to take shortcuts that I don’t want to take, I want to get a system with low coupling and high cohesion.

The implication is that he wouldn’t be developer enough to create a system with low coupling and high cohesion if he used powerful tools like TypeMock. Oren is inconsistent here because he’s otherwise all for developer empowerment as illustrated by his support for Jeffrey Palermo’s excellent post regarding treating developers as professionals.

The fact is that nobody is really in favor of limiting themselves. Taken seriously, these kinds of statements indicate a profound lack of trust in their own competence. I’ve never yet met a software developer above Jr. level with that particular insecurity. I’ve met a veritable host who lack trust in the competence of people they work with, however, which is what I suspect is really behind these statements. Either way, it’s hard to take that argument seriously.

I’m the Anti-DI

Or so I’ve been painted. The fact is that I’m not, though my use for it is pretty limited. Since I don’t need it as a foundation for mocking objects in unit testing, it’s only really useful when I have a service with more than one implementation and I’ve found that even then DI isn’t necessarily the answer. I’ll deal with at least two reasons behind this in a future post(s).

I’ve been musing about software architecture lately and trying to come up with a framework to help choose when to go with more as opposed to less—something that’ll help me feel less arbitrary in my choices. I mean, software design is something of a dark art, but how much of that is inherent and how much is simply being too lazy to formulate good internal guidelines?

My latest ruminations have revolved specifically around Inversion of Control in general and Dependency Injection in specific. Here’s the thing: for the development I do right now at a small reading glasses company, I’m reluctant to implement any of the stronger separation of concerns architectural tools like IoC and a DI framework. The DI zealots would draw from that the conclusion that I am a sub-standard designer and a compromised developer—at least, judging from explicit claims that every project should start out with formal DI (please note both the "every" and the "should").

Pain is an Efficient Teacher

Perhaps too efficient. Most developers have had the experience of working on a project where decisions made in the expedience of the moment came back to hurt them. In a recent DotNetRocks episode, Scott Cate gives an example of hard-coding to a specific search engine and coming to find that they need to generalize in order to allow alternatives. He wishes that they had used a DI framework from the start (in this case, Castle Windsor) to make that an easy change. Scott tells about how they were lucky this need was discovered in time for a major release upgrade so they could take the time to go the full DI route.

There’s two problems with this example when translating to my own situation. First, Scott’s is a commercial product that they sell to large clients and thus the scope is far beyond anything I work on here. Second, I wonder how much pain was really involved in making the change.

You see, pain magnifies itself in memory. You consider all the might-have-beens and how much the problem could penetrate even as you work to ameliorate it. And you consider things you can do in the future to avoid this pain—both the immediate pain and the imagined pain of all those might-have-beens.

The YAGNI Nazi

This tendency to artificially inflate potential scope is the driving force behind YAGNI. YAGNI is the reminder that you can go too far in countering the mere possibility of future pain and that you can plan too far in advance of actual need. I’m a huge fan of YAGNI simply because it has allowed me to realize significant time savings and produce useful software for clients quickly and efficiently.

In many software design discussions, however, I feel like a lone YAGNI voice in an over-designed wilderness. I’m accused of negligence or stupidity (though typically through implication as opposed to outright accusation) because I look for the justification of larger design patterns before blindly implementing them. People make the unearned assumption that understanding a given principle must mean that I would adopt it. This comes through loud and clear in the discussions with DI proponents. It seems entirely foreign to them that I could both understand it and decide to give it a skip on some of my projects.

Sometimes this judgement comes in the form of a logic chain. This happens when designers use the strengths of one principle to piggy-back in a second. Unit testing is most often the piggy-back victim here. That’s because the benefits of unit testing are well-understood and apparent in projects of even minimal scope. Scott Cate admits in that DotNetRocks episode, for example, that a large part of his current push for the Model View Presenter pattern as well as DI is to facilitate unit testing (when he also took pains to point out that implementing Castle Windsor is not without cost he went on my list of DI heroes).

The problem with a logic chain, however, is that you’re right back where you started if a link breaks. What if I can achieve strong unit testing without IoC and DI?

So here is my YAGNI wisdom (for what it is worth): you don’t protect yourself from future change by throwing every possible technique or pattern at every project. You protect yourself from future change by refactoring ruthlessly to keep your code as change-ready as possible.

The Bottom Line on Dependency Injection

So here’s where I think I’ll come down as a rule on implementing DI: I’ll use it as soon as I actually need to support multiple services in a given object. Since I don’t need DI in order to mock dependencies in unit testing (yay TypeMock), this means that I’ll look to implement DI when I have to support multiple services in a given object in production. I have two examples of related projects here that will serve as useful test cases.

The first is a need to integrate customer orders into Dynamics GP. We have many customers who send us orders in a variety of ways and with a variety of processes for approving and tweaking them before we let them hit our system. It’d be nice to have a single module integrated into Dynamics GP that can accept orders in the variety of formats we end up with and go to town. I’ve already got this project up and running for a single customer using a generalized interface and I think DI can help extend that further faster than I can without it.

The second is the initial processing when we receive EDI documents from our clients. Since EDI X12 isn’t really a standard so much as a soup of possible ways to do things, each customer tends to put their own unique spin on what fields are significant and how they should be interpreted. I’ve hated our current EDI software for months (and not just because it uses FoxPro at its base) and I’d dearly love to pull that processing into an internal framework I can use and extend on my own.

Both projects will take me a while, which is appropriate because I’ve been saying all along that DI seems like something suited to larger-scoped projects. I’ll post my experiences as I go, assuming I actually get the time to tackle either one.

Both of these examples are also an inversion of typical DI examples because in both I technically have a service that needs to act on multiple objects. That said, I’ve been an architect long enough to understand that while services and objects might differ conceptually (though defining either is an exercise in nit-pickery), it’s trivially easy to turn each into the other (if you bother differentiating them at all—see nit-pickery above). Sometimes it is quite useful to invert them as I plan to do here.

Bob Lee, creator of Google’s Guice project (the Dependency Injection framework for Java) has twice left comments here urging me to check out his talk introducing Guice. I resisted this because a) I don’t do Java and b) I figured I’d had enough with a later video recommended by Nate. This afternoon, I broke down and watched it and I’m glad that I did.

It turns out that the things that grated on me from the first Guice video I viewed came mainly from Kevin Bourillion—mostly unfair comparisons to alternatives and boosterism. Bob is very personable, open and seems honestly intent on communicating both how Guice works and the reasons for their design choices. He’s also very clear when discussing real-world reasons for using Guice. I found the video both informative and refreshingly free of the taint of propaganda that so permeates a lot of the explanations of dependency injection I’ve encountered so far.

For one, there are several times where Bob describe’s the unit testing origins and impetus of Guice. One objection I raised in my original post on Dependency Injection is that people are largely talking around the major motivation unit testing plays in the adoption of dependency injection. At one point Bob shocked me when he actually said, "If there’s any reason to use [Guice], it is unit testing." Pardon me while I swoon.

Early on, Bob gave a list of benefits (and one detraction) to implementing dependency injection by hand. I’ve taken that list and expanded it a bit because it helped clarify my thinking on dependency injection. Below is a matrix that includes Bob’s original Factory example, his original evaluation of DI by hand, Guise DI (from what I can tell, I’m open to correction on this one), and TypeMock. To understand it, recognize that some of the items involve three practical objects: A client that calls a service that needs a service implementation. The service implementation is the piece you’d mock.

	Factory	Manual DI	Guise	TypeMock
1. Tests bypass dependency manager (i.e. a factory-like entity)
2. Explicit binding (cannot create client without providing service implementation)				N/A
3. Reuse client with multiple service implementations
4. Client doesn’t have service implementation dependency at compile time
5. Service doesn’t have service implementation dependency at compile time
6. No factory needed to create either service or client
7. Architecturally neutral
8. No new dependency in non-test code
9. Able to mock code-generated or external objects without wrapping

This isn’t an attempt at a complete list, but it does identify the cleft points in the different strategies for me. The key to loose coupling is there in numbers 3 and 5 which is where TypeMock doesn’t achieve the architectural grace that DI can. The advantage that TypeMock enjoys is in numbers 7, 8, and 9. Guise comes close to achieving number 7, actually, but there’s enough of a learning curve needed to grasp the intricacies of DI in Guise (that could really bite you if you failed to fully understand their implications) that I wasn’t quite willing to give it to them.

Now, how you weight those factors for your projects (and you should be weighing them on a per-project basis) will help lead you to where you might be most comfortable. You can guess how I weigh them in relation to the majority of the projects I work on.

I can’t believe the potent response I’ve gotten on my posts about Dependency Injection. Ayende Rahien responded individually to each of my posts himself, which is more than a little bit intimidating all on its own and a couple of development heavyweights left comments directly. Nate describes Ayende’s posts as the cavalry arriving and links to a couple of other responses. All of these posts disagree with me, though a post by Aaron Jensen indicates that he’s at least willing to consider the possibilities.

All of this should have been foreseeable, really, as soon as I decided to publicly post my skepticism of a technique in the midst of its ascendency just as mature tools are penetrating its development space. Surely something that popular must have some foundation in reality/fact and give measurable real-world benefit, right?

The Benefit of Dependency Injection

Its boosters see DI as assisting in loose coupling of objects. Anyone around when OOP came in and trounced all comers knows the benefits of loose coupling, right? Darn skippy. Benefits include ease of maintenance and being able to refactor elegantly.

The thing is, DI itself doesn’t really do much for loose coupling as I pointed out in my posts. DI alone has little discernable benefit.

DI aids loose coupling when you create container objects or a framework to make dependency management and injection easy. It’s essentially an additional layer of abstraction that allows you to create a dependency on the object or framework in place of the actual objects themselves. This is what Ayende means when he talks about a "powerful container". This is also the key behind Google’s Guice framework for Java and Nate’s own Ninject project as well as the Windsor project mentioned by both Nate and Ayende.

See a pattern there? I did. All the heavyweight advocates of DI rely on powerful, relatively invasive tools to realize the benefits of Dependency Injection. Until you get those tools, the benefits of DI seem mainly theoretical. The fact that the addition of a framework or container objects unlocks the benefit makes me wonder if there isn’t a different pattern effectively at work here.

Theoretical Benefits

Which leaves me where I started, really. You see, context is everything and the context of the vast majority of discussions around Dependency Injection involves using mock objects in unit testing. Bright young developers looking to do unit tests without hitting the database search for tools that will help them accomplish this task. They’ve heard of mock objects and figure that mocking is the key to solving their problem.

The trouble is that everywhere they look, mock objects are telling them about "design for testability" and the importance of loose coupling. It’s such a litany that many of them have come to buy into the party line because hey, "it’s just good design."

I’m sorry, but that is bunk. You cannot separate good design from the context and resources of your individual project. If my client or company needs web pages that validate users against Active Directory and then allows them access to reports based on their group memberships, then telling me that I need some robust loose coupling framework in order to test my user maintenance library is flat out wrong.

Or take a project like Subtext. I love that project even if it’s been a couple of months since I’ve done anything substantive for it. Telling me that the architecture of Subtext should be crammed into a dependency injection mold just to increase our ability to use mock objects in our unit testing makes me shudder (not that anybody has done anything of the sort, mind. I’m talking theoretically here). In the whole, that’s a lot of overkill in order to extend your test coverage.

YAGNI Baby

I don’t want to come across as too much of a shill, but I have to agree with Eli Lopian (CTO of TypeMock, so hardly a disinterested party) when he says this has YAGNI written all over it. Have I been involved with projects that took a hit because a dependency had to change? Absolutely. But I’ve also been with projects that took a serious architecture hit they never came even close to needing. Outside of major corporate infrastructure projects, stringent loose coupling just isn’t as much benefit as more naturally encapsulated OO techniques.

So here we have a lot of architecture big wigs selling DI as best practices and not bothering to contextualize those statements at all. Again, Ayende is a good example of this. In Dependency Injection: More than a testing seam, he wraps up with "Dependency Injection isn’t (just) for testing, it is to Enable Change." Nice. So if I don’t use DI, my projects can’t change at all? That’s his implication and he’s not alone.

It all comes down to cost vs. benefit and that’s what I’m not seeing enough discussion of, here. What are the costs of using DI? Balance that against how often you really need to change dependency configuration. Add in how hard it would be to go from not using DI to using it and what kinds of triggers you’d use to identify when a project is big enough that the risks of architectural change justify the cost of wide-spread DI. All of these proponents are trying to claim that DI has little or no cost and/or that the benefits are incalculable. I’m sorry, but I’m not taking that bait.

Resisting the Hard Sell

Frankly, this whole thing puts me in mind of a classic hard sell technique. A customer comes to you wanting widgets. You tell him he can’t have widgets without blodgers but that’s okay because blodgers have so many additional benefits that really they should be coming to you for blodgers anyway. Indeed, all professionals are already using lots of blodgers and they all agree that it’s a good idea to have them and really you can buy yourself some widgets as well! And you’re in luck because we have plenty of the highest quality blodgers in stock right now!

So it is with DI and mock objects. I want mock objects so that I can test code without actually traversing the wire and hitting a database. Loose coupling is so far down my project priorities that standard OO practices are more than sufficient. Only, everywhere I go for mock objects, I’m being told that all the best projects are loosely coupled so what kind of incompetent fool am I for not implementing DI even before I ever thought of needing a mock object framework?

Fortunately, I grew out of that kind of bullying before I got out of high school. If I ever get into a project where loose coupling is a big enough concern that I have to consider seriously intrusive techniques in order to reduce the risks of dependency change, believe me, I’ll be looking into one of those magic DI containers. There’s some seriously cool mojo there. I just don’t have that need and since I can mock objects without it, I will.

On Better Testing

A lot of people talk about the benefits of TDD. I tend to agree with a lot of their tenets. I’m not entirely sold on "Test First" (the heart of TDD), but I’m to the point where I’m going to give it enough of a shot to see how much difference it makes and in what direction. Since TDD is such a big deal and considered by most to be extremely important, I wonder why they let themselves be hampered by the DI crowd trying to ride their coattails.

The biggest roadblock to ubiquitous testing for me came when I ran into methods that had latent processes embedded in them. This was mostly the case with database access, but also web services or anything else that crossed the network or accessed important global resources. Mock objects were the obvious solution to the problems I ran into. I balked, however, at the degree of change needed to use any of the mock object frameworks I ran across. So I lived with gimped unit tests until I ran across a stronger mocking tool.

I can’t be the only one to do this. How much have we set consistent unit testing back by insisting on selling DI as a prerequisite to using our mocking tools?

Development Principles

Enforcing purity tells people that you’re not as interested in solving their problems as you are in enforcing your code of conduct. "You can have mock objects, but only if you join our priesthood" they seem to say.

Again, Ayende is an example of this attitude when he says, "The main weakness of Type Mock is its power, it allow me to take shortcuts that I don’t want to take, I want to get a system with low coupling and high cohesion." As a personal statement, I guess this is fine. If Ayende doesn’t trust himself to write code appropriate to his requirements and resources, then by all means, he’s free to handcuff himself with less powerful tools.

That’s not at all what he means, though. Too often this kind of statement isn’t intended to actually target the speaker. The speaker is perfectly confident in their own abilities—in this case to create low coupling and high cohesion. It’s the abilities of other developers they want to constrain. It’s another manifestation of the "programmers not doing things my way are idiots" meme used against Visual Basic for so long.

Personally, I’m a fan of powerful tools that let me do things my way. If there’s something I’m doing wrong, please tell me. If there’s a principle or pattern that you think might be useful to me, I want to hear it. I want to hear the best case you can make and as much of the cost/benefit as you can include. But I’ve been around long enough to know that no principle or pattern is useful in every context so I’m just not going to buy arguments that claim that anything that needs X should also be using Y unless you can show me how Y is a fundamental and indivisible part of X.

And I’m sorry but DI is not a fundamental and indivisible part of mock objects.

Infinite Cost Arguments

I had a post a couple of months ago on my personal blog about safety and infinite cost arguments. Summary: I don’t buy arguments that boil down to one side having an infinite cost any more than I buy arguments that claim one side is free of all cost. This tenet applies to the Dependency Injection equation—loose coupling is a beautiful thing, but the absence of DI isn’t a sign of sure disaster.

Frankly, loose coupling isn’t an absolute value anyway, so arguments that every decrease in coupling is automatically worth the cost are absurd on their face. With DI containers and tool frameworks, I’ve come to see that DI can have benefits that aren’t solely mock object/unit testing related. I just don’t think that DI is a fit for every project because the cost is not zero and the alternative cost is not infinite. Frankly, I don’t see DI as a benefit for most projects but then, nobody has bothered to make that argument.

As such, I’d appreciate more exploration in the middle and discussions that admit that mock objects that don’t require dependency injection would be a huge boon for widespread penetration of comprehensive unit tests. Or discussions that explain why I’m mistaken in my statements above.

A Caution

Most developers I know have been on projects where some dependency got spread throughout the code in such a way that any changes in that dependency created a nightmare. I’ve certainly been there. In those situations, the problem was improper encapsulation and the tight binding that brings. Comparisons between DI and those examples are like comparing a bag lady to Ms. America. Yeah, there’s a difference and all, but the fact Ms. America had a manicure isn’t it. If you’re going to make an argument I can buy using comparisons, you’ll need to compare the best of the alternatives to the best of DI.

What I’m saying is that while you should put your best argument forward I’m going to balk at straw men put in place of my position.

And the fact of the matter is that you can get pretty far achieving loosely coupled architecture before you get to DI. I’m not saying DI can’t do it better, I’m just saying that there are a lot of techniques out there that even medium-sized projects already routinely implement that provide perfectly adequate change protection. DI is on top of, not instead of, those techniques.

I’ve been giving poor Nate Kohari a hard time over at Discord & Rhyme. He has been very patient in Defending Dependency Injection. His attitude stands in sharp contrast to Ayende Rahien's post today about testing Linq for SQL. Ayende’s snide "(Best Practices, anyone?)" is exactly the attitude I lamented in my original post on Dependency Injection when I said

I do wish that people would admit that DI doesn’t have compelling applicability outside of Unit Testing, however. I’m reading articles and discussions lately that seem to take the superiority of DI for granted. And I’ve read mock object examples that seem a little bit condescending about dependency injection—as if your projects should already incorporate this technique.

I find myself reflecting more and more on how liberating TypeMock is and come back to wondering again if Dependency Injection would exist as a pattern at all if it weren’t for unit testing.

My hypothesis (entirely untestable, so technically a mere speculation) is that what happened is that in order to achieve 100% code coverage in their unit tests with methods that read databases or call web services or have other potentially nasty dependencies, someone came up with the neat idea to invert control of those dependencies so that unit tests could feed in an abstracted object that fakes those calls and the real logic of the methods could be tested without bothering the poor database (or whatever). That’s a noticeable, pervasive, broad architectural change, though, especially if it is only used for unit testing, so in order to defend their idea, developers began looking for additional justification for this pattern. Since nobody but Microsoft uses providers (and that really only in Asp.Net) and software factories are so last year, they cannibalized the strengths of both and pushed them down to the object level and called it done. I’m not saying they did so maliciously or dishonestly. I’m just saying that they needed further justification and that the justifications they chose are already served by applying other useful patterns.

Seriously, with providers that allow you to contextualize defined sub-systems and something that allowed you to Mock objects in testing without having to feed those objects to the tested class, can you think of a good reason to use Dependency Injection? Any dependency that needs to be altered more frequently than a contextual provider framework would allow needs to be exposed for access by calling objects anyway, I’m thinking.

This thinking was crystallized as I explored some of the DI frameworks referenced by Nate. As I watched Kevin Bourillion and Bob Lee explain their Guice for Java framework (a DI framework created by Google), I was struck by the number of references to testing. From that presentation alone, it looks like the main impetus for Guice was unit testing and the other benefits described would have been as easily satisfied with a provider framework.

Come to that, it could be that solid DI frameworks like Guice and Castle Windsor have a niche when it comes to making providers easy to create, control, and use.

Still and all, I wonder what would have happened if they had invented something like a Java version of TypeMock for their mock objects first.

I got an interesting comment on my post about Dependency Injection from Nate Kohari. I responded on my post and in true blog fashion, Nate developed his response into a blog post. Cool.

Since my continuing objections haven’t yet reached the level of a full-on post, I’ve limited myself to comments there. If you’re interested in the topic, I recommend giving it a look and jumping in if you want to weigh in. I haven’t changed my mind yet, though I can see the attraction for those who use dependency injection frameworks. I have reservations even with the frameworks, though.