One of the things I am seeing less of lately is the understanding that reasonable people can and will disagree with one another—without either of them being any the less reasonable or intelligent for doing so. It seems to me that people become so invested with the “rightness” of their ideas that they deny the possibility that those who disagree with them may be equally intelligent and well-informed. You see it a lot in politics, but I think that this attitude has crept into development discussions as well.

I saw a manifestation of this in action after a recent Stack Overflow podcast wherein Joel had the temerity to question Robert Martin's SOLID principles (SOLID was the topic of a recent podcast with Scott Hanselman that Joel had apparently heard). I highly recommend both podcasts. It didn't take long for some bright stars in the Alt.Net universe to talk about Joel jumping the shark or the state of his imperial wardrobe. I'm not sure why the impulse to denigrate those who disagree with you is so strong, but it appears to be nearly universal. We go from the belief that somebody is wrong to the conclusion that they are incompetent or ignorant without taking time to draw breath, really.

I think this impulse is not only wrong, but damaging. It represents a voluntary limitation of our ability to engage in important dialog and stretch our understanding.

Education

One crutch of those who participate in this hidden hubris is the belief that people who disagree must be missing data. You can see this most commonly when someone tells you to “educate yourself” or its milder form “try it and you'll see”. The underlying message in those statements is that if you knew what they know, you'd do what they do.

And that may be correct. It really could be the case that someone is missing data and if they had that data they might agree. Since software development is so change-driven, much of development blogging is motivated by the desire to teach others things they might not have heard about before. You have to honor the often unrewarded efforts of those who take the time to put information out there for the benefit of those seeking education.

The problem is that telling someone who questions your tools or methods to educate themselves is arrogant, even if you are completely sincere and honestly well-meaning. You see, there are really only two possibilities for someone questioning you: they lack data or they have arrived at a different conclusion after weighing the data themselves.

In the first case, telling them to educate themselves notifies them of your superiority (you are in possession of data they lack) at the same time you deny access to yourself as a resource (you are directing them elsewhere for acquiring that data). It's a dismissive brush off. In the second case, telling them to educate themselves is a judgement of their experience and conclusion without the benefit of explanation or debate. In both cases you are saying that you are better than they are and that they shouldn't be allowed to participate in the discussion as a result.

If you honestly believe that someone disagrees with you due to lack of information, a better response would be a series of questions or even challenges geared towards examining their point better. “Have you considered” questions or “I disagree because” statements are more helpful; they give others the chance to respond and have the courtesy of taking someone seriously enough to invest in the discussion.

Weighty Matters

What if I'm already educated and I still disagree? What if I've done the recon, seen the scene, danced the dance, bought the souvenir and I still beg to differ with your Great Truth? How can two intelligent people look at the same data, having access to the same information, and arrive at two legitimately different conclusions?

Allow me to educate you. (heh)

You see, most decisions are complex and involve competing principles. Do I take the time to add an abstraction layer that will be useful later or do I YAGNI my way through with the simplest solution that works right now? Do I hassle with System.Data.Common to support multiple database providers or allow a strong dependency on SQL Server? Do I use test first to drive out my design or am I confident that my planned design is decoupled enough and tests ex-post-hackto will be adequate? All of these decisions break down into factors that we weight according to our experience and expectations.

I'll illustrate what I mean. Lets take, for example, deciding if Georgette Heyer is better than Meg Cabot. It can be argued (and I would, indeed, so argue) that they have comparable ability with characterization, prose, story, and plot. However, Meg Cabot sets her characters in modern settings and Georgette Heyer's books are chiefly set in Regency England. Two people could easily disagree over which author is best if one has a strong weight on “historical regency setting” without either being any the less intelligent or in need of education.

Back to software development. Consider some of the factors that might go into deciding to use TDD.

• Force strongly decoupled design
• Ensure a minimum level of test coverage
• Early API exploration and (pseudo) documentation
• Commit project to a test framework
• Commit project to a mock framework
• Commit project to automated testing infrastructure
• Writing tests without the benefit of intellisense

How you weight each factor will determine your judgement of TDD. Someone who is not worried about forcing decoupled design (either because decoupling doesn't pay off in their environment or because they feel they can achieve decoupling without force) will be less likely to chose TDD for their project.

And it gets even more complicated when you realize that each factor can, in turn, be comprised of additional factors with their own weight and trade offs (you could easily break “decoupled design” down further, for example).

Learning to look for underlying principles and the weights that others may apply can be interesting as well as useful. More importantly, it opens up the gray areas and opportunities for honest disagreement (without denigration) that are vital when working with IRealWorld.

Educating Yourself

As software professionals, we owe it to ourselves, our employers, and our clients, to be educated in our craft. That's a not inconsiderable burden in a field that grows by leaps and bounds year after year with no sign of slowing any time soon. It is hard enough learning all the concepts, patterns, and practices (not to mention tools, environments, and platforms) that it is often tempting to find a core of experts that you rely on to make decisions for you (and there is some benefit in doing so initially as you come up to speed on the intricacies of our field).

But to be truly educated, you have to go beyond what your experts say and learn about the principles that exist underneath. That's extra work but more importantly it is extra responsibility. It may be that when you break a practice down into its constituent principles that you will find yourself in a situation where “Best Practices” aren't, in fact, best. To me, it is the ability to not only make that determination but to then act on it that truly makes a “professional”.

Many of the interesting .Net bloggers are part of the Alt.Net crowd; evangelizing Dependency Injection, Design for Testability, Test Driven Development, SOLID design and other development practices that they find useful in their work. It doesn't take long reading these blogs to pick up on what looks like an unforgiving attitude towards those who don't use the latest acronyms in their software development.

This acrimony is unfortunate because most often what is at the heart of those who question the standard Alt.Net toolset isn't so much principle as it is context.

A Fundamental Assumption

Unfortunately, discussing software development at all imposes an initial assumption that too often goes unexamined—the assumption that the software development you are doing is substantially similar. I'm afraid that this assumption is violated more often than we realize.

Most prominent development bloggers are consultants, conference speakers, tool vendors, or framework providers of one sort or another. As such, they are used to considering the problems of very large applications with many cross-cutting concerns. Is it any wonder, then, that they are strident about the importance of segregation and testing? For the majority of their hands-on development, everything that increases separation and independence is a benefit because isolating fail points and mitigating the cost of touching individual elements carries a heavy multiplying effect on overall complexity. No wonder they want to use a DI framework on every project and insist that other developers practice TDD. It's all up-side in such projects because the costs disappear in the noise and the benefits stand out.

Not all of us work in those kinds of environments and/or applications, however. Many of us work on projects that are wildly dissimilar. Our applications are narrowly focused, single use, and with limited distribution. For us, implementing solution-wide DI eats up half the project configuration overhead and delivers functionally zero chance of eventual benefit.

I'm unsure how many find themselves in similar (smaller) development shops. I suspect quite a few. We might even be a majority of actual, working developers. We're pretty much absent from the development blogger population, however.

A Problem

What that means for us small-shop developers is that we end up having to do a lot of on-the-fly translation. Up-scale bloggers have shown themselves averse to “thinking down” so we end up having to analyze the methods and techniques they discuss to see where the fail points are for our projects. If your QA team is the folks in the warehouse and your architecture a straight-on table insert, is TDD really going to pay off? What about bug tracking? Or continuous integration?

These translations are interesting and part of why I like smaller development. Our problem space is largely undiscovered country. We can't really trust anything labeled “best practice” and we have to do our own experiments to find what works and what doesn't and at what point it starts to make sense to engage different methodologies.

It'd be nice, though, if some of the gee-whiz bloggers would discuss the fail points of their methods and tools. Or point out the trade offs that various techniques are likely to entail. No technique or method is so useful that it is universally applicable, so where are the borders?

Worse, though, are those who actively stifle discussion of such fail points. Mention that you'd rather not use Dependency Injection, for example, and you can find your professionalism and/or intelligence questioned. Indeed, for many the very fact that you don't <insert favorite practice here> indicates that you must never have tried it. Or if you have you must have been doing it wrong.

This friction fosters resentment and shuts down discussion. Perhaps one reason we don't see more developers willing to discuss alternatives to the popular band wagons of today is a reluctance to enter what amounts to a hostile arena. As the troubles on Wall Street filter down to Main Street (where us small-shop developers live) the last thing we need is other developers calling us unprofessional in public. Particularly when those other developers are well-known and respected in their community.

I know this from personal experience. I'm pretty secure in my current position. My company is healthy and has a marked competitive advantage that I am key to delivering. My boss respects me as do the owners of the company. Even so, my gut clenched tight and my heart sank to my knees when I came across a twitter message from a luminary in our field (someone any informed .Net developer would recognize) dismissing me as unprofessional and my points as unworthy of consideration.

Seeing something like that will make you wonder if expressing your doubts is worth the risk, even when it isn't directed at you.

Your environment can have a profound effect on how you develop software. The details of what I discuss here have zero practical meaning outside of the .Net world (though you can probably find parallels in other environments). That's because .Net developers have access to tools that invalidate rules of software design that are fundamentally important elsewhere (before you question whether an environment can effect what is good design, consider the difference between good design in C and, say, Prolog). For .Net, the free availability of a tool like Typemock makes a major design consideration simply disappear—namely, testability. Typemock literally robs the term “testability” of meaning in .Net design considerations. That's a freedom that should leave other developers gasping in envy.

Typemock's Magic

I don't want to be a running commercial for Typemock here (they're certainly not paying me to say any of this), but it is a truly revolutionary tool and its effect on development in .Net needs to be examined and understood. The thing is that using Typemock means that you can unit test literally any public method of any public class, regardless of any and all internal dependencies that class might have. And you can do so without changing the design and/or architecture of your software at all.

In other words, using Typemock means that everything is testable. Unit testable. Seriously. Everything.

Profound Effect

Stepping into a world where literally everything is testable is like stepping into a world where you wake up at a designated spawn point instead of dying—fundamental considerations about what is risky and what isn't need to be remapped. Which makes listening to .Net developers discuss testability considerations of various designs a little like listening to World of Warcraft players who are afraid to get too attached to characters that might die—it seems like a lot of effort for something that has little meaning.

Which is why I had such a hard time reading Jeffrey Palermo's latest blog post entitled “Inversion of Control is NOT about testability”. Since I know Jeffrey Palermo is a .Net developer, my initial response is a big fat “Duh. He must be using Typemock.” Sadly, this is not the case.

Freeeeeeedom!

That's too bad because there are some very interesting things in that post that get obscured by continually dragging in testability. For one, he gives a good explanation of Inversion of Control accompanied by a discussion of why you might want to use it outside of unit testing. His statements about coupling in particular piqued my interest. He said:

NOTE: There is no such thing as LOOSE COUPLING.  There is coupled, and not coupled.  Either a type is coupled to a type or it is not.  More accurately.  Loose coupling exists much like the concept of "cold".  When we say "close the door, you'll let the cold out" on a hot day.  Cold doesn't exist.  Either heat is there or it isn't.  We use cold to communicate, but it serves as a shortcut for the lack of heat.

Now, he got hold of a bad analogy (because there are degrees of heat—it's not a binary value—so there is nothing wrong with describing something as “less hot” just as we say "less coupled") and his point is technically wrong (because whether you call it loose, deferred, proxied, or something else, when you are coupled to an interface that a class implements, you have a relationship to that implementing class that is similar enough to coupling that an adjectival modifier is an acceptable descriptor) but the core of his point is still a good one. Tying your class to an interface is a profound change and it is probably a good idea to examine what “loose coupling” means (i.e. how “loose” modifies a coupled relationship).

He later said something I found profound:

In order to do anything useful in software, you must couple. You must couple to something. (emphasis his)

and he later concluded with

You must decide where coupling is appropriate and where it is not.

Importing Designs

Here's why it's a tragedy that Jeffrey didn't make use of his Typemock-granted freedom from testability concerns: that's all he said about coupling. He wasted so much unnecessary time with things that have no value or meaning in a .Net environment that he didn't better explore what coupling means and when to mask coupled relationships.

And here's my wider point: concerns about testability are imported from other environments. .Net folks discuss testability as if it has value in and of itself (because in other environments it does). We forget that testability is no more of a first-order good than quality is. “Testable design” only has value in the things it allows us to do—namely, unit test our classes. If we can unit test our classes as easily no matter what design patterns we choose, then that frees us up to explore other aspects of design choices. It isn't that <le design du jour> ceases to have value, it's just that testability is no longer a factor in evaluating its utility.

Those that realize this will have a competitive advantage over those who do not. If I can test no matter my project structure, then I can choose a design that more exactly fits other aspects of my development needs. Which means I can allocate resources more efficiently than someone who has design concerns that don't fit the actual environment. I can choose <Der Entwurf des Tages> when and where it is actually, immediately useful instead of doing so in order to achieve testability. And efficiency kicks butt in the marketplace.

So, uh, forget all I just said. Spend lots of time making sure your .Net projects are testable. Also: Typemock sucks. Don't bother going there...

You might say that I’ve had issues with Dependency Injection in the past. Well, with all the things I’ve learned not to do, I thought I’d go into a case where I’m considering dependency injection because it looks like it might be a good fit.

Project Background

Like many (most?) developers, I’ve been involved for many years in a particular vertical market. Like most, my involvement in that vertical market includes acquiring a substantial amount of very specific domain knowledge. I’m sometimes reluctant to admit it, but I spent a good decade and more of my career writing software designed for Multi-level Marketing companies. Specifically, I wrote programs that calculate the commission payments to their distributors. If you are familiar with that domain, my condolences. If you are considering entering that domain, talk to me (it’s not all bad).

Now, many accounting software packages deal with paying commissions, but compared to even the simplest MLM, their capabilities are pretty primitive. I won’t go into the full complexity of walking a tree with money on the line, but here’s what’s important for this post:

• The data you have to process is well contained, finite, and known in advance.
• Data concurrency tends not to be an issue (you read distributor information at the beginning and you write payout and promotion information as you go or at the end).
• Every company has their own, unique twist for qualifications, promotions, and payouts—indeed, commission plan differentiation is a key business differentiation.
• The only thing absolutely common to every commission plan is that you walk a tree of distributor nodes, processing orders at those nodes, and calculating payouts and/or promotions on them.

Most programmers working in that domain spend time designing their dream "bonus engine". One that can encapsulate the common elements so that their programming only has to deal with the payout and promotion logic. This turns out to be harder than you’d think to do right (if only due to the temptation to put more in the engine than really belongs there).

My Bonus Engine

Since I’m still getting occasional contract work programming commission payouts (and because I’ve seen a bonus engine or two and know what works and what doesn’t), I went and created an OSS bonus engine. Those of you dying to critique my code-fu to bring me up for well-deserved mockery, here’s your chance. So far I’m the only dev on the project, but that’s only because I don’t know anybody else masochistic enough to join me (hey, if you’re that nuts, let me know).

Right now, this engine is still pretty raw, but it works. Mostly. There are INode and IVolume interfaces for distributors and orders respectively. There’s a NodeBase object that implements INode with nifty events already implemented for easy inheritance and feedback and there’s a BonusManager object used to kick off a bonus run.

The tricky bit for this post is the data access. You see, when programming commission payouts, I seriously don’t care where the data comes from or where it is going. My clients all have their own tricky little data schemas and I’d just as soon not deal with the ins and outs of their systems. Abstracting that part out is beneficial all round.

In my initial design, I figured that something like that is a classic call for an Asp.Net-like provider model. This works well because the client can implement a provider (or have me create one for him) that we can drop in and we’re golden. That means that my eventual Distributor objects can be ignorant of all data implementation details and make a call to MLMBonus.DataProvider.BonusData.Promote() to notify of a promotion (for example).

Making the method references static seems like a good call because the distributor object has no other contextual information available. The engine is walking a tree and calling interface methods so the distributor and volume objects don’t really know much more about the outside world than their relationships to each other.

Because this is my first ever implementation of a "Provider Framework", I went digging for some sense of what kinds of things to look out for. While I initially went with an interface structure, I eventually decided to use Joel Ross' well written guideline for creating your own .Net providers.

There’s Always One

Unfortunately, I got some push-back when talking with my favorite client. You see, he’s chafing at the static nature of my provider model. He wants to make a grab for the holy grail of bonus commissions—executing mini-commission runs on-demand (most likely for partial distributor trees)—and the static provider just isn’t cutting it for him. mini-commission runs allow you to provide some interesting features to your distributors and that’s never a bad thing.

To support his functionality, he would need to be able to alter the data provider "on the fly" and/or be able to initiate multiple runs in the same application space without allowing them to step on each other. There’s just no way that static providers are going to work for that functionality. Naturally, I’ve been thinking about how best to work out meeting his need (mainly because I can see this feature will be something others will like but also because he’s my favorite client). Fortunately, my engine isn’t widely disseminated so it’s a good time for wholesale architectural alterations if we want to make them.

I’ve considered a number of ways to tackle the problem, playing with things like anonymous methods and context managers. They all turned ugly fast so it’s time that I face the fact that this functionality is pretty much tailor made for Dependency Injection.

Where To Inject

So where’s the best place to inject this little dependency? There are four options as far as I can tell.

• constructor injection
• property (or setter method) injection
• method parameter injection (since the number of methods affected will be very small).
• framework injection (Spring.Net or Castle Windsor for example)

Constructor and property injection are both object level references carried by the dependant object and have little to distinguish them in effect. Conceptually, I’ve heard that constructor injection is best for dependencies that are crucial to an object—something the object simply cannot do without. I think I agree with that guideline but I also like the convenience of being able to do grunt-work things in constructors, so if I were in charge of the objects, I’d include both a base constructor and one that includes the dependency as well as a setter method of some kind.

Method parameter injection is tempting, though. With method parameter injection, a client can set the dependency on BonusManager and rely on BonusManager to send the dependency to the Pay and Promote methods on INode. That way the reference to the data provider is short lived inside INode and IVolume objects. Thinking it over, though, I’m not sure why I’m so worried about carrying a bunch of references when garbage collection isn’t going to happen mid-run anyway. Yeah, there’s some overhead involved in tracking the reference on each INode object, but it’s negligible (are pointers in a 64 bit environment 64 bits long? They’d pretty much have to be right? I’ve never bothered to find that out—it’s never been significant before).

Framework injection simply isn’t going to be helpful in this case, I don’t think. For one, I’m not familiar enough with any DI Frameworks to make it worthwhile. For another, I don’t want to introduce a framework dependency on the project. And finally, I don’t want to commit future clients to a DI Framework that may be an issue in their environment.

Injecting

So my next task for the engine will be to muck around with the EngineInterfaces library. I’m thinking I’ll leave the provider object as-is (the BonusDataProvider abstract class) and simply add a property to INode to inject that sucker. In addition, I’ll update NodeBase to have an appropriate property and a new constructor that allows you to set the provider property on object creation.

Since I have the provider stuff already in place and no changes are needed to make this work, I’ll likely leave it alone. That way I won’t break anything currently working.

The only thing left is to decide if I want the IVolume dependency injected via a field or a method parameter on Pay. I’m pretty sure I don’t need the fine-grained control of allowing different orders for a single node to pay down a different path, but do I really want to commit to that? My arguments above against method parameter injection seem pretty solid for the IVolume object as well, but I haven’t completely convinced myself that they apply as strongly.

This post is from my new digs at The Runtime, cross-posted here for your edification. I’ll continue here for the foreseeable future, so no need to jigger your subscription if you don’t want to.

I thought I’d take a shot at introducing myself here as suggested by Jay and maybe dispel any pretense at being a thinker, heady or no. Frankly, it’s probably long past time that I put together some kind of background post about myself if only to give those who disagree with me a way to discount my arguments out of hand.

I suspect this’ll be long as I do go on sometimes. You can skip to a recent, similar post if you only want the heart of my current situation.

Street Cred

I’ve been playing with computers in one way or another since my fifth grade class acquired a then-new PET computer (I recognize the CBM Model 4032 from that link as the one we had). I think it had 8k RAM and we saved and loaded our programs using cassette tapes. I bought my own used Atari 800XL from hard-earned savings as a young teen and have owned a personal computer of some sort continually ever since (frankly, the Atari made it all the way to my Sophomore year of college when PC-envy finally became overwhelming).

It didn’t occur to me to take computers seriously as a career until after I graduated college, though. That’s right, I went through college working in the computer center and making money on the side setting up personal computers and local networks for small-companies and individuals without realizing that I could actually major in them. The one intro to programming class I took (which I remember was Pascal-based) was so easy (if only from familiarity and years of being a computer geek) that I treated it as a study hall.

After graduation, I discovered that my computer skills were worth more on the market than anything I’d actually studied so that’s what I did. This was the mid 1990s so mildly talented lighting fixtures could find programming jobs. Kind of like the mid 2000s, really.

Jacking a Career

I started my actual, salaried career working for a couple of vertical market software vendors who dominated their respective niches by using Pick (which had it all over those stuffy relational databases for the ability to map real-world information). It was with Pick Basic that I discovered the true meaning of Spaghetti Code. With great power comes great responsibility and with great responsibility comes the opportunity to make very large mistakes (none of them mine, of course).

Which means that Visual Basic 6 was an upgrade in sophistication for me. Fortunately, I learn fast—my single greatest career asset. When Visual Studio and the .Net framework came around, I was all over that in turn.

I started my own company with a very talented developer and friend in early 2000 serving the Multi-level Marketing vertical market providing independent compensation programming. If you’ve ever been in one of those presentations where they describe how you can earn money from the sales of people you recruit and they mention "levels" or "generations" you’ve pretty much seen the the design documents I was given. Ever wonder how those were programmed? I did, too. I’ll just say here that walking a tree with money on the line is an addictive form of adrenaline.

It’s a small and competitive niche, though, so it was only really viable while the market was booming. Also, it turns out that I had a lot to learn about marketing and sales.

Developer In Da House

Since then, I’ve been an in-house developer and/or development manager for various small to medium-sized companies. I’ve learned enough about how businesses operate to be a good interface to upper management (I can speak all the relevant dialects including accounting, sales, management, and operations) and I work in the trenches enough to not have completely lost the respect of actual developers.

My current position is with a small reading-glasses wholesaler where I am the development department. We’re relatively top-heavy in the IT department than you would think we need, but that’s because the company’s value-add in the marketplace is our ability to orchestrate complex vendor interactions (with up to 120 day lead-times) to provide a flexible product-line at a high-enough volume to satisfy our retailers.

I also contribute to a couple of open source and volunteer projects, the most notable being Subtext. I feel guilty about bringing that one up, though, because it’s been months since I’ve done anything helpful there.

My Hood

Since my in-house projects are small and self-contained, I tend not to use many formal development processes. My users are all within fifty feet of me and I’ve trained them not to hesitate in letting me know when something goes wrong or when they think of a better way to do something. If you’re doing things right in the first place, this isn’t really as disruptive as you’d think. My longest project in the last two years has taken less than a month to go from the start of development to deployment. Some of that is because I am just that good, but a lot of it is that we’re not doing anything that complex. It’s a sweet-spot, I admit.

All of which makes any formal development process so much overkill.

Flashing it Hardcore

I’ve been around long enough to have seen silver bullets come and go. It has gotten to the point where the only dogma that I’ll tolerate is anti-dogma. I am a Microsoft fanboi, but that doesn’t mean I don’t explore other technologies and ideas. I’m not into didacticism of any flavor.

I do have a core set of practices that I tend to use. They each have two criteria:

• I understand how they work.
• I’ve used them and verified that they work as expected.

There’s nothing like real-life to expose flawed base assumptions or principles.

As you can see, this tends to make me a process skeptic. I love iterative development because I’ve used it and seen that it works. I like Agile as an idea and implementation of iterative development, but I’ve never actually seen it in action so I try to keep my discussions of it on a theoretical level.

I’ve been known to create and maintain the occasional unit test, but I’ve found that even that can be overkill on many of my projects. Test Driven Development is interesting and all but I can’t really see myself using it. It’d be a pain to retrain my way of thinking and I’m not convinced that the payoff is there (and yeah, I’ve heard the testimonials). I’m not going to move to an experiment until I understand how the payoff is supposed to come. And with TDD, I just don’t see it.

This attitude is what keeps getting me in trouble with the Dependency Injection crowd. I’ve worked out a specific, relatively narrow set of circumstances where it can be useful, but as I found out when I actually tried to implement it, those circumstances turn out to be extremely rare in an already well-architected solution.

Busting Out

Skepticism aside, software development fascinates me and I truly cannot leave new stuff alone. I echoed a sentiment by Jay Kimble recently that says essentially that claims of universal applicability are always wrong. But there’s a flip side I want to acknowledge as well: every process, tool, or technology is useful somewhere.

What that means for me is that every popular tool or technique deserves a fair hearing and an attempt at understanding to see if there is something there that I can use. Rejecting something new merely because it is popular is no better than adopting it for the same reason. I admit that this is a weird blend of skepticism and optimism. That said it turns out that, while uncomfortable, it’s really useful for growth as a developer and maintaining the ability to jump into new situations and still find yourself in familiar territory.

Software development is fun and I can’t seem to be able to leave it alone. Fortunately, it’s a useful addiction—not unlike, say, air. And if you’ve stuck with me this far, you’re obviously interested in it yourself (or you’re my wife who reads every post if only because she knows I’ll eventually ask her what she thought of it). I hope you’ll stick around and join the conversation if so moved, even if only to tell me where you think that I am wrong.

A couple of months ago, I mentioned a project I had coming up that might benefit from using Dependency Injection. The use-case on this is simple enough. We receive all of our EDI text files to a specific directory. Those files need to be processed into standard, internal temporary tables. Since each of our vendors uses the EDIFACT X12 fields differently, we need to customize parsing the files according to vendor.

This is actually one of the larger projects I undertake at our small in-house development shop so it needs more up-front architecture than most and it seemed like a good time to test my understanding of Dependency Injection and maybe develop some familiarity with a DI framework. For one, I know in advance that I’m going to want an interface for my EDI processors and I know for certain sure that I’m going to add file processors in the future.

Those of you familiar with DI might be able to see the ripples around the reefs I eventually cracked up on from here.

The Basic Solution

The solution isn’t that complex in theory.

• I’ll want a service that monitors a given directory for new files with a given file mask. Both the directory and file mask are in a config file that reloads when altered (all handled by built-in .Net processes).
• I’ll want a simple interface that the service can share with implementing classes.
• I’ll want classes to process the files that are found.

See, simple.

Failure

As much as this looked like a DI shoo-in at the time, it turns out to be a bad fit. I figured this out after digging through the Castle Windsor documentation trying to find a way to iterate through all the registered objects that implement a specific interface. I was frustrated at not being able to find one. It took me a while to realize that I was trying to bend the framework to do something it hadn’t been intended to do—invoke methods on arbitrary registered objects. A DI framework is supposed to handle injecting a class that implements a given interface into the class that depends on that interface.

Indeed, DI frameworks in general are meant to abstract and centralize the complexities of implementation object selection. As such, they are built around the assumption that the run-time already knows what class it wants to use for a given interface (either in a configuration file or determined programmatically at run-time).

It took me a while to figure out that my use of the pattern itself is at fault in this mismatch. I’m not injecting a dependency at all. My service isn’t dependent on classes that implement IEDIProcess, it just determines if there are any available that apply to a newly created file and invoke it correctly if there are. In other words, I’ve already architected around the conceptual dependency here and Dependency Injection is an additional complexity with no purpose.

If DI should be used anywhere in this solution, it’s in the processes themselves to separate out the data updates. And, as I’ve mentioned before, there’s no payoff for me in worrying about my data dependencies.

Busted

So it turns out that shoe-horning a design pattern just to play with it didn’t work out too well. Who’d have thought that could happen?

The broader take-away, at least for me, is that you look for a solution/pattern/tool based on the attributes of your problem. In other words the problem comes first and the solution flows from it. Having a pattern looking for a problem to solve is like having a hammer looking for something to fix—both methods are hit or miss.

Reading the beginning of Joel’s second section of his talk at Yale clarified one reason I find myself so at odds with much of the hard-core Dependency Injection crowd (has Joel really achieved the level of fame that we can dispense with using his last name as Phil Haack suggests? Did you know who I meant right off?). Anyway, I am an in-house developer in a small company and that has a huge effect on my architectural decisions.

In-house Development

I described it a couple of months ago as simply "small company development", but Joel’s right that the more significant aspect of it is that it is in-house—technically, my company probably has close to the same number of employees as Joel’s Fog Creek does, but our development environments couldn’t be more different. Now, Joel describes a bleak picture of in-house development and to be honest, it’s mostly true (particularly on the consulting side). I’ve been fortunate enough that my full-time gigs have been very high quality (alleviating much of the down-side that Joel dislikes so much). His first point about not being able to explore new technologies and never being able to do things "the right way" hasn’t applied to me, for example.

Return on Investment

What he got right in a universal way, no matter the quality of your in-house programmer job, is this bit.

Once the core functionality is there, the main problem is solved, there is absolutely no return-on-investment, no business reason to make the software any better.

It is this feature of in-house development that elevates the status of YAGNI to near-absolute authority. With in-house development, you won’t win more customers with new bells and whistles, extra flexibility, or optimization of pretty much any kind.

Dancing While Juggling

It’s not just return on investment that hampers the in-house developer, though, it’s also the environment in a broader sense. While the environment is relatively stable (the next section explores how that plays out), it is also dominated by large external forces. Whatever market niche your company occupies, the chances are that there is a 500 lb. gorilla creating software for your business. Barring an actual niche vendor, business software like SAP, Dynamics, or Solomon are happy to come in and run your business. The 500 lb. gorilla’s software likely handles 80% of the software needs for your company. Unfortunately, it’s the extra 20% that provides the true market value for what your company does (good ole 80/20 rule. Where would we be without it?).

In-house developers find themselves dancing around this 500 lb. gorilla finding (or creating) integration points to make it do what the company needs. If the business software is good, you develop a love/hate relationship with it as you poke and prod looking for the openings you’ll need to do your job (if it’s bad, it’s a simple hate relationship—this can get bad enough that it becomes a good idea to look for change, either of software vendor or job).

Since management is generally clueless about computer stuff and couldn’t care less as long as they can do what they want, they’ll spend their time thinking up new things to do. Phrases like "wouldn’t it be cool if" are nearly as dangerous as "hey, ya’ll, watch this". These requests and changes come flying in with irregular frequency and require deft handling by the development manager (whatever their official title) if you want to have anything like a chance to get necessary changes implemented.

So you end up dancing around your business software vendor while management continually tosses new shiny objects your way. You need a specialized kind of Agile here if you’re going to function.

Programming for Change

Institutional ADD aside, the environment itself tends to be relatively stable and, best of all, predictable. This makes planning for the future relatively simple. Your degree of certainty about environment and usage is way higher than it would be in a computer software company. I know that there is literally zero chance that I’ll be asked to make my programs run against MySQL or Oracle, for example (about the same chance I have of needing to run on Linux or Mac).

This environment makes issues like Separation of Concerns and its twin offspring Inversion of Control and Dependency Injection much less... pressing. At the very least, it means that The Last Responsible Moment can come much later than it does in "true" software development companies.

A Shipping Example

I’ll use programming a shipping interface using vendor-provided online web services to show what I mean.

A Shipping Interface in a Software Company

Now, if I were at a software company looking to hit against a vendor-provided shipping web service, I can pretty much guarantee that whatever anybody says, the scope of this project is going to include multiple vendors and frequent changes. I would be best off planning early to keep the query mechanism for the service general and make sure that the internal and external expectations are separated. I’d probably want an IShippingService interface (at a minimum) and I’d probably better look at services from several different shipping companies to see what kind of commonalities and features I want to ensure are supported.

The extra time up front spent on architecting a generalized structure will pay off in spades both as the services themselves change and as new shipping vendors come and go with the flexing of the marketplace. The chance of change approaches certainty.

An In-house Shipping Interface

Now consider the same need in an in-house environment. The biggest obvious difference is that your requirements are likely to be as specific (and short) as "tie our shipping rates to the FedEx web service for pricing". Nine times in ten, your shipping department has a detailed contract with a specific shipper complete with daily pickup schedules and a computer in the warehouse that actually belongs to the shipper printing out labels and such like. The chances of using a different shipper, let alone multiple shippers, is remote (if you’ve never worked in wholesale, this homogeneity can be a surprise).

In this environment, spending a lot of time generalizing your solution up front is unlikely to ever pay off. Your best bet in this case is to isolate your shipping module in its own class library to encapsulate it somewhat and to allow you to reference it in multiple places if necessary. You can crib much of the data structure right from your shipping vendor without huge concerns that you are backing yourself into an untenable corner you’ll later regret.

Running Blind with Scissors

Now, could it happen that you need to make substantial changes to your in-house shipping module? Absolutely. And it may be that your shipping needs are as complex as it would be for a computer software company (particularly if you have to ship wholesale into Canada <shudder>). All that I am saying is that the Last Responsible Moment can come after the fact. The conclusion from my small company development post would probably be better if an in-house development modifier were added.

The best "lean" skill [an in-house] programmer can develop is to program with an eye on refactoring. Moving to something better when it becomes clear that it is needed isn’t a lot harder than if you had implemented it in the beginning (and has the added benefit that you know it is needed) if you have kept refactoring in mind throughout. Leave "quick and dirty" programming for the amateurs. Solid refactor-ready code is the best legacy you can create and one that doesn’t age as people get over themselves or discover some new Great New Thing™.

Being ready and willing to refactor ruthlessly is the best lean practice you can adopt as an in-house developer.

Environmental Architecture

I’ll call architectural choices that have a pervasive impact on a project "environmental". Things that change how or where you instantiate objects (acting as a replacement for "new" for example), or that require that all your classes are tied to interfaces, or that force isolation of your UI layer all have a global impact. Certain patterns and techniques penetrate deeply and fundamentally and as such, they need to be used with care and, more importantly, understanding.

The thing about environmental architecture is that it requires a familiarity with the specific technique or pattern in order to understand anything that is happening in the project. As such, its cost piles up in a small in-house development shop. Before too long, the only person who can make any changes to the code is the poor sap who put it there in the first place. Environmental architecture has inertia and weight that needs to be considered specifically and individually on a project by project basis.

Note that this has more to do with the project size and reach than it does with the architecture itself. If you’re doing a lot of small, well-isolated projects (as most of my development projects are), environmental architecture doesn’t have the time or scope to amortize the skill dependency cost let alone implementation and support costs involved.

It’s this dynamic that makes YAGNI king in small-company in-house development.

Dependency Injection Cross-talk

All of this came together for me while reading Joel’s Yale talk. Much of my discussion with DI advocates has these buried assumptions that our development needs and environments are substantially similar. I’m not sure this is the case. I deal mostly with small, discreet projects that take days (in rare cases weeks) to finish. My deployment can be as simple as a one-click "publish" from the developer’s machine.

Various fanboys and architect nazis want to dismiss me or make me feel inferior because I don’t adhere closely to their Great New Thing™. Whip me, beat me, call me "Mort", but the fact of the matter is that I don’t think it is wise to cost my company the time and effort let alone commit them to spending that time and effort in perpetuity just so that I can join the in crowd. The warnings of doom and gloom simply have not panned out in the reality I occupy.

I’ve been putting off a follow-up on Dependency Injection for a couple of months now. The amount of heat I anticipate receiving is so disproportional to the probable light gained that it makes me hesitate. This weekend, I picked up on a stream of referrals from a post at InfoQ that mentions my Dependency Injection post (though not the follow-ups). It does a reasonable job of spelling out the conversation that happened, though I was feeling picked on until it brought in Eli Lopian’s contribution to the discussion. In whole, it’s a good summary. The real pile-on happens in the comments and it illustrates so much of what I dislike in software development sloganeering that it has jolted me out of my reluctance to respond (I’d have responded there, but you have to register to comment and I hate that).

There are two things that contribute to the silver-bullet, band-wagon boosterism that sends up red flags for me.

Good for What Ails You

This is characterized best by a comment by Steven Devijver when he says

It’s an interesting discussion. Just as interesting as asking oneself: shall I wear underwear or shall I try something really nifty today?

It’s the same kind of thing pointed out by Jay Kimble about a month ago as he expressed his concern with Dependency Injection.

I have been asked "Why don’t you like this pattern or that pattern? I mean [Some famous blogger] says s/he always uses them.  I think they are smart or at least they seem to be.  You must be a 'Mort' or a pretender..."  At least that’s the way it has always been stated to me.

The claim made by these individuals is that The Pattern (it can be any pattern, but this is increasingly frequent when referring to Dependency Injection) is universally applicable and should be used in all cases, preferably by default. I’m sorry, but this line of argument only shows the inexperience or narrow focus of those making the claim.

Claims of universal applicability for any pattern or development principle are always wrong.

Study any pattern or development principle in enough depth and you’ll find the edge cases and counter-indicators applicable to it. This is true of even basic principles like data normalization and object oriented design.

Variations on the universal applicability include the pseudo cost-benefit comments like that from Gabriel Lozano-Moran.

If you anticipate [no] potential future problems using a DI framework like Spring.NET and it takes you a couple of minutes to implement this hardly violates the YAGNI principle.

I call this "pseudo cost-benefit" because it seriously under-values initial implementation, training, and the burden of long-term skill set dependency created by implementing so invasive a pattern—a point I made at the end of my post about small company line-of-business software development a bit back. Yeah, you can probably add Spring.Net to a given project in just a couple of minutes, but that treats Spring.Net familiarity as a sunk cost (or worse, null cost) and commits that project indelibly to Spring.Net familiarity. You may feel that familiarity with Spring.Net is a given for any competent developer, but that proves my point that this is merely a variation of treating DI as a given for all projects in all situations.

Save Me From Myself

This argument is made in a comment by Ole Friis.

When I use DI, I see it as an advantage that I am forced into splitting up my application in well-defined services with a nice interface.

The term "forced" here indicates that he doesn’t trust himself to split his applications into "well-defined services with a nice interface" on his own. In this, Ole Friis is echoing the response I got from the intimidating Oren Eini when I suggested that TypeMock makes arguments that couple DI with unit testing go away.

The main weakness of Type Mock is its power, it allow me to take shortcuts that I don’t want to take, I want to get a system with low coupling and high cohesion.

The implication is that he wouldn’t be developer enough to create a system with low coupling and high cohesion if he used powerful tools like TypeMock. Oren is inconsistent here because he’s otherwise all for developer empowerment as illustrated by his support for Jeffrey Palermo’s excellent post regarding treating developers as professionals.

The fact is that nobody is really in favor of limiting themselves. Taken seriously, these kinds of statements indicate a profound lack of trust in their own competence. I’ve never yet met a software developer above Jr. level with that particular insecurity. I’ve met a veritable host who lack trust in the competence of people they work with, however, which is what I suspect is really behind these statements. Either way, it’s hard to take that argument seriously.

I’m the Anti-DI

Or so I’ve been painted. The fact is that I’m not, though my use for it is pretty limited. Since I don’t need it as a foundation for mocking objects in unit testing, it’s only really useful when I have a service with more than one implementation and I’ve found that even then DI isn’t necessarily the answer. I’ll deal with at least two reasons behind this in a future post(s).

I’ve been musing about software architecture lately and trying to come up with a framework to help choose when to go with more as opposed to less—something that’ll help me feel less arbitrary in my choices. I mean, software design is something of a dark art, but how much of that is inherent and how much is simply being too lazy to formulate good internal guidelines?

My latest ruminations have revolved specifically around Inversion of Control in general and Dependency Injection in specific. Here’s the thing: for the development I do right now at a small reading glasses company, I’m reluctant to implement any of the stronger separation of concerns architectural tools like IoC and a DI framework. The DI zealots would draw from that the conclusion that I am a sub-standard designer and a compromised developer—at least, judging from explicit claims that every project should start out with formal DI (please note both the "every" and the "should").

Pain is an Efficient Teacher

Perhaps too efficient. Most developers have had the experience of working on a project where decisions made in the expedience of the moment came back to hurt them. In a recent DotNetRocks episode, Scott Cate gives an example of hard-coding to a specific search engine and coming to find that they need to generalize in order to allow alternatives. He wishes that they had used a DI framework from the start (in this case, Castle Windsor) to make that an easy change. Scott tells about how they were lucky this need was discovered in time for a major release upgrade so they could take the time to go the full DI route.

There’s two problems with this example when translating to my own situation. First, Scott’s is a commercial product that they sell to large clients and thus the scope is far beyond anything I work on here. Second, I wonder how much pain was really involved in making the change.

You see, pain magnifies itself in memory. You consider all the might-have-beens and how much the problem could penetrate even as you work to ameliorate it. And you consider things you can do in the future to avoid this pain—both the immediate pain and the imagined pain of all those might-have-beens.

The YAGNI Nazi

This tendency to artificially inflate potential scope is the driving force behind YAGNI. YAGNI is the reminder that you can go too far in countering the mere possibility of future pain and that you can plan too far in advance of actual need. I’m a huge fan of YAGNI simply because it has allowed me to realize significant time savings and produce useful software for clients quickly and efficiently.

In many software design discussions, however, I feel like a lone YAGNI voice in an over-designed wilderness. I’m accused of negligence or stupidity (though typically through implication as opposed to outright accusation) because I look for the justification of larger design patterns before blindly implementing them. People make the unearned assumption that understanding a given principle must mean that I would adopt it. This comes through loud and clear in the discussions with DI proponents. It seems entirely foreign to them that I could both understand it and decide to give it a skip on some of my projects.

Sometimes this judgement comes in the form of a logic chain. This happens when designers use the strengths of one principle to piggy-back in a second. Unit testing is most often the piggy-back victim here. That’s because the benefits of unit testing are well-understood and apparent in projects of even minimal scope. Scott Cate admits in that DotNetRocks episode, for example, that a large part of his current push for the Model View Presenter pattern as well as DI is to facilitate unit testing (when he also took pains to point out that implementing Castle Windsor is not without cost he went on my list of DI heroes).

The problem with a logic chain, however, is that you’re right back where you started if a link breaks. What if I can achieve strong unit testing without IoC and DI?

So here is my YAGNI wisdom (for what it is worth): you don’t protect yourself from future change by throwing every possible technique or pattern at every project. You protect yourself from future change by refactoring ruthlessly to keep your code as change-ready as possible.

The Bottom Line on Dependency Injection

So here’s where I think I’ll come down as a rule on implementing DI: I’ll use it as soon as I actually need to support multiple services in a given object. Since I don’t need DI in order to mock dependencies in unit testing (yay TypeMock), this means that I’ll look to implement DI when I have to support multiple services in a given object in production. I have two examples of related projects here that will serve as useful test cases.

The first is a need to integrate customer orders into Dynamics GP. We have many customers who send us orders in a variety of ways and with a variety of processes for approving and tweaking them before we let them hit our system. It’d be nice to have a single module integrated into Dynamics GP that can accept orders in the variety of formats we end up with and go to town. I’ve already got this project up and running for a single customer using a generalized interface and I think DI can help extend that further faster than I can without it.

The second is the initial processing when we receive EDI documents from our clients. Since EDI X12 isn’t really a standard so much as a soup of possible ways to do things, each customer tends to put their own unique spin on what fields are significant and how they should be interpreted. I’ve hated our current EDI software for months (and not just because it uses FoxPro at its base) and I’d dearly love to pull that processing into an internal framework I can use and extend on my own.

Both projects will take me a while, which is appropriate because I’ve been saying all along that DI seems like something suited to larger-scoped projects. I’ll post my experiences as I go, assuming I actually get the time to tackle either one.

Both of these examples are also an inversion of typical DI examples because in both I technically have a service that needs to act on multiple objects. That said, I’ve been an architect long enough to understand that while services and objects might differ conceptually (though defining either is an exercise in nit-pickery), it’s trivially easy to turn each into the other (if you bother differentiating them at all—see nit-pickery above). Sometimes it is quite useful to invert them as I plan to do here.

I typically work with small companies who need to customize software to fit their business practices. A lot of companies have critical competitive advantages embedded in the way that they do things and need to ensure that their software doesn’t get in their way. That typically means that I deal with specific vertical markets (either at the vendor or client level) and dance with 500lb. gorillas to make things work the way companies expect them to. It’s business programming in the trenches and can be nasty, brutish, and, well, not short so much as constrained.

The Typical Battlefield

This is the underbelly of software development. Because the companies are small, being hired is a matter of impressing a very small group of non-technical people—often a single person such as the CEO or President. I’ll sometimes have a full development team, but occasionally I get to do it all myself. That degree of variation means that my methods and opinions have to a) scale well and b) not suck (because I’ll often be implementing them myself).

Because being paid depends on non-technical people evaluating technical work, a lot of really bad things can happen down here. This is where the president’s fresh-out-of-college nephew gets to prove he didn’t waste four years of tuition. Or where that new English degree and some aptitude can turn into a technical career. It’s also a place where developers whose self-confidence exceeds their actual ability can avoid public accountability. For a while.

What that means in practical terms is that I work with small companies who have extraordinary needs and the means and motivation to meet them. It also means that I see a lot of train wrecks (some admittedly of my own making). I’ve found that most of these train wrecks have their origins in one of two fundamental problems.

Over Confidence

You know those eager puppies that get so excited with each person who stops to pet them that they wet themselves? I sometimes see developers who are like this each time they discover a new process, technology, or design pattern.

I don’t fault the underlying impulse, really. Excitement with new technology is typical in IT people for the same reason that air is a common addiction. People who hate change tend to be filtered out of IT pretty quickly. To extend the initial puppy analogy, most developers learn to get by with simple tail-wagging when they find a previously unknown technology that solves a common problem.

What leads to train wrecks are those developers who take off in the midst of their excitement and begin implementing new technologies or techniques without bothering to understand them thoroughly. Like the developer I mentioned in a previous post that implemented n-tier design in each project separately, they may have a vague idea of how the thing is done but they end up with something that is worse than if they’d never heard of the Great New Thing™ in the first place.

If you suspect that you are this kind of developer, stop. Take a breath. Dig deeper. Don’t proceed until you can identify at least two drawbacks and can explain why the Great New Thing™ was invented in the first place. If this is a developer on your team, I’ve heard you can get good results with rolled-up newspapers and consistent, immediate feedback.

A Hammer Finding Nails

I have a daughter afflicted with artism. To her, every surface is an opportunity to improve with color and/or texture. You’ll find developers and architects like this as well—treating each project as an opportunity to improve with their favorite pattern, practice, or technology. This is the opposite of the previous problem because these individuals know their stuff, know it well, and have likely used it successfully before. Indeed, the more knowledgeable the designer, the more prone they are to this particular vice.

The problem here isn’t a lack of detailed understanding of the Great New Thing™, but rather lack of focus on the real needs of the project. Here’s the simple truth: in the world of small business development, a short project delivered quickly is usually better than a perfectly-crafted but more complicated project delivered even a little bit later. In short, It’s a YAGNI world.

I have to be careful here because I’m not saying that any given design principle or practice is a bad idea. What I’m saying is that it is really easy to over-architect these projects. The company has so many needs and can improve in so many ways that it is tempting to go into full framework-building mode and spend months putting something comprehensive together when all that was needed is a basic customization or a specific process.

In some places, I’ve had to pull the YAGNI flag out so often that I’ve considered having a stamp made to save time. Maybe I should make a suggestion to Think Geek.

Keep it Lean

Whether coming from a background of large development or yearning to do so, it is really easy to include things just because they are good design or a "best practice" without bothering to review their actual impact on the current project. The problem is that each pattern, technique, and tool not only commits the project itself forever, it also sets up a barrier for future work by creating a developer familiarity dependency that can be a significant hurdle for later modifications (particularly if a developer only thinks they are familiar with the pattern, technique, or tool).

Each technique, pattern, technology, and design principle should be evaluated in light of immediate need and weighed against future scenarios that include the possibility that you wont be the one doing the work. Indeed, I recommend a slight bias in favor of not implementing a Great New Thing™ because of the tendency of really good developers and architects towards heavy designs that can end up hurting small companies in hundreds of little ways that add up to death by a thousand cuts.

The best "lean" skill a programmer can develop is to program with an eye on refactoring. Moving to something better when it becomes clear that it is needed isn’t a lot harder than if you had implemented it in the beginning (and has the added benefit that you know it is needed) if you have kept refactoring in mind throughout. Leave "quick and dirty" programming for the amateurs. Solid refactor-ready code is the best legacy you can create and one that doesn’t age as people get over themselves or discover some new Great New Thing™.