Many of the interesting .Net bloggers are part of the Alt.Net crowd; evangelizing Dependency Injection, Design for Testability, Test Driven Development, SOLID design and other development practices that they find useful in their work. It doesn't take long reading these blogs to pick up on what looks like an unforgiving attitude towards those who don't use the latest acronyms in their software development.

This acrimony is unfortunate because most often what is at the heart of those who question the standard Alt.Net toolset isn't so much principle as it is context.

A Fundamental Assumption

Unfortunately, discussing software development at all imposes an initial assumption that too often goes unexamined—the assumption that the software development you are doing is substantially similar. I'm afraid that this assumption is violated more often than we realize.

Most prominent development bloggers are consultants, conference speakers, tool vendors, or framework providers of one sort or another. As such, they are used to considering the problems of very large applications with many cross-cutting concerns. Is it any wonder, then, that they are strident about the importance of segregation and testing? For the majority of their hands-on development, everything that increases separation and independence is a benefit because isolating fail points and mitigating the cost of touching individual elements carries a heavy multiplying effect on overall complexity. No wonder they want to use a DI framework on every project and insist that other developers practice TDD. It's all up-side in such projects because the costs disappear in the noise and the benefits stand out.

Not all of us work in those kinds of environments and/or applications, however. Many of us work on projects that are wildly dissimilar. Our applications are narrowly focused, single use, and with limited distribution. For us, implementing solution-wide DI eats up half the project configuration overhead and delivers functionally zero chance of eventual benefit.

I'm unsure how many find themselves in similar (smaller) development shops. I suspect quite a few. We might even be a majority of actual, working developers. We're pretty much absent from the development blogger population, however.

A Problem

What that means for us small-shop developers is that we end up having to do a lot of on-the-fly translation. Up-scale bloggers have shown themselves averse to “thinking down” so we end up having to analyze the methods and techniques they discuss to see where the fail points are for our projects. If your QA team is the folks in the warehouse and your architecture a straight-on table insert, is TDD really going to pay off? What about bug tracking? Or continuous integration?

These translations are interesting and part of why I like smaller development. Our problem space is largely undiscovered country. We can't really trust anything labeled “best practice” and we have to do our own experiments to find what works and what doesn't and at what point it starts to make sense to engage different methodologies.

It'd be nice, though, if some of the gee-whiz bloggers would discuss the fail points of their methods and tools. Or point out the trade offs that various techniques are likely to entail. No technique or method is so useful that it is universally applicable, so where are the borders?

Worse, though, are those who actively stifle discussion of such fail points. Mention that you'd rather not use Dependency Injection, for example, and you can find your professionalism and/or intelligence questioned. Indeed, for many the very fact that you don't <insert favorite practice here> indicates that you must never have tried it. Or if you have you must have been doing it wrong.

This friction fosters resentment and shuts down discussion. Perhaps one reason we don't see more developers willing to discuss alternatives to the popular band wagons of today is a reluctance to enter what amounts to a hostile arena. As the troubles on Wall Street filter down to Main Street (where us small-shop developers live) the last thing we need is other developers calling us unprofessional in public. Particularly when those other developers are well-known and respected in their community.

I know this from personal experience. I'm pretty secure in my current position. My company is healthy and has a marked competitive advantage that I am key to delivering. My boss respects me as do the owners of the company. Even so, my gut clenched tight and my heart sank to my knees when I came across a twitter message from a luminary in our field (someone any informed .Net developer would recognize) dismissing me as unprofessional and my points as unworthy of consideration.

Seeing something like that will make you wonder if expressing your doubts is worth the risk, even when it isn't directed at you.

And I'm okay. Indeed, last week saw a payoff for one of those hygiene things you do because you know that you “should”.

A Logging Story

Allow me to share what happened (feel free to skip this section).

We receive orders from a number of different sources. In addition to EDI, we have spreadsheets, flat-files, and we even originate a few ourselves under Vendor Managed Inventory (VMI) agreements. One of the things we've done recently is create a single “process pipeline” such that once orders enter the pipeline, their processing from that point forward is uniform. Thus, a new client can send us whatever they want to (or can), and all we have to do is get the order details into a set of order import tables and away they go. i.e. we create a single customer-specific import routine to start their orders off instead of an entire order-processing process (yes, that was the previous solution when adding new customers—don't ask, it wasn't me).

So yeah. One entry point into the pipeline (it isn't fancy enough for the appellation “workflow”) is a set of order import tables. A simple console application is called by the import routine that created the entries. All it does is take the orders identified and create them in Dynamics GP. It is generally called by nightly import processes that run unattended.

This routine has been running for months now and frankly, I don't think about it much. That is, until last Thursday when I received an email with the subject “ImportOrders Log: Error”.

The cause of the error isn't really important (one of our customer records hadn't been updated with the new shipping method and GP Web Services choked). In a batch of 146 orders, the last 5 were skipped. It took me fifteen minutes to determine what had happened and why and another fifteen for our data guy to put together a fix—all before the rest of that order had been processed for the day.

Our internal order processors would never have caught this error so logging and notification saved us a couple of angry phone calls and pissed off store managers (and hence account managers and executives).

Chainsaws

So here's the rule:

Any process that will run unattended must have robust logging that will not only proactively notify you of any errors, but that also allows you to rebuild the circumstances and conditions surrounding errors encountered.

There are a number of good logging toolsets out there. I highly recommend finding and using one rather than “rolling your own”. A logging toolset will give you more flexibility and stability than you could give yourself and frankly, who needs the headache of slaying a dragon that has already been taken out so very thoroughly?

For this project I used the Enterprise Library Logging application block. Subtext uses Apache's log4net. I'm willing to bet that there are others of similar quality. Key features include being able to refine (through configuration) logging levels and logging details like where the log is kept and what form it will take.

Deforestation

As with any good general rule, there is a real temptation for over-application. A little logging saved a ton of aggravation so imagine the benefit I'd get if I logged everything! Yeah, well, don't forget your friend YAGNI when determining if and what to log. Pile up enough of these things and before you know it every little project can be so encrusted with “must haves” that you end up with huge unwieldy project frameworks.

Not that there is anything wrong with a good framework for enterprisey applications. But that's not what I had here. All I have is a simple console app that runs when needed to import orders into Dynamics GP. All the potentially enterprisey stuff happens later. I needed logging that kept track of events and notified me of errors. I didn't need much more than that.

Your environment can have a profound effect on how you develop software. The details of what I discuss here have zero practical meaning outside of the .Net world (though you can probably find parallels in other environments). That's because .Net developers have access to tools that invalidate rules of software design that are fundamentally important elsewhere (before you question whether an environment can effect what is good design, consider the difference between good design in C and, say, Prolog). For .Net, the free availability of a tool like Typemock makes a major design consideration simply disappear—namely, testability. Typemock literally robs the term “testability” of meaning in .Net design considerations. That's a freedom that should leave other developers gasping in envy.

Typemock's Magic

I don't want to be a running commercial for Typemock here (they're certainly not paying me to say any of this), but it is a truly revolutionary tool and its effect on development in .Net needs to be examined and understood. The thing is that using Typemock means that you can unit test literally any public method of any public class, regardless of any and all internal dependencies that class might have. And you can do so without changing the design and/or architecture of your software at all.

In other words, using Typemock means that everything is testable. Unit testable. Seriously. Everything.

Profound Effect

Stepping into a world where literally everything is testable is like stepping into a world where you wake up at a designated spawn point instead of dying—fundamental considerations about what is risky and what isn't need to be remapped. Which makes listening to .Net developers discuss testability considerations of various designs a little like listening to World of Warcraft players who are afraid to get too attached to characters that might die—it seems like a lot of effort for something that has little meaning.

Which is why I had such a hard time reading Jeffrey Palermo's latest blog post entitled “Inversion of Control is NOT about testability”. Since I know Jeffrey Palermo is a .Net developer, my initial response is a big fat “Duh. He must be using Typemock.” Sadly, this is not the case.

Freeeeeeedom!

That's too bad because there are some very interesting things in that post that get obscured by continually dragging in testability. For one, he gives a good explanation of Inversion of Control accompanied by a discussion of why you might want to use it outside of unit testing. His statements about coupling in particular piqued my interest. He said:

NOTE: There is no such thing as LOOSE COUPLING.  There is coupled, and not coupled.  Either a type is coupled to a type or it is not.  More accurately.  Loose coupling exists much like the concept of "cold".  When we say "close the door, you'll let the cold out" on a hot day.  Cold doesn't exist.  Either heat is there or it isn't.  We use cold to communicate, but it serves as a shortcut for the lack of heat.

Now, he got hold of a bad analogy (because there are degrees of heat—it's not a binary value—so there is nothing wrong with describing something as “less hot” just as we say "less coupled") and his point is technically wrong (because whether you call it loose, deferred, proxied, or something else, when you are coupled to an interface that a class implements, you have a relationship to that implementing class that is similar enough to coupling that an adjectival modifier is an acceptable descriptor) but the core of his point is still a good one. Tying your class to an interface is a profound change and it is probably a good idea to examine what “loose coupling” means (i.e. how “loose” modifies a coupled relationship).

He later said something I found profound:

In order to do anything useful in software, you must couple. You must couple to something. (emphasis his)

and he later concluded with

You must decide where coupling is appropriate and where it is not.

Importing Designs

Here's why it's a tragedy that Jeffrey didn't make use of his Typemock-granted freedom from testability concerns: that's all he said about coupling. He wasted so much unnecessary time with things that have no value or meaning in a .Net environment that he didn't better explore what coupling means and when to mask coupled relationships.

And here's my wider point: concerns about testability are imported from other environments. .Net folks discuss testability as if it has value in and of itself (because in other environments it does). We forget that testability is no more of a first-order good than quality is. “Testable design” only has value in the things it allows us to do—namely, unit test our classes. If we can unit test our classes as easily no matter what design patterns we choose, then that frees us up to explore other aspects of design choices. It isn't that <le design du jour> ceases to have value, it's just that testability is no longer a factor in evaluating its utility.

Those that realize this will have a competitive advantage over those who do not. If I can test no matter my project structure, then I can choose a design that more exactly fits other aspects of my development needs. Which means I can allocate resources more efficiently than someone who has design concerns that don't fit the actual environment. I can choose <Der Entwurf des Tages> when and where it is actually, immediately useful instead of doing so in order to achieve testability. And efficiency kicks butt in the marketplace.

So, uh, forget all I just said. Spend lots of time making sure your .Net projects are testable. Also: Typemock sucks. Don't bother going there...

Though work and family keep me pretty busy, everyone needs a hobby. Mine is online gaming. My current addiction of choice is City of Heroes where my favorite class to play is the Scrapper. Scrappers engage in close-range fighting with the enemy and are good at dishing out damage, which means they tend to go after the biggest bad-guys they can find in a group leaving others on the team to manage the lesser minions. Since this puts the scrapper in the middle of chaos with a limited scope of influence, scrappers often acquire what’s called "scrapper-lock"—they lose awareness of external contexts and miss important developments (like that their own health is in the red and it’d be a good time to pop a health inspiration).

This came to mind recently during a podcast on Dot Net Rocks. In this episode Carl and Richard talk with Udi Dahan about SOA. For all that Udi tends to discuss issues way above my scale enterprise-wise, I enjoy hearing him flog his hobby horse. He has the ability to make complex concepts understandable by using solid examples that relate well to his points.

Near the end of this episode (around 57:08), Carl asks "If there’s one big mistake that people tend to make, what is it?" Udi answers with our tendency to view challenges through the lens of our current obsessions and assumptions. His response reminded me strongly of scrapper-lock. It is really easy to look for solutions in terms of our current abilities and mind-sets. It is really difficult to take a step back and make sure you are taking on a problem in its entire context.

Stay on Target

There are two important factors to achieving IT goals. The first is identifying the target. This is actually the harder of the two factors because we bring so many unacknowledged assumptions to the task. The first assumption that messes us up is that we work for IT. We don’t. We work for companies, universities, governments, non-profit organizations, ad-hoc teams, and ourselves (and a multitude of other entities too varied to name). I tend to concentrate on corporate development so I’ll talk about working for a company below, but you can substitute any of the above just as easily.

In considering a goal, it is important to have not just a complete understanding of the goal itself, but of the reasons behind that goal. This is what makes software development such a challenge, by the way. You could train pretty much anyone to write the actual code. It’s learning to map the code to the needs of your company that causes real strain.

Take, for example, the requirement that you "hit a target roughly six inches in diameter from a distance of 150 feet." Which target did you see?

In this situation, you can ask all kinds of clarifying questions about the target (what color, outdoors or indoors, how much force etc) but the one question that is going to provide the best guidance is "why?"

Understanding the purpose gives you a load of related context that allows you to skip a lot of stupidity and identify buried assumptions. The reason this is so difficult is that in order to understand the purpose, you often have to understand the business (domain, university, non-profit et al) and most IT people resist doing so.

Gearing Up

Once you understand the target, deciding how you are going to reach it offers its own batch of problematic assumptions. Too often, we let our pet projects or obsessions intrude in this process. Are you creating a web interface because it’s cool and what you know or because it’s the best way to meet your business need? Are you implementing a new enterprise framework because the company needs it or because you really want to try out the whole SOA thing?

This is where integrity becomes important because you’ll often find that the best way to go is less interesting than the one you’d actually prefer. Every project requires trade-offs and those can leave you lots of room to nudge things in directions that might offer a slight advantage, but might also just scratch a personal itch to do things the "right" (i.e. your) way.

As the subject matter experts in our organizations, we have a lot of trust. It is easy to abuse that trust unintentionally by allowing ourselves to give too great a weight to our own interests (it’s also easy to abuse this trust intentionally, but that’s another post entirely). This tendency drives many of the fads you see sweeping through software development now and then (Extreme Programming anyone? Some good things are there, sure, but not nearly to the extent many of us convinced ourselves existed).

Acquiring a Strategic Overview

Because lock-in is so easy to fall into, it’s important that we develop some habits that overcome it. Here are some of my ideas, but please add or modify them in the comments if I’ve missed something useful.

• Develop some key general principles. It’s important that these be both "general" and "principles"—i.e. fundamental ideas that are valid regardless of programming environment or language. YAGNI is a good example of a general principle and one of my core tenets.
• At the beginning of any new project, make sure you ask the most important question: why? Further, make sure you understand the answer. It also helps if you take steps to understand your domain. If you program for businesses, for example, introductory courses in accounting and management are extremely useful. I use information from the one college accounting class I took way more often than information from my one college computer science class.
• Periodically review your tools. Is there a new kid in the Unit Testing world that might have some sweet capabilities (and an easy conversion utility)?  Has someone slain the dependency injection demon for Mock objects?
• Periodically review alternative environments. This is the hardest for me personally because I’m rather attached to MS Visual Studio and the .Net framework. Why should I bother with REST or Ruby or Eclipse? The important thing to learn here is perspective and the important perspective is not your own but that of the company (or whatever) that you work for. However comfortable you may be with Asp.Net, for example, it may be that your company can realize a substantial benefit by adopting Ruby. I’ll never forget the roadblock one developer on my team overcame in a day using python that I was afraid would set us back a week.
• Finally, make sure you are aware of and take the time to understand the current crop of buzzwords. Agile, SOA, TDD, Grid Computing, all of these have value or people wouldn’t be buzzing about them. It’s easy to become jaded after you’ve applied your third silver bullet and the slavering monster is still healthy and looking your way. Declaring all silver bullets useless and hibernating isn’t going to help you. When management comes to you with a new solution they read about in their trade magazine, you’ll get a lot farther if you can speak knowledgeably about its strengths and applicability in your environment. And who knows, you may just find something that slays one of your own dragons while you’re at it.