And I'm okay. Indeed, last week saw a payoff for one of those hygiene things you do because you know that you “should”.
A Logging Story
Allow me to share what happened (feel free to skip this section).
We receive orders from a number of different sources. In addition to EDI, we have spreadsheets, flat-files, and we even originate a few ourselves under Vendor Managed Inventory (VMI) agreements. One of the things we've done recently is create a single “process pipeline” such that once orders enter the pipeline, their processing from that point forward is uniform. Thus, a new client can send us whatever they want to (or can), and all we have to do is get the order details into a set of order import tables and away they go. i.e. we create a single customer-specific import routine to start their orders off instead of an entire order-processing process (yes, that was the previous solution when adding new customers—don't ask, it wasn't me).
So yeah. One entry point into the pipeline (it isn't fancy enough for the appellation “workflow”) is a set of order import tables. A simple console application is called by the import routine that created the entries. All it does is take the orders identified and create them in Dynamics GP. It is generally called by nightly import processes that run unattended.
This routine has been running for months now and frankly, I don't think about it much. That is, until last Thursday when I received an email with the subject “ImportOrders Log: Error”.
The cause of the error isn't really important (one of our customer records hadn't been updated with the new shipping method and GP Web Services choked). In a batch of 146 orders, the last 5 were skipped. It took me fifteen minutes to determine what had happened and why and another fifteen for our data guy to put together a fix—all before the rest of that order had been processed for the day.
Our internal order processors would never have caught this error so logging and notification saved us a couple of angry phone calls and pissed off store managers (and hence account managers and executives).
Chainsaws
So here's the rule:
Any process that will run unattended must have robust logging that will not only proactively notify you of any errors, but that also allows you to rebuild the circumstances and conditions surrounding errors encountered.
There are a number of good logging toolsets out there. I highly recommend finding and using one rather than “rolling your own”. A logging toolset will give you more flexibility and stability than you could give yourself and frankly, who needs the headache of slaying a dragon that has already been taken out so very thoroughly?
For this project I used the Enterprise Library Logging application block. Subtext uses Apache's log4net. I'm willing to bet that there are others of similar quality. Key features include being able to refine (through configuration) logging levels and logging details like where the log is kept and what form it will take.
Deforestation
As with any good general rule, there is a real temptation for over-application. A little logging saved a ton of aggravation so imagine the benefit I'd get if I logged everything! Yeah, well, don't forget your friend YAGNI when determining if and what to log. Pile up enough of these things and before you know it every little project can be so encrusted with “must haves” that you end up with huge unwieldy project frameworks.
Not that there is anything wrong with a good framework for enterprisey applications. But that's not what I had here. All I have is a simple console app that runs when needed to import orders into Dynamics GP. All the potentially enterprisey stuff happens later. I needed logging that kept track of events and notified me of errors. I didn't need much more than that.