And I'm okay. Indeed, last week saw a payoff for one of those hygiene things you do because you know that you “should”.

A Logging Story

Allow me to share what happened (feel free to skip this section).

We receive orders from a number of different sources. In addition to EDI, we have spreadsheets, flat-files, and we even originate a few ourselves under Vendor Managed Inventory (VMI) agreements. One of the things we've done recently is create a single “process pipeline” such that once orders enter the pipeline, their processing from that point forward is uniform. Thus, a new client can send us whatever they want to (or can), and all we have to do is get the order details into a set of order import tables and away they go. i.e. we create a single customer-specific import routine to start their orders off instead of an entire order-processing process (yes, that was the previous solution when adding new customers—don't ask, it wasn't me).

So yeah. One entry point into the pipeline (it isn't fancy enough for the appellation “workflow”) is a set of order import tables. A simple console application is called by the import routine that created the entries. All it does is take the orders identified and create them in Dynamics GP. It is generally called by nightly import processes that run unattended.

This routine has been running for months now and frankly, I don't think about it much. That is, until last Thursday when I received an email with the subject “ImportOrders Log: Error”.

The cause of the error isn't really important (one of our customer records hadn't been updated with the new shipping method and GP Web Services choked). In a batch of 146 orders, the last 5 were skipped. It took me fifteen minutes to determine what had happened and why and another fifteen for our data guy to put together a fix—all before the rest of that order had been processed for the day.

Our internal order processors would never have caught this error so logging and notification saved us a couple of angry phone calls and pissed off store managers (and hence account managers and executives).

Chainsaws

So here's the rule:

Any process that will run unattended must have robust logging that will not only proactively notify you of any errors, but that also allows you to rebuild the circumstances and conditions surrounding errors encountered.

There are a number of good logging toolsets out there. I highly recommend finding and using one rather than “rolling your own”. A logging toolset will give you more flexibility and stability than you could give yourself and frankly, who needs the headache of slaying a dragon that has already been taken out so very thoroughly?

For this project I used the Enterprise Library Logging application block. Subtext uses Apache's log4net. I'm willing to bet that there are others of similar quality. Key features include being able to refine (through configuration) logging levels and logging details like where the log is kept and what form it will take.

Deforestation

As with any good general rule, there is a real temptation for over-application. A little logging saved a ton of aggravation so imagine the benefit I'd get if I logged everything! Yeah, well, don't forget your friend YAGNI when determining if and what to log. Pile up enough of these things and before you know it every little project can be so encrusted with “must haves” that you end up with huge unwieldy project frameworks.

Not that there is anything wrong with a good framework for enterprisey applications. But that's not what I had here. All I have is a simple console app that runs when needed to import orders into Dynamics GP. All the potentially enterprisey stuff happens later. I needed logging that kept track of events and notified me of errors. I didn't need much more than that.

I was thinking today about something I learned the hard way and how I wish someone had taught me what to look for earlier in my career. Then it occurred to me that perhaps I could perform that service for others.

He Hates Me!

Have you ever found yourself in a situation where someone important where you work doesn’t seem to like you very much? I have. In fact, at my first real programming job (programming in Pick of all things) my manager’s boss obviously had a hard time with me. I would hear from my manager that her boss was concerned that I wasn’t "cutting it" and that I should make sure I highlight my accomplishments when we interacted. And there were definitely a couple of times when he cut me pretty short shrift.

Since then, I’ve been in other situations where someone important seems to have prejudged me in some way that is, at the least, inaccurate, and at worst potentially career limiting.

That experience left me insecure enough that I went looking for a new job.  I knew I was better than the raise I wasn’t getting and programmers were very scarce then. I found a position that paid better and where I had more responsibility (though still programming in Pick). This turned out to be a good move because it let me leave on good terms with the company I was at.

Going Detective

Since there was such a pronounced disconnect between what I considered reality and the perception of others at the company, I decided to dig a little. After a couple of months, I approached some of the insiders there. I asked them what their perception of my work had been, if or when it had changed, and what they thought of me leaving as I did. This turned out to be an instructive exercise.

Long story short, it seems that my manager, the one that I thought was on my side and happy with my work, had been denigrating me in meetings with her superiors. A project couldn’t be completed in the desired timeframe, for example, not because the project was complex but because I couldn’t handle the requirements.

Here’s the Scoop

Since this happened, I’ve seen subtler indications of the same pattern. The thing I eventually came to realize is that work relationships, unlike pretty much any other, start off in the black. When someone hires you for a job, they do so with the conviction that you can do the work required. Any manager or company grand poobah that you interact with will be prejudiced in your favor initially as a result. The instinct is to validate the hire decision.

Relationships with company managers start off without all the typical prejudices we deal with in more social circles—not because managers are nicer, fairer people but because they worked those prejudices out before hiring you (or trusting others to hire you for them). Indeed, your immediate supervisor’s attitude should be the default attitude towards you for all company higher ups.

Which is why, when someone you don’t work with treats you differently than your immediate supervisor does, it should send up a major red flag. They are almost certainly getting feedback that you aren’t privy to. Finding out what that feedback is and where it is coming from could be vital to your career. Expending a little effort and/or political capital can head off major pain headed your way.

What I’m saying is that a negative attitude has to be generated. It is not innate. It comes from somewhere and had to overcome an initial positive bias in order to become strong enough for you to perceive.

Caveats and Addendum

Any one of the following could probably be its own post but here are some things to keep in mind.

• I’m not saying that you should go looking for indications that people hate you. Insecurity breeds reasons to be insecure. If the disconnect manifests, though, be aware that the fault probably isn’t in your professional conduct and no amount of "trying harder" is going to be useful.
• When you screw up (and you will screw up), expect to see that show up in the attitude of your manager and possibly in higher ups (depending on the severity and scope of your screw up). If, over time, you cannot reaffirm the good sense your manager had to hire you in the first place, it’s probably a good time to look for a new job. If, over time, you cannot regain that good attitude in higher ups, it probably means your manager isn’t communicating well. How you handle that depends on the manager. My tendency, however, is to nudge my manager to repair the damage for me because I’m in no position to do so myself. Just letting it go is a bad idea because your manager is letting bad impressions accumulate.
• When you come into a company, you enter as part of a group. There will be prejudices between groups so expect to see that manifest. Marketing managers aren’t going to like programmers. Adjust accordingly.
• Oh, and there is a very small possibility that you are simply a jerk. If every new job ends up with you experiencing prejudice, misunderstandings, and outright betrayal, you might want to take a closer look at that pattern. Close inspection will reveal that the common denominator in all those situations is you...

Steve Harman had a post back at the beginning of the month about stuff you'd tell a young developer. It's a reaction to a similar piece by Jeremy Allison. It's an interesting topic, so I thought I'd waste a few pixels on it myself.

If it's not what you love, don't do it

I wouldn't generalize this to other fields, but for software development, I think this is a good thing to keep in mind. A lot has been made in the past by career counselors and other gurus about "finding your bliss" or similar nonsense. I think that's mostly a crock. You can be a good doctor, mechanic, or professor without liking what you do. Sure, those who love what they do will tend to excel at it and rise to the top of their field, but you have to ask yourself how much better off people at the top of a field are vs. those who are merely good enough.

That said, I think software development is somewhat unique as a career. Software development advances incredibly fast and keeping up with new technologies and ways of doing things takes a non-trivial amount of effort. You have to paddle hard just to stay even and that means experimenting and learning on your own. You could probably do fine if you have Arnold Schwarzenegger level drive, but short of that, loving what you do is about the only way to get you where you need to be.

Reputation is important

People who have been around the block know that the difference between developers can be extreme. Top developers are 5 to 20 times more productive than their peers. Given some of the complete incompetents I've known, I suspect that there's really no upper limit on that number.

Unfortunately, both Jeremy and Steve use this point to jump into the virtues of Open Source software. Open source contributions can enhance your reputation, but that has nothing to do with people being able to look at your code—it's more about working with other developers who can then form their own opinions of you as a developer. That's helpful, but limited.

A good reputation is someone talking about how well you did a job for them. It's your name coming up when someone asks "do you know any good developers?" A good reputation is more important than mere code sample availability and the wider your reputation can penetrate the better off you are.

How important is reputation? Let me put it this way: I haven't gotten a job from anyone who didn't know somebody I'd worked with since 1997. Your reputation and your skills are your only true security in this field.

Never stop asking why

The vast majority of people in any field are content to learn what they have to do to get the job done. Top-tier people are the ones who are asking pesky, even impertinent, questions all the time. Jeremy puts this under the heading of "Learn the architecture of the machine" but that's too limiting. Learn as much as you possibly can about why and how things come together to get the job done.

There's two parts to this, really. The first is that you don't stop just because you got something working. Find out why iterating backwards through an array fixed your bug. Look for the best event to use for control loading rather than just the first one you found that happens to work. Learning how to parse technical documents, navigate help files, and put a concise Google query together are key skills you'll need to master.

The second part of this is harder: if you don't know something people are talking about, ask. This becomes harder to do over time because you start believing your own hype. It's easy to believe that inner voice that's saying you can bluff your way through this conversation and look up anything important later. The thing is, the really bright people, the ones you most want to impress, aren't going to be fooled. I've also found that top technologists enjoy getting intelligent questions almost as much as they love answering them. Note the qualifier there and spend the mental effort to make your questions intelligent, though.

Your employer is more important than the community

This goes directly counter to both Jeremy and Steve and I'm going to tank any chance I have with digg or reddit, but it needs to be said.

Jeremy looks like he's coming from that staunch anti-corporation Slashdot tradition that talks in moral terms about the beauty of open source software and the depravity of Microsoft. Reality check: it is not a common goal of open source developers to create "the greatest, most beautiful work of art that all of you can create together." That's certainly not why I contribute to open source projects.

Open source developers are no more virtuous than corporate developers. While you may program for fun and the satisfaction of a job well done, the choice of project you work on is inherently a selfish one. Always. Whether you program for a paycheck, community recognition, or because you need working blog software that you can tweak when you want to your choice is based on internal needs, not virtues.

If they're both selfish, shouldn't I be saying that they're equally important? I don't think so and it's not just because a company will help support your family. The reason your company is more important is that your company has to compete in the market place. It has no choice. Because a company has to charge money for its products or services, the company is forced to provide things that other people want badly enough to part with their hard-earned cash. In practical terms, it means that a company has a reality check that can't be ignored because sooner or later they're going to have to go out there and prove themselves in the marketplace. While the tech bubble bursting was tremendously painful when it happened, imagine the crap pile we would have if it didn't burst. Ever.

In addition, companies force you, the developer, to do work that you don't want to do. Developers hate that, which is perfectly understandable. Being forced to work with unreasonable managers, clueless users, and hopelessly disconnected marketing weenies is a drag. If you're good, though, it's also a rich source of opportunities to tackle things that you might never have thought needed tackling. And it forces you to justify your dogma of choice to a group of supremely skeptical, self-confident people who have the welfare of their families on the line. Yeah, that's not fun, but it means that you'll have to learn to understand and articulate things you just know are true. With the added bonus that you'll find that they sometimes aren't so true after all.

Finally, be wary of advice from old timers

Not me, of course. My advice is nigh infallible. Just ask me, I'll tell you. If you go awry following my advice, it's probably because of errors in implementation.

Okay, maybe not. I've blogged about recent errors I've made, so I have to assume there will be others. Be careful about who you listen to. Be aware that everybody has their own hobby horses and pet peeves and it's a human trait to cram those prejudices into everything (particularly advice). Also be aware that internet communities trend towards consensus as much as any other human community does. Consensus is dangerous and will tend to mask assumptions and pitfalls. Develop a well-honed BS filter and use it liberally.

While you're at it, the sooner you learn to pass your own ideas and theories through a well-developed skepticism the better off you'll be.