Skill Set Development

by Jacob 25. May 2007 06:28

 I said yesterday that

The skill sets [of developers and QA people] aren't simply unrelated—they are, to an extent, opposed.

 I think that deserves some explanation because for some reason this is not obvious even to people in IT.

The reason that developers and testers require fundamentally different skill sets is that they have fundamentally different responsibilities. What it comes down to is that it is a developer's job to make software work and it is a testers job to make "working" software break. In fact, I knew I had found the right QA Manager for XanGo when one of the first things he said to me was "hey, you wanna see something cool?", opened Excel, and demonstrated how to bring his computer to its knees in three easy steps. I know I'm not much of a tester because while I did think it was cool, I had forgotten those steps practically before he had finished his demonstration.

Separate is Better

When you are creating software, you want to take the quickest path to a functional application. Your emphasis is, correctly, on delivering an end product. You don't want to spend time thinking of every possible situation that can go wrong. You make your program as robust as it needs to be given its use and the importance it has to those using it.

When you are testing software, you want to take the quickest path you can find to break the software. You start with common uses and expand from that point to ensure that the finished software actually does all the things it is intended to do.

Is the skill set used for testing software useful for developers in creating it (and vice versa)? I get this question sometimes from people hoping to save the budget for a QA department. Their real question is "can't we just tell the programmers to do a better job testing their software?" The answer to that is "you save money, but only in the short term."

The thing is, we've found as an industry that testing works best if you have a group of people whose sole responsibility is to test software. This makes sense if you look at it as a case of separation of interests leading to a better combined result. This is why I consider the two skill sets ultimately opposed. When looking at a piece of software, is your intent to make it work or to break it? You really cannot do both at the same time and do them both well.

I'll take it one step further: developers are better off cultivating a perspective of getting their software out the door and testers are better off cultivating a perspective of preventing software from getting out the door until they cannot break it. Could someone, technically, develop both skill sets? Absolutely. I even know one developer who seems to have both built-in. I just don't see why anybody would want to cultivate both when companies are much better served having those functions separated.

Rarity and Value

I also stated yesterday that

QA is a wholly different skill set from development. A skill set, by the way, that is rarer (and hence more valuable).

I think this may be the more controversial statement because testers tend to be paid less than developers of comparable experience. I think this is unfortunate and based on being able to get away with it more than it represents true market value. Testers are often seen as Jr. Developers and that's unfortunate. I think this is possible because those who are truly good at testing are extremely rare. If you've ever had the chance to work with a top tester, though, you'll know what I'm talking about.

The best testers are meticulous to a degree that developers simply find unnatural. They test every edge condition and look for new ones wherever they can. They are almost obsessively detail oriented, even when they don't have to be. And they record everything they can think of.

Top testers improve the developers they work with. That's because a top tester will document fail points and develop hypotheses about why the errors occur in addition to where and under what conditions.

In addition, top testers analyze more than just the software they are testing. They'll assemble things like common fail points and patterns of error--mainly for their own benefit. Indeed, I've found that I sometimes have to dig to get their analysis because they sometimes don't realize the value of their findings to developers.

More subtly, they'll review common bug reports and feature requests and identify areas of improvement that users simply haven't thought of. This comes from their familiarity with the software they are testing. It isn't uncommon for the testers to know the software better than the developers who created it. This makes sense if you consider that a developer often visits any given piece of functionality in mostly unrelated chunks and a tester reviews it as a comprehensive whole.

How Much Testing?

My first experience with the benefits of a strong QA team came at Jenkon in the 90s. I was evaluating our software estimates because they tended to be off by a relatively constant margin. When I started tracking project estimates against actual delivery over time (I first had to start keeping that data. You'd be surprised how few people actually keep it beyond project completion), I found that thorough QA takes almost exactly as long as creating the software in the first place. For some reason, our estimators had it in their head that QA only took a quarter of the development time. Correcting this misperception was instrumental in reining in constant project overruns.

I haven't seen this ratio change since then. Every time someone thinks it has changed, it turns out that they are either skimping on QA or they're not counting all the actual QA time spent. It's been a couple of years since I had access to a decent QA team, though, so I could be off.

Technorati tags: , , , ,

Tags: , , , ,

Management | Programming

Comments


May 30. 2007 05:13
Philk
Loved this article and the last one about winning the QA argument ( I'm losing at the moment )

If I send this article to our CEO do you think he'll give me a pay rise ?


 Brice Richard 
May 30. 2007 13:16
Brice Richard
I think the premise of this article is waaaayyyyy off base. As a software developer myself I consider myself a damn good tester....I test each functionality as I add it as well as its context to its comprehensive whole.

I think any good developer worth his merit is a GREAT tester. After all, if testing requires a detail-oriented person with meticulously refined skills in finding things wrong in software, shouldn't a developer share the same attributes?

If nothing else, testers should know how to effectively document and break apps, and developers should know how to CREATE and TEST their own work. After all, NO ONE understands the logic of your thought processess and how they've been incorporated into your application better than you...


May 30. 2007 14:52
Jacob
Brice,

It isn't about how good a developer you are or aren't. It's about efficient use of time and resources. Asking developers to cover their own QA is like asking trapeze artists to work without a net. They might be the most skilled trapeze artist ever created, but no matter how good they are, they might, heaven forbid, miss. No matter how top-notch your developer-fu is, you are going to, occasionally, miss. You know that deep in your heart of hearts and not having QA means you stand the risk of breaking something important. This makes a developer paranoid and stressed when they don't have to be. This stress and paranoia causes them to spend more time testing the same features than a QA person would and with less effect because they are too close to the code and will tend to test things that therefore already work.

The trapeze artist is a relatively extensible analogy because it covers both top developers as well as new developers looking to get better. Having a net allows them to learn without fear of breaking something live. Even better, though, a QA team provides feedback and detail about what they did to break an app and how to reproduce the error. They do a lot of research so that developers don't have to—research that they are good at and trained for.

Finally, why pay highly skilled developers to learn yet another skill, and one they aren’t well suited for, when you can pay someone else to specialize in it? The benefits of specialization appear very early in the development cycle with QA. I started looking for qualified QA folk around the three developer mark.

Like nightly backups and other hygiene costs, though, people tend to wait on getting a qualified QA team in place until they’ve been bitten. That’s a human trait, so it can be hard to overcome until experience teaches you that the cost of not having it in place is higher than the overhead of a QA team.


 David Thompson 
July 3. 2007 18:35
David Thompson
I have two comments here.  The first one has to do about the distinction between what QA is versus what testing is.  QA encompasses a lot more than just testing code.  QA ensures the quality is built into the product by ensuring the requirements are well-thought out and testable, ensuring the best design to meet the requirements, documenting the testing effort including the results, and creating the metrics so that the software design process can be measured.  QA is preventative whereas testing is a Quality Control activity which focuses on identifying defects in the product that is produced.  It is also possible to have QC/testing without QA.

The second comment has to do with Brice Richard’s reply.  Coders are good testers, but are they good enough?  A primary distinction between a tester and a coder is the following:  A coder wants to prove his code is right, the way he designed it and a tester wants to prove the code is wrong per his understanding of the requirements and what users are capable of (just think of a computer illiterate and what they might do when prompted for a response).   I have tested a lot of programs in my day and there have been numerous times when I test a program and break it very quickly.  I’ve brought the failure to the programmer and I find out that the code was never intended to be used that way – so the programmer never tested it that way.

It should be a requirement that all programmers test their code, but it’s been my experience that programmers forget (or don’t have the ability to) put on their tester glasses on when they begin testing their code.  Therefore, the software development lifecycle needs both programmers and testers.  


July 7. 2007 15:33
Jacob
@David Thompson: Good points both. Not being a QA guy, I tend to lose focus on the distinctions between QA and Quality Control (aka testing). I value both as activities overseen by a competent QA department.

As for coders doing testing, to me, it's not just a case of coders being reluctant to put on their tester glasses as it is an area for profitable separation of concern. Not that coders should be "above" testing or ignorant of testing concerns, just that there's a point just past Unit Testing where diminishing returns kicks in with a vengence.


 Mahesh Kumar 
August 15. 2007 11:10
Mahesh Kumar
I would like to know a formula for skill set development as an hr manager
mahesh


August 15. 2007 13:38
Jacob
Mahesh,

So would everyone. The trouble with skill set development is that it's an internally driven excersize. Frankly, HR folk are the bane of skill set development in my experience. I've left at least two jobs because I developed skills that made me a more valuable employee but raises were capped from HR to a meager three or four percent of current salary. Since they didn't increase my compensation to reflect my better value to the company, I left for a company that would give me a salary that better reflected my value to the company.

See, that's the conflict when you talk about skill set development within a company context. The company wants better-skilled employees, but the company also feels that they shouldn't have to provide both the opportunity for development and give raises that reflect the better value to the company afterwards. It's short-sighted, but so common it hurts. What companies have to realize is that market value is market value and you can't keep quality employees if you pay them below market value.


 Toni 
November 30. 2007 17:43
Toni
Brice - I appreciate that you believe that you can test your own code as well as a QA.  I have found that it just isn't true.  I think the two sides have an inherently different thinking process.  I'm not a good developer because I will code & test until hell freezes over, trying to code for every single possible error condition - not what someone wants if they are actually expecting software to be delivered!  I am an excellent tester, because in my twisted brain, I can throw a possible condition that may not have been originally considered.

As previously stated, a developer concentrates on making something work, they generally live in a limited (functional) context.  A good Tester has an evil streak Smile  Our purpose is to validate that the software fits use/requirements as well as how it integrates with interfacing systems, usability and how errors are managed in unexpected circumstances.

My experience has found that the best 'star' developers skills lie in solving complex problems in elegant ways, not necessarily flawless code...I could still break their code!  Personally, I have more respect for a beautiful solution that may have 'bugs' than a 'hacked' solution that has fewer 'bugs'.  

A Tester also has the benefit of experiencing many different styles of coding (from multiple developers) and how different developers may solve similar problems.  I have also found that (with a truly talented QA/Tester) those 'star' developers develop a very high respect for a talented tester.  I hope that you have the pleasure to experience such a relationship in your work!

Comments are closed

Information

    Recent Posts

    Calendar

    <<  September 2010  >>
    MoTuWeThFrSaSu
    303112345
    6789101112
    13141516171819
    20212223242526
    27282930123
    45678910

    View posts in large calendar
    Disclaimer
    The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

    © Copyright 2010 Scruffy-looking Cat Herder