Tuesday, November 8, 2011

On the Alleged Death of Testing

Out of respect for your time, I'll give you the bottom line up front as a simulated interview that I privately hoped for, but never came. After the mock interview is a supporting narrative for those of you more interested in my thinking on the matter.

Q: There's been a lot of talk recently about testing being dead, so my first question is testing dead?
A: No.

Q: Some of those talking about the alleged death of testing are saying that it's not that testing as a whole is dead, but that testing as it is commonly understood today dead. Is it?
A: No.

Q: Ok, so is testing as it is commonly understood today dying?
A: Not that I can see.

Q: Then why all the talk about testing "as we know it" being dead?
A: IMHO? Wishful thinking.

Q: How so?
A: The state of the testing practice is not evolving nearly as quickly as development, business, or products containing or depending on software. The testing practice is on long-term life support, and that life-support seems to be the ticket to immortality.

Q: Should testers be concerned about this evolution gap?
A: Only if they are afraid of change, have stagnated in their own professional development, and/or believe what they are doing today "is right and will continue to be right for at least as long as they will be testers".

Q: Should others be concerned about this evolution gap?
A: Yes - particularly consumers who care about appropriate quality for cost of products depending on software, and businesses who care about delivering those products with an appropriate balance of speed, cost, market satisfaction and profitability.

Q: What can testers do to narrow this evolution gap?
A: IME, not much. All they can do is embrace evolution as it happens vs. fighting it and thus slowing the evolution yet further and focus on adding value to their businesses rather than focusing on doing what they have decided is important.

Q: If it's not the testers, who can lead the narrowing of the evolution gap and how?
A: Consumers must stop paying for products that depend on software that doesn't achieve their cost for quality bar; Businesses must commit to improving the balance of cost, speed, market satisfaction and profitability; Executives, middle-managers and line-managers must admit that they don't understand the business value they could be, but aren't, getting from testing, get themselves educated and become part of the solution instead of part of the problem.


Yes, you read that correctly. I "wish" that testing as it is commonly understood today, at least as it is commonly understood by the corporations, managers, and testers I train and consult with, were dead -- or better still, had not contracted the life-threatening diseases that have required testing to spend the last 30 or so years on life-support systems so restrictive as to limit it's value adding ability.

The problem here, at least in my opinion, is simple.
  • Testers have become so focused on legitimizing their discipline and their role that they've forgotten that their role and their discipline exists to serve the companies that employ them.  
  • Conversely, the managers and executives of companies that employ testers are regularly directing testers to do these things they don't really understand rumored to be "best practices"-- and of course, then blame the testers when those "best practices" don't provide sufficient value.
  • There is no "basic training" for testers, and there is no "manager basic training" that covers testing and test management as it relates to value to the business -- and even if you believe the training exists, you can't deny that:
    a) many folks wouldn't agree that training is of value
    b) even more folks who "should" have received that training haven't even heard of it
Put more simply, software testing today, in my opinion, is dominated by:

The under-informed
leading the under-trained
to do the irrelevant.

The simple truth is that companies don't want to pay for testers and they don't want to have to think about testing.  Companies want to create products, as quickly and cheaply as possible, that generate revenue or reduce costs at an acceptable degree of risk to the company.  Product quality is a second or third order effect derived from those corporate desires.

It seems clear to me that testing, then, should be all about helping companies create products that are viable for generating revenue and/or reducing costs, quickly and cheaply, while identifying, mitigating, and/or controlling business risk -- not protecting the end-user from that annoying bug.  Protecting the end-user is called consumer-advocacy (an honorable discipline to be sure, but I'm willing to bet that it's not the task your company is hiring testers to perform).

So when I say that I wish testing as we've come to know it were dead, what I mean is that I think the propensity of the testing discipline is stuck in trying to force employers to pay them to be consumer-advocates when what those employers really need are trusted advisers focused on helping the company and its products achieve success along an often conflicting continuum of 'success criteria'.

Personally, I'd not be sad if the role of "tester" went away because I know that I have the skills and knowledge to add value to companies by helping them to deliver products (whether those products are software, contain software, use software, or are designed/developed/manufactured using software) faster, cheaper and appropriately fit for use & purpose, while helping those companies identify, mitigate and control associated business risks.

Can you say the same thing?

Would your employer agree? 

Think about it.
Scott Barber
Chief Technologist, PerfTestPlus, Inc.

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."


Alberto Savoia said...

Good post Scott.

The phrase "test is dead" is a bit of a Rorschach test; it's interpreted by different people in different ways, reflecting their views, opinion, fears, wishes, hopes, etc.

As one of the "test's executioners", here's my take on it: http://www.youtube.com/watch?v=X1jWe5rOu3g


Unknown said...

Brilliant keynote Alberto, thanks for sharing!

Yvette Francino said...

Interesting discussion! Last night at our local user group meeting, recruiters confirmed the trend towards testers needing to grow their technical skills. http://itknowledgeexchange.techtarget.com/software-quality/recruiters-advice-to-testers-get-technical/

Ben Kelly said...

Nice. This post pretty much nails it from my perspective, Scott.

I liked Alberto's keynote. The phrase 'don't take it literally, but take it seriously' is sensible. For web startups the game has changed. For everyone else, things will change too - perhaps more slowly, but that's a bit like saying water is wet. Neither of those things is anything new.

I want test theatre (in the same way that the TSA performs security theatre) to die. Soon. The propagation of the release fast, fail fast paradigm will no doubt go some way to assisting that. Unfortunately, I don't think that's enough to spell the ultimate demise of the fake tester. I don't know yet what if anything will.

Unknown said...

Thanks Ben.

I think you're on to something there... "On the long slow death of fake testing: Sign the DNR & pull the plug!" :)

Mike Dwyer said...

Thanks for sending the youtube. I really enjoy having my brain spun in the cranium.
But I have to say that when I look at the ink spots (thanks alberto) I see testing as it is practiced today being a major propulsion unit in the "doing 'it' right" death spiral.
What we have talked about is testing as the core part of the navigation system, helping the emergence of the right 'it'.
I need to get the spinal column reattached to the brain stem and then I will blog about it.

Dan Alloun said...

Hi Scott and thank you for this excellent post.
Few days ago I was discussing this "Test is Dead" buzz and I'm feeling even better now to see that I'm not an isolated brain.
However sometimes we need such kind of "slogan" to wake up and be sure we can move forward :-)


Savita said...

Thanks for writing this beautiful thought!

Jerry says that Quality product is cheaper in long run.

Employer, Customer and Tester should realize that human skills can not be replaced with automation.

Jesper L. Ottosen said...

trackback: http://jlottosen.wordpress.com/2012/04/17/the-core-purpose-of-softwaretesting/