Wednesday, March 7, 2012

A Context-Driven Approach to Delivering Business Value

This is Part IV in a series of entries inspired by the following quote from the "about page" of hosted by Cem Kaner:
"...However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now..."
And James Bach's blog update (Context-Driven Testing at a Crossroads):
"I’m the last of the founders of the Context-Driven School, as such, who remain true to the original vision. I will bear its torch along with any fellow travelers who wish to pursue a similar program."
If you haven't done so already, I recommend starting with:

So far I've established that I'm a Context-Driven guy. For completeness, I should also share that I'm a guy who is most comfortable operating as part of a healthy team that embraces Agile principles, but who recognizes that Agile is not the most appropriate or effective answer for all organizations, teams, or situations.

I've also noted that I find the notion of "product" in both Context-Driven and Agile principles to be too subtle of a reference to the fact that the propensity of software is developed in a business context for my tastes. This is mostly due to many, many personal observations of individuals involved in the process of developing and delivering software emphasizing some aspect of the software over business value -- from individuals who self-identify as Context-Driven, Agile or neither.

The reality that I have lived in since beginning my career as a technologist is that, business is the primary context-driver behind the development of the propensity of software and that money is the primary context-driver behind business (yes, I know, that's a broad generalization, with somewhat ambiguous qualifiers -- I'm going to ask you trust that I'm happy to support and specify that statement if needed, but for the time being, please accept the premise... at least while reading the remainder of this post.)

If you accept that premise, I believe (and I'm sure folks will point out if and where my logic is flawed) the following is a fair and logical model for testing in the context of business:
  • Business is driven by money
  • Businesses develop & deliver products & services for the purpose of generating or protecting revenue (money)
  • Sometimes businesses also develop & deliver tools & processes for internal use for the purpose of making it faster/easier/cheaper to generate or protect revenue (subsequently "support revenue")
  • The more cheaply and quickly that product/service/tool/process (subsequently "product") can be developed, customized and/or implemented sufficient to generate/protect/support revenue, the better
  • Testing is the activity focused on finding and providing information of interest to people who matter (stakeholders) about the product
  • Therefore- The primary reason businesses pay for testing is because:
    • they want as much information as possible
    • for a reasonable cost
    • available to stakeholders involved with developing, customizing, implementing, assessing, managing, and/or making business decisions 
    • related to the relevant product
    • with the expectation of that information allowing the product to start generating, protecting, or supporting revenue more quickly and cheaply
    • than it would if the business had *not* paid for testing.
In other words, the primary reason that testing happens in a business context is to help the business achieve their primary goal, which is to make money. Or even more simply, businesses choose to pay for testing *only* because they believe it costs less to pay for it than it will cost to *not* pay for it in the long run.

See the graphic below for another way to think about this.

(Larger view, .pdf format)

I could follow the same logic (but I will spare you for the time being) related to what most businesses treat as their next most significant context-driver, Risk.

I'd like you to reflect for a moment and notice some things about what I just did:
  1. I made a case that testing is all about money in a business context
  2. I never mentioned "requirement", "test case", "exploratory", "automated", or "tester"
  3. In fact, I made my case without ever mentioning "software"
Needless to say, this was intentional. I submit that the logic above applies equally well (or equally poorly) to physical products, qualitative services, software tools, process templates, or most anything else businesses use to generate, protect or support revenue. That is because I believe that while  software is certainly different from physical products in some important ways, from a business perspective, the goals & decision making processes are very much the same as for other types of products; thus requiring the same kinds of information.

In fact, I'll go a step further, software development is not all that different than physical product development *except* for what I have labeled in the graphic as "Productization". In both cases, what we are really referring to prior to Productization is "Research and Development" (again, that's an assertion deserving of far more discussion and support than makes sense for this post. Consider this a bookmark for a future post).

So, what's my point? Testing in a business context is done to help the company make/keep money. Testing is not done to ensure (or assure) quality, to protect the customer, or to make the software "good". Testing done with the intent of improving the quality of the product is *only* valuable until the product achieves a certain minimum quality as determined by the business to be acceptable for achieving business goals (and in actuality is, even then, only incidentally valuable). Any testing done that cannot be linked to generating, protecting or supporting revenue is, in effect, working against the goal of the business (since that testing cost money, but does not provide corresponding monetary value).

Ergo, all that study, advancement, & education about how to test better may have made you a better tester, but did it make you more valuable to your business? I say, only if you have managed to apply that knowledge in a manner that made more of your testing more focused on business value and as a result helped your business make or keep more money.

I'll go out on a limb and say that for most of you, the training and education you received over the last 10 (or more) years was far more focused on things like test process, techniques, tools, and metrics than on providing business value with the skills and knowledge at your disposal.

And *that* is why testers haven't gained more respect, why ridiculous statements like "Testing is Dead" get so much traction. It's not that testING is dead, it's that testERS are the only ones who seem to be interested in calling attention to testing as separate line item under "product development". The reality is that testing gets done pretty much everywhere that research and development happens, products are made, or services are developed -- physical, conceptual, or software. The value of testing to the business is being realized (to an acceptable degree to the business) in areas other than software even when the word "testing" is completely absent from the business radar. When it comes to software, on average, testing is on their radar, and testing is not providing the value they are looking for (or, at least is not being communicated in a way they understand).

That thought process is what has led myself and several collaborators (I'll allow them to self-identify if/when they so choose) down this road of deeply exploring business value as it relates to testing (and software in general). What I've included in this post is little more than a condensed summary of the background to our research, but now seemed like as good a time as any to let the cat out of the bag a little and gauge the public reaction and counter-arguments before our research goes far enough that it would make us angry (as opposed to grateful, sad and/or confused) if/when folks find flaws or gaps in our thinking.

Oh, and if you're wondering what the connection is between all of this and Context-Driven, it lies in the "generate, protect and/or support revenue" statement.  For example, testing related to revenue generation may focus on functionality or regulatory compliance; testing related to revenue protection may focus on maintainability or legal defense; testing related to supporting revenue may focus on business process improvement or cost reduction. Of course there are many, many other potential context-drivers, but that too is a discussion for another day.
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
Director, Computer Measurement Group

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."


Catherine Powell said...

I've ridden enough startups down to know that the best software ever written is no good unless it brings you revenue and value measured in metrics that matter to the business. That's really independent of whether you subscribe to a "schools" idea, or if you think context-driven testing is wonderful (or terrible).

More here:

And best of luck - let's spend some time talking about WHY so that we can really effectively figure out WHAT and HOW.

Calkelpdiver said...

My friend you have swung for the fences and drove in a bases load home run with this one.

Straightforward and succinct post. Love the graphic, and I'll definitely be using it in the future (with your permission of course).

I don't know what else to say buddy, just a fracking great post.

David Greenlees said...

I'm with Calkelpdiver. F'ing awesome dude!

Will there be a part V? Please make a part V!

I think you've nailed a poosible gap in the training market. I couldn't agree more on your comments in that area. I've been to many... All focused on the quality of the /product/. Business drivers didn't rate a mention!

The graphic is top-notch too.

Phil said...

Hope there is a lot more to come and be discussed around this - there's plenty on what the best test management tool is, why doesn't my work - but little on showing your value to the business. Looking forward to reading more

mark said...

Hello Scott,

Great points! I've been coming to this realization during my years of testing and for the last couple of years in my current organization I've been actively fighting the sentiment that testing is owned by the test team and performed at the end of the release cycle.

I wish that it was true that, "... testERS are the only ones who seem to be interested in calling attention to testing as separate line item under 'product development'." But all too often there are product developers that also forget they are providing business value and not just churning out products that others are responsible to validate.

Thanks for the post, you've get me thinking of some new ways to express the issue.

-Mark V. Alexander

Unknown said...

To address several of the above comments:

- Thank you all! I'm feeling very "not alone" on this topic -- which is a good thing. Though I have to admit, I keep wondering "When are the naysayers going to present counter-arguments?!?" There must be some, right? If there's no counter-argument, then there really *is* no reason for things to be the way they are!

- Nope, there will be no Part V. However, there will be many parts to come on the Business Value topic (Just look for "Business Value" in the title... I'm afraid to do numbered parts, 'cause I don't want folks to feel they need to read them in sequence)

- Funny you should ask... I was all ready to publish this and thought to myself "Self, I should probably stick a copyright notice on the bottom of the pic, just in case someone decides to use it for something."... so I did. Which means that you are all free to do with it as you choose, so long as you don't edit/recreate it with trivial changes without providing attribution to the original.

- No, testers aren't the only group that fails to give Business Value the focus "the business" would claim it is paying for. IMHO, this is an IT (development) wide (or wider, but I'm only comfortable speaking on my experiences) issue in varying degrees. That said, TesterLand is my homeland, so I'm gonna start here. Maybe this is a topic that TesterLand can rally around and actually *lead* a positive industry advancement for a change instead of always seeming to follow -- or not, but a guy can dream, yeah?

Mike Kelly said...

Excellent post Scott. I've already forwarded it to several people. As mentioned, the graphic is fantastic.

David Greenlees said...

Scott - Put this on LinkedIn... Then you'll get some naysayer reaction!

*insert evil laugh here*

Shrini Kulkarni said...

>>> In other words, the primary reason that testing happens in a business context is to help the business achieve their primary goal, which is to make money. Or even more simply, businesses choose to pay for testing *only* because they believe it costs less to pay for it than it will cost to *not* pay for it in the long run.

So - testing will live as long as it is cheaper to pay for it than not to pay for it. If you reckon what Alberto said in GTAC "Test is dead" - Speed first quality second" and What James Whittaker said in eurostar 2011 (a paraphrase) - "it is easy and cheaper to fix production bugs as you can test and deploy the fix for a small (1%) of your users - testing is dead - it does not make sense to pay for it.

The business context in todays world - especially in software product makers world (likes of twitter, facebook, linkedin) - has made "not doing testing" more viable than doing it.

What do you say?

Shrini Kulkarni said...

>>> Testing in a business context is done to help the company make/keep money

While testing keeping mind related business context is often valuable - making connection between day today acts of testing and larger business model and making money is OFTEN difficult.

Try explaining above statement to an entry level tester in company like Microsoft or google. Large size and diversified portfolio, complex social, political and economical landscape of business - makes an individual tester look small and insignificant with respect to "making money" part.

Again would not it be unreasonable for a tester in big organizations to have visibility of money making of the organization?


Unknown said...

@Shrini -- several comments:

1) James & Alberto (if you listen closely) really weren't saying that the activity of testing is dead, they were saying (my words) that the way that the propensity of organizations that employ the overwhelming majority of testers in the world are thinking about testing in an antiquated and ineffective way.

2) Also remember that "cost" has a lot of aspects to it. For instance, Google/Facebook can take some risks w/ 1% of users... their revenue stream comes from adds, so as long as the add providers don't go away... Financial services companies, however, can experience *huge* nth order financial impacts by just one user experiencing one production defect -- the direct cost would certainly be less than the cost of testing up front, but what about the total loss of revenue due to reduced credibility from the NYT article blasting the company for the defect? Put a price tag on that!

3) Why are you (and most others) so focused on making the most Jr. Tester responsible for figuring all this stuff out on their own? It's just another example of all the "we're all testers, but we're all completely separate & trust nothing we don't come up with ourselves" mentality that (IMHO) is significantly contributing to our credibility issues as a field/craft/whatever. It's not the Jr. Tester's job to figure out the subtle details of the "Making Money" part. That's the job of the Test Manager (or Director, or VP) & it's their job to funnel the appropriate information down, guide the strategy and tasks and in general take accountability to and responsibility for serving the best interest of the business first... not the project first.