Top 10 Tips for Test Managers from STP Online Summit

I had the pleasure of hosting the another Online Summit, delivered by Software Test Professionals: Test Management: Bridging the Gap Between Tests and Stakeholders.  The online summit format consists of 3 sessions each for 3 consecutive days.  The sessions for this summit were:

• Invigorating Your Test Team – Anne-Marie Charrett
• Talking to Triangles – Dawn Haynes
• Mentoring Testers: Leading from the Inside Out – Brian Osman
• Getting Your Testing MBA – Catherine Powell
• Whiteboard Test Team Management – Paul Holland
• Test Management for COTS Applications – Virginia Reynolds
• Motivating and Managing Distributed Testers – Karen Johnson
• The “Testing Bubble”:  Motivating, Insulating, and Protecting Tester – Ben Yaroch
• Top Tips from the Summit – Scott Barber
• Speaker Panel – Scott Barber, Karen Johnson & Ben Yaroch

One of my duties as host was to try to summarize the most valuable nuggets of information from across all of the presentations into a "top tips" list.  This is what I came up with:

Scott's Top 10 Tips for Test Managers from:

Desperately Seeking "Performance Unit Testing" Examples

I've been talking about what I term "Performance Unit Testing" in classes and training courses for a long time. I've been teaching (more inspiring with hints toward implementation) client development teams about it for almost as long. Problem is, all I've got is stories that I can't attribute (NDAs and such) and that simply doesn't cut it when trying to make a point to someone who doesn't (or doesn't want to) get it.

So I'm looking for attributable samples, examples, stories, and/or case studies related to "Performance Unit Testing" that I can use (clearly, with attribution) in talks, training classes, maybe even blogs & articles. If you have something, please email me.

If you're not sure if you've got what I'm looking for, lemme share some desired attributes of what I'm looking for:

Agile Sprint Sanctity Valued at Over $13M?!?

During STPCon last week (which, BTW, was fabulous, but more on that in another post, 'cause I've got to get this off my chest) I was a panelist for The Hard Stuff: Questions About Agile. During the course of the discussion, someone asked a question that I heard as the following:

"... but what should I do about our sprints getting messed up when [executive] comes in and tells us to stop what we're doing and add [feature X] before the end of the following week so s/he can finalize the $13 Million deal with [new client Y, but only if the feature X is implemented by then]..."

Let's Test 2012

The first (as far as anyone I know is aware) Context-Driven conference in Europe is quickly approaching. On May 7-9, 2012 in Stockholm, Sweden, Let's Test "A European conference on context-driven testing - for testers, by testers" will take place.

This is a CAST inspired conference, meaning that it focuses on in-depth exploration of topics, includes facilitated discussion as part of every talk (i.e. speakers don't get to "run out of time" as soon as they hear that "hard question") and conferring only increases between and after sessions. It's a fabulous format! If you haven't experienced it, and you are passionate about testing, you really want to -- it will change your perspective on conferences forever.

I am proud to say that I will not only be attending Let's Test 2012, but that I am honored to be on the program with some first-run content that I'm very excited about:

A Full Day Tutorial: Context Appropriate Performance Testing, from Simple to Rocket Science
A Keynote: Testing Missions in Context From Checking to Assessment
 

Trust is a Cornerstone to Delivering Business Value

In my last post about Metrics I introduced the notion of trust as it relates to Business Value by stating:

"Failing to trust 'the Business' does NOT add Business Value"

I'd like to generalize that statement further to say "A lack of trust that individuals or groups involved in the project are primarily focused on helping the business succeed undermines business value".

Now, I can only imagine the reaction many testers are having while reading this. For instance "If I trust the developer when they say 'This is fine, you don't need to test it', we'll have major bugs make it to production." And anyone thinking that would be absolutely right -- because that is not the *kind* of trust I'm talking about.

When I say trust, I don't mean "Trust others to tell you how to do your job" or "Trust others to do what you believe is correct/best" or even "Trust others to be successful in accomplishing what they have been assigned to accomplish on time, on mission, on quality, and on budget"

When I say trust, I mean "Trust others to approach their role with integrity" and "Trust that others are doing the best they can to make the decisions or take the actions appropriate to their role and responsibilities based on the information they have" and "Trust that if you haven't been assigned to do or to be the decision maker about something, that task or decision is better handled by someone else -- whether or not *you* have the information necessary to make sense out of why.

Business Value of (Software Test) Metrics

I was *totally* in the middle of another blog post when I came across the latest from Cem Kaner on context-driven-testing.com and after reading it, I (cognitively speaking) had no choice, but to save that other post as a draft and write this one -- and by that I mean, there's no way I was going to be able to focus on anything else until I got this down and out.

So, click the link below and read the post... no seriously, it's required reading for what I have to say. (Ok, if you want you can start at the line prior to the 2 bullet points about half-way down the screen -- the backstory is optional reading)

Metrics, Ethics, & Context-Driven Testing (Part 2)

Did you read it? No? I mean it. Go. Read. I'll wait.

10 Take Aways from STP Summit on Agile Transitions

I had the pleasure of hosting the fourth Online Summit, delivered by Software Test Professionals: Agile Transitions.  The online summit format consists of 3 sessions each for 3 consecutive days.  The sessions for this summit were:

• What is Agile and What's it Got to do With Testing? – Scott Barber
• How "Agilists" vs. "Traditionalists" View Testing – Robert Walsh
• Keys to Transitioning to Agile Testing: Lessons Learned from the Trenches – Bob Galen
• The Secret to Successful Agile Test Automation – Lisa Crispin
• Culture & Inter-Focus Area Interactions – Selena Delesie
• Avoiding Agile Perversion – Lanette Creamer
• Excelling as an Agile Tester – Henrik Andersson
• The Lone Tester In an Agile World – Michael Larsen
• Top Take Aways  – Scott Barber  
• Speaker Panel – Scott Barber, Robert Walsh, Lisa Crispin, Selena Delesie, Henrik Andersson, Michael Larsen

One of my duties as host was to try to summarize the most valuable nuggets of information from across all of the presentations into a "top take aways" list.  This is what I came up with:

Scott's Top 10 Take Aways from:

Business Value of Testing: Find Bugs ≠ Mission

My introduction to software testing was as a performance tester. Before the completion of my first project, I had a firm understanding of the primary mission of performance testing. That understanding has changed very little to this day, though I have improved how I communicate that mission. Currently, I phrase the (generic) primary mission of performance testing as:

"To find how many, how fast, how much and for how long; to assist business w/ related technical and/or business decisions and/or; to support related improvement activities as prioritized by the business."

That certainly doesn't mean that all performance testing efforts include every aspect of that mission, but I'm hard pressed to imagine something that I'd call performance testing that includes none of the aspects of that mission. It is also true that I have experienced all manner of secondary missions as part of a performance testing effort (missions related to fail-over, and denial of service attack security, for example). Those combinations, permutations, variations and additions are all where context comes into play.

However; when I was first asked to help out with some functional testing, I quickly realized that I didn't really know what was expected of me, so I asked. As you might imagine, folks looked at me like I'd grown a second head... a green one... with scales and horns. After about the 4th time I asked the question, got a funny look, watched as it became clear I was serious and received some variant of the answer:

"Find bugs"

Processing may take up to 60 seconds?!?

Seriously?!? This was a simple ccard transaction for a storage unit! Admittedly, it only took about 20 seconds, but it was still long enough for me to push the button, read the text, exclaim "You've *GOT* to be *KIDDING* me!!", my 12 y/o son to ask "What?", me to respond "60 seconds to process a payment on line, " him to reply "That's stupid", me to launch snipping tool & grab a capture before it processed my payment.

Grrr.... Hey, I've got an idea, why don't they give me a unit free for the next 10 years in exchange for 25 hrs of performance testing/tuning (and that would *still* be less than my typical bill rate) so that other folks don't have to deal with this crap.

Context-Driven Testing Crossroads: Addendum

I guess I wasn't as done talking about this as I thought. Earlier today, I posted the following comment (except with a few extra typos that I chose to fix below) on Tim Western's blog in response to his post Is the Context Driven School of Testing - Dead?:

"A point that I think many miss is that this is not just about individual testers.

50 years ago (more or less) testING began fighting a rather arduous battle to establish an identity separate from developMENT. This, eventually, led to testERS establishing an identity separate from developERS.

A Context-Driven Approach to Delivering Business Value

This is Part IV in a series of entries inspired by the following quote from the "about page" of context-driven-testing.com hosted by Cem Kaner:

"...However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now..."

And James Bach's blog update (Context-Driven Testing at a Crossroads):

"I’m the last of the founders of the Context-Driven School, as such, who remain true to the original vision. I will bear its torch along with any fellow travelers who wish to pursue a similar program."

If you haven't done so already, I recommend starting with:

• Part I: Is Testing Dead? Dunno, but the Context-Driven School Is 
• Part II: With the Context-Driven School "closed" what's next?
• Part III: Context-Driven School (of thought): "I'm not dead yet... I feel happy!"

---

So far I've established that I'm a Context-Driven guy. For completeness, I should also share that I'm a guy who is most comfortable operating as part of a healthy team that embraces Agile principles, but who recognizes that Agile is not the most appropriate or effective answer for all organizations, teams, or situations.

I've also noted that I find the notion of "product" in both Context-Driven and Agile principles to be too subtle of a reference to the fact that the propensity of software is developed in a business context for my tastes. This is mostly due to many, many personal observations of individuals involved in the process of developing and delivering software emphasizing some aspect of the software over business value -- from individuals who self-identify as Context-Driven, Agile or neither.

The reality that I have lived in since beginning my career as a technologist is that, business is the primary context-driver behind the development of the propensity of software and that money is the primary context-driver behind business (yes, I know, that's a broad generalization, with somewhat ambiguous qualifiers -- I'm going to ask you trust that I'm happy to support and specify that statement if needed, but for the time being, please accept the premise... at least while reading the remainder of this post.)

2 Cents on Ethics

I really had no plan to chime in on the blog conversation between Michael Bolton and Cem Kaner, but after the amount of time I've spent today having email discussions with folks who (apparently) were interested in my 2 cents, I've decided to go ahead and share. I feel it important to point out that as I have spoken to neither of them regarding this conversation, I most certainly don't want to give the impression that I am speaking for either of them.

(As a side note, I'm seriously beginning to wonder if I shouldn't just add a "Notes and Disclaimers" box to my blog... then again, that would be about the same as prefacing all my notes and disclaimers with "Allow me to provide some context" -- which would seem rather redundant coming from me. {grin})

Anyway, it would seem that it all started with Michael's post Why Pass vs. Fail Rates Are Unethical (Test Reporting Part 1) that, if not inspired, certainly contributed to Cem's post Contexts differ: Recognizing the difference between wrong and Wrong which, unsurprisingly, triggered the following post by Michael I Might Be Wrong (But Not For Me)

Ok, all caught up? Good. Lemme share what I think might be happening here and while I'm at it share my model for approaching ethics-related situations in business environments (testing or otherwise).

Context-Driven School (of thought): "I'm not dead yet... I feel happy!"

This is Part III in a series of entries related to the following quote from the "about page" of context-driven-testing.com hosted by Cem Kaner:

"...However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now..."

If you haven't done so already, I recommend starting with:

• Part I: Is Testing Dead? Dunno, but the Context-Driven School Is 
• Part II: With the Context-Driven School "closed" what's next?

---

Ok, so maybe not "happy" but I couldn't resist the Monty Python reference.

James Bach stated on his latest blog update (Context-Driven Testing at a Crossroads):

"I’m the last of the founders of the Context-Driven School, as such, who remain true to the original vision. I will bear its torch along with any fellow travelers who wish to pursue a similar program."

With the Context-Driven School "closed" what's next?

This is Part II in a series of entries related to the following quote from the "about page" of context-driven-testing.com hosted by Cem Kaner:

"...However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now..."

If you haven't done so already, I recommend starting with Part I: Is Testing Dead? Dunno, but the Context-Driven School Is

---

Much like when one completes an educational program at one institution and ponders whether or not to enroll in another program (and if so, which one), or to enter the workforce and continue their learning along the professional development or self-education path, I think it's fair for those who have come to self-identify as members of the Context-Driven School to be asking themselves similar questions.

And much like completing an educational program does not equate to losing the lessons learned (as opposed to the lesson's taught) in the program, the Context-Driven Principles and the lessons many of us have learned by studying in (or, for that matter, rebelling against) the Context-Driven School remain despite Cem's announcement that (in my words) the school is now closed.

Is Testing Dead? Dunno, but the Context-Driven School Is

Well, I'm sure this is a bit of a shocker for many of you, but the following quote comes from the "about page" of context-driven-testing.com hosted by Cem Kaner:

"...However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now..."

This is Part I of a series of entries on this topic. Links to subsequent parts will be added to the bottom of this entry as they are posted.

---

Of course, this doesn't  negate or erase the Context-Driven Principles, and Cem has committed to keeping the original content on landing page of the revised site:

"...When you land on this site, you see the context-driven-testing.com landing page (the Principles) as it was when we originally published it. I’ll keep it that way (with the same set of Principles), because several people have found it useful..."

To my way of thinking, the *most* important point made by Cem on the About Page is the following:

..."This notion of evolution comes with a built-in assumption: If my thinking will evolve to something else in the future, it must be wrong today. Progress on my path to better understanding and practice of testing (and of anything else that I’m serious about) includes discovering what needs to be changed in my thinking, and changing it.

This is an important aspect of science. We don’t run experiments to confirm what we already know. We run experiments to prove that what we think we already know is wrong. And to help us develop something better..."

This is the point I'd like folks to focus on.

Curse of the Performance Tester?

Seriously?!? After wrapping gifts until nearly 5am (I was behind by even my standards due mostly to work travel, client commitments & preparing to close the corporate books for 2011), and getting up before 8am to celebrate Christmas with my boys, I finally stole a few minutes when I noticed they'd both fallen asleep on the couch to play with *my* new toy (i.e. install Skyrim), only to be foiled by...

10 Take Aways from STP Summit on Metrics

I had the pleasure of hosting the third Online Summit, delivered by Software Test Professionals: Deliver Value with Testing Metrics: Move Beyond the Myth.  The online summit format consists of 3 sessions each for 3 consecutive days.  The sessions for this summit were:

• My Testing Metrics Allergy – Dawn Haynes
• The Math Behind the Lies – Doug Hoffman
• Change the Questions for More Valuable Answers  – Scott Barber
• Controversies in Quality Metrics – Yvette Francino
• Telling The Testing Story – Fiona Charles
• More "T" and Less "BS" – Jon Bach
• A New World: Metrics That Really Count – Harry Robinson
• Risk Metrics & Reporting – Julie Gardiner
• Top Take Aways  – Scott Barber  
• Speaker Panel – Scott Barber, Fiona Charles, Jon Bach, Doug Hoffman, Harry Robinson, Yvette Francino

One of my duties as host was to try to summarize the most valuable nuggets of information from across all of the presentations into a "top take aways" list.  This is what I came up with:

Scott's Top 10 Take Aways from: