Lessons from NEXT2012 in Romania

I often see folks blogging about what they learned, were inspired by, or impressed them about attending an event. it is far less often when I see a headliner, or promoted presenter blog about the lessons they learned or what inspired or impressed them after the event. I've often wondered why that is.

For me, it has a lot to do with needing to quickly shift gears upon completing an event to catch-up on all the things that I put off to prepare for the event, figure out what immediate stuff landed in my inbox while I was ignoring it, and to follow-up on leads, lessons, inspirations and curiosities from the event itself.

Well, I'm going to make a concerted effort to do better about posting my lessons from events, starting with NEXT2012, hosted by SoftVision, held in Cluj-Napoca, Romania, Oct. 26-27
So, what were my take-aways from NEXT2012?

• I'm *really* excited about how I'm now organizing and packaging my performance-related materials (more on that in a separate post).
• SoftVision did a fantastic job organizing and handling logistics.
• I am seriously impressed with the people I interacted with on both a professional and technical level.
• Those same people are social, collaborative, friendly and are able to enjoy their work and create enjoyable work environments while being professionally and technically impressive.
• Romania (as well as several surrounding areas not widely considered "software/technical powerhouses") is an emerging market worth watching.

Agile Sprint Sanctity Valued at Over $13M?!?

During STPCon last week (which, BTW, was fabulous, but more on that in another post, 'cause I've got to get this off my chest) I was a panelist for The Hard Stuff: Questions About Agile. During the course of the discussion, someone asked a question that I heard as the following:

"... but what should I do about our sprints getting messed up when [executive] comes in and tells us to stop what we're doing and add [feature X] before the end of the following week so s/he can finalize the $13 Million deal with [new client Y, but only if the feature X is implemented by then]..."

Let's Test 2012

The first (as far as anyone I know is aware) Context-Driven conference in Europe is quickly approaching. On May 7-9, 2012 in Stockholm, Sweden, Let's Test "A European conference on context-driven testing - for testers, by testers" will take place.

This is a CAST inspired conference, meaning that it focuses on in-depth exploration of topics, includes facilitated discussion as part of every talk (i.e. speakers don't get to "run out of time" as soon as they hear that "hard question") and conferring only increases between and after sessions. It's a fabulous format! If you haven't experienced it, and you are passionate about testing, you really want to -- it will change your perspective on conferences forever.

I am proud to say that I will not only be attending Let's Test 2012, but that I am honored to be on the program with some first-run content that I'm very excited about:

A Full Day Tutorial: Context Appropriate Performance Testing, from Simple to Rocket Science
A Keynote: Testing Missions in Context From Checking to Assessment
 

Trust is a Cornerstone to Delivering Business Value

In my last post about Metrics I introduced the notion of trust as it relates to Business Value by stating:

"Failing to trust 'the Business' does NOT add Business Value"

I'd like to generalize that statement further to say "A lack of trust that individuals or groups involved in the project are primarily focused on helping the business succeed undermines business value".

Now, I can only imagine the reaction many testers are having while reading this. For instance "If I trust the developer when they say 'This is fine, you don't need to test it', we'll have major bugs make it to production." And anyone thinking that would be absolutely right -- because that is not the *kind* of trust I'm talking about.

When I say trust, I don't mean "Trust others to tell you how to do your job" or "Trust others to do what you believe is correct/best" or even "Trust others to be successful in accomplishing what they have been assigned to accomplish on time, on mission, on quality, and on budget"

When I say trust, I mean "Trust others to approach their role with integrity" and "Trust that others are doing the best they can to make the decisions or take the actions appropriate to their role and responsibilities based on the information they have" and "Trust that if you haven't been assigned to do or to be the decision maker about something, that task or decision is better handled by someone else -- whether or not *you* have the information necessary to make sense out of why.

10 Take Aways from STP Summit on Agile Transitions

I had the pleasure of hosting the fourth Online Summit, delivered by Software Test Professionals: Agile Transitions.  The online summit format consists of 3 sessions each for 3 consecutive days.  The sessions for this summit were:

• What is Agile and What's it Got to do With Testing? – Scott Barber
• How "Agilists" vs. "Traditionalists" View Testing – Robert Walsh
• Keys to Transitioning to Agile Testing: Lessons Learned from the Trenches – Bob Galen
• The Secret to Successful Agile Test Automation – Lisa Crispin
• Culture & Inter-Focus Area Interactions – Selena Delesie
• Avoiding Agile Perversion – Lanette Creamer
• Excelling as an Agile Tester – Henrik Andersson
• The Lone Tester In an Agile World – Michael Larsen
• Top Take Aways  – Scott Barber  
• Speaker Panel – Scott Barber, Robert Walsh, Lisa Crispin, Selena Delesie, Henrik Andersson, Michael Larsen

One of my duties as host was to try to summarize the most valuable nuggets of information from across all of the presentations into a "top take aways" list.  This is what I came up with:

Scott's Top 10 Take Aways from:

Business Value of Testing: Find Bugs ≠ Mission

My introduction to software testing was as a performance tester. Before the completion of my first project, I had a firm understanding of the primary mission of performance testing. That understanding has changed very little to this day, though I have improved how I communicate that mission. Currently, I phrase the (generic) primary mission of performance testing as:

"To find how many, how fast, how much and for how long; to assist business w/ related technical and/or business decisions and/or; to support related improvement activities as prioritized by the business."

That certainly doesn't mean that all performance testing efforts include every aspect of that mission, but I'm hard pressed to imagine something that I'd call performance testing that includes none of the aspects of that mission. It is also true that I have experienced all manner of secondary missions as part of a performance testing effort (missions related to fail-over, and denial of service attack security, for example). Those combinations, permutations, variations and additions are all where context comes into play.

However; when I was first asked to help out with some functional testing, I quickly realized that I didn't really know what was expected of me, so I asked. As you might imagine, folks looked at me like I'd grown a second head... a green one... with scales and horns. After about the 4th time I asked the question, got a funny look, watched as it became clear I was serious and received some variant of the answer:

"Find bugs"

2 Cents on Ethics

I really had no plan to chime in on the blog conversation between Michael Bolton and Cem Kaner, but after the amount of time I've spent today having email discussions with folks who (apparently) were interested in my 2 cents, I've decided to go ahead and share. I feel it important to point out that as I have spoken to neither of them regarding this conversation, I most certainly don't want to give the impression that I am speaking for either of them.

(As a side note, I'm seriously beginning to wonder if I shouldn't just add a "Notes and Disclaimers" box to my blog... then again, that would be about the same as prefacing all my notes and disclaimers with "Allow me to provide some context" -- which would seem rather redundant coming from me. {grin})

Anyway, it would seem that it all started with Michael's post Why Pass vs. Fail Rates Are Unethical (Test Reporting Part 1) that, if not inspired, certainly contributed to Cem's post Contexts differ: Recognizing the difference between wrong and Wrong which, unsurprisingly, triggered the following post by Michael I Might Be Wrong (But Not For Me)

Ok, all caught up? Good. Lemme share what I think might be happening here and while I'm at it share my model for approaching ethics-related situations in business environments (testing or otherwise).

Context-Driven School (of thought): "I'm not dead yet... I feel happy!"

This is Part III in a series of entries related to the following quote from the "about page" of context-driven-testing.com hosted by Cem Kaner:

"...However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now..."

If you haven't done so already, I recommend starting with:

• Part I: Is Testing Dead? Dunno, but the Context-Driven School Is 
• Part II: With the Context-Driven School "closed" what's next?

---

Ok, so maybe not "happy" but I couldn't resist the Monty Python reference.

James Bach stated on his latest blog update (Context-Driven Testing at a Crossroads):

"I’m the last of the founders of the Context-Driven School, as such, who remain true to the original vision. I will bear its torch along with any fellow travelers who wish to pursue a similar program."

With the Context-Driven School "closed" what's next?

This is Part II in a series of entries related to the following quote from the "about page" of context-driven-testing.com hosted by Cem Kaner:

"...However, over the past 11 years, the founders have gone our separate ways. We have developed distinctly different visions. If there ever was one context-driven school, there is not one now..."

If you haven't done so already, I recommend starting with Part I: Is Testing Dead? Dunno, but the Context-Driven School Is

---

Much like when one completes an educational program at one institution and ponders whether or not to enroll in another program (and if so, which one), or to enter the workforce and continue their learning along the professional development or self-education path, I think it's fair for those who have come to self-identify as members of the Context-Driven School to be asking themselves similar questions.

And much like completing an educational program does not equate to losing the lessons learned (as opposed to the lesson's taught) in the program, the Context-Driven Principles and the lessons many of us have learned by studying in (or, for that matter, rebelling against) the Context-Driven School remain despite Cem's announcement that (in my words) the school is now closed.

10 Things About Testing That Should Die

I've taken some heat for discussing the whole "is test dead" concept due to a feeling that I was validating the concept of testing being unnecessary. Allow me to clarify my position. I do not believe, for one heartbeat, that testing as an activity is in any way unnecessary. I do believe that there are things related to the current state of and common beliefs about testing that should die. With that said...

Scott Barber's Top 10 Things About Testing That Should Die: 

10. Egocentricity

Face reality testers, neither the product nor the business revolve around you. Think about it. No business, no product, no developers => no need for testers. In fact, if developers wrote perfect code, there'd be no need for you. You are a service provider and your primary clients are the managers, developers, and/or executives. Your secondary clients are product users and investors. So stop whining and stomping your feet when your clients don't make decisions you like with the information you provide. It's not your call. If you want it to be your call, get on track to become a project manager, product manager, or executive, otherwise, get right with the fact that you provide a service (hopefully a valuable one) and get back to providing it.

Top 10 Automation Tips from STP Online Summit

I had the pleasure of hosting the second Online Summit, delivered by Software Test Professionals: Achieving Business Value With Test Automation.  The online summit format consists of 3 sessions each for 3 consecutive days.  The sessions for this summit were:

• Software Test Automation, Where's the Value? – Scott Barber
• Achieving Value with Automated Acceptance Tests – Linda Hayes  
• Developer Level Automation Value from "Velcro" to "Velocity"  – Rob Sabourin  
• The Simple and Powerful Build Verification Tests (BVT) – Alan Page  
• Functional Automation is the Gateway Drug to Continuous Delivery – Adam Goucher  
• Test Automation at the API Layer...Why Bother? – BJ Rollison 
• Experiences with Adding Value Through Test Automation – Mark Fewster  
• Test Automation Misconceptions and Money Pits – Hans Buwalda  
• Top Tips for Addressing the Challenge – Scott Barber  
• Speaker Panel – Scott Barber, Linda Hayes, Alan Page, Adam Goucher, Mark Fewster & Guest Panelist Griffin Jones 

One of my duties as host was to try to summarize the most valuable nuggets of information from across all of the presentations into a "top tips" list.  This is what I came up with:

Scott's Top 10 Automation Tips from:

Making Every Test Count

This is from a while back, but I wouldn't call it dated.  It's a webinar, it runs for 48 min.  I like it, for whatever that's worth.  ;)

Abstract:

Do you ever find yourself wondering what the point is to executing this test... again!?!  Have you ever felt like the purpose of a test is to ensure there is a check mark in a particular check box?  Are you ever asked to get *more* information in even less time with even fewer resources than the lst test project you worked on?

In this presentation, Scott Barber will introduce you to a variety of tips and techniques you can apply to virtually any testing you do as you strive to make ever test you execute add value to the project.


--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Scott Barber Interviewed by Matt Heusser; Podcast

Two part podcast on the STP site. I say some interesting stuff... or at least I say some stuff that's interesting to me. :)

Twist #52 - With Scott Barber

Twist #53 - The Return of the Barber
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Uruguay surpasses world with professional development program for software testers.

The Centro de Ensayos de Software (CES), a non-profit software testing laboratory in Uruguay, has recently launched a program that is certain to become the new “gold standard” in professional development for software testers.  The program, endorsed by the Universidad de la Republica (Uruguay), the Universidad Castilla La Mancha (Spain), and sanctioned by the Uruguayan IT Chamber (CUTI), is the most comprehensive, affordable, and publicly available training program for software testers on the market.  Based on my market research and comprehensive review of the program, I have no reservation in rating it as market leading.

Software Testing, the software development activity responsible for identifying issues with software and providing a wide variety of quality-related information to stakeholders and decision-makers prior to release, is the primary job of many millions world-wide, yet the majority of software testers learn their craft entirely on the job.  Yes, there are various “take a class or two, pass an information-based (not a skill-based) test, and receive a certification” programs – some more respectable than others and most far more expensive than the CES program.  There is even a new certificate coming to market that involves three, one month, on-line courses where students are taught and assessed by experienced testers and university professors, but none of those rise to the level of the CES’s program.

Latest Column -- The controversy surrounding the schools of software testing

My latest column...

Periodically, discussions break out in various software testing communities around the Web regarding the schools of software testing.

As I write this, there are discussions going in SQAForums, on the Software-Testing Yahoo! group, and various blogs that (at least up to the time I started writing this piece) reside on or are fed to Testing Reflections. In principle, I'm always pleased when these discussions break out. The point of identifying the schools in the first place was to increase the overall awareness of the diversity in ideologies, practices, and values (i.e. schools of thought) in our field and to stimulate discussion about the situational pros and cons of each. That said, the discussions that actually take place tend to drift off in one or more directions that end up being disappointing, unnecessarily confrontational, and generally not useful.

After witnessing this pattern, participating in these recent discussions, and listening to comments from those who followed the discussions for several years, I've identified several areas in which these discussions go awry. Below, I call those out and share my thoughts about each.

Read the rest of the column.
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Gentleman, Start Your Engines!!!

My most recent column, inspired by a surprise trip to the Brickyard 400, has just been posted on TechTarget in which I discuss the distinction between "delivery" and "done" when it comes to testing the performance of software systems.

Countless hours of development are now in the past. Testing indicates that everything is ready for the big day. The whole team is on hand, and the world is watching. It's the moment of truth; time to find out if all of the hard work is going to pay off. Anticipation builds until the command is given…

"Gentlemen, start your engines!"

The cars come to life. They take a few pace laps and at last, the green flag drops. In fewer than 90 seconds the cars are back on the front stretch approaching speeds of 200 mph -- the pinnacle of stock car performance.

This summer I worked on a project in Indianapolis. Usually when I travel to remote client sites I fly home on the weekends, but there was one weekend that I chose to stay. I chose to stay for two reasons. First, the flights for that weekend were insanely expensive and second, I have some friends in Indianapolis whom I'm always happy to have an excuse to visit. As luck would have it, the flights were expensive because that was the weekend of the Brickyard 400, and one of the friends I wanted to spend time with had a spare ticket, which I shamelessly accepted when he offered.

During the pomp and circumstance leading up to the start of the race I realized what a fabulous example the race was of one of my most-quoted sound bites related to performance testing: "Don't confuse delivery with done."

 

See the column for the rest of the story.

 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

How to Ask (and Not Ask) for Free Consulting

James Bach has posted a great blog about how to and how not to ask industry leaders for assistance.

http://www.satisfice.com/blog/archives/70

This rang true with me and my experiences, but some folks seemed to find his perspective to be arrogant or rude. Below I've copied a representative quote and my response.

But the way he handled it, and because I know that James Bach is a very experienced person in answering forum like questions, it looks as if Bach planed it all and maneuvered the poor guy to this corner, maybe to show him how he should behave. The way Bach handled it is IMHO was one of the worse that I have seen. Instead of getting healthy results (the guy understands his mistake, apologizes and learns from it) it looks like Bach did what ever he could to insult the guy in order to get that kind of reaction. I can learn a lot from James Bach but I am not going to take this approach as a good example to learn from. As Linda said, it doe’s him no credit. 

I have to disagree. I admit that I consider Jim to be a close personal friend. I further admit that my first impression of James Bach was that he was a pompous ass. It was only after meeting him that I came to absolutely adore conversing with him for all the reasons that can be taken as "pompous ass" to anyone who approaches him with defensiveness and self-righteousness.

Paint the room heuristic

The other day, my wife asked me if I could finish painting the bedroom before my conference call in 90 minutes. Naturally, I said that I could and like a good husband, I immediately got started. It wasn't until my phone rang that I realized that I hadn't made it in time. Luckily enough, it was no problem to delay the call by 30 minutes.

While I was finishing up, I realized what had happened. When my wife asked me if I could accomplish the painting in a certain amount of time, my thought process was...

1. If I do it now, it will make her happy.
2. If it takes a little too long, the worst that will happen is that she'll be a little grumpy until I finish, but once I'm done she'll be happy.
3. Once I start, no one is actually going to make me stop before I'm finished... I mean, who wants a mostly painted room?!?
4. I completely overlooked the fact that delaying the phone call could be problematic.

Tester thinking...

Say you were given the following requirements...

•   Users shall be able to enter any of nine predefined data objects
•   User interface shall consist of nine blocks of three rows and three columns
•   Each row, column and/or block shall accept only one member of each data object

What am I describing?