Friday, March 9, 2012

Context-Driven Testing Crossroads: Addendum

I guess I wasn't as done talking about this as I thought. Earlier today, I posted the following comment (except with a few extra typos that I chose to fix below) on Tim Western's blog in response to his post Is the Context Driven School of Testing - Dead?:
"A point that I think many miss is that this is not just about individual testers.

50 years ago (more or less) testING began fighting a rather arduous battle to establish an identity separate from developMENT. This, eventually, led to testERS establishing an identity separate from developERS.

TestERS continue to fight battles... battles related to achieving organizational respect, related to gaining the freedom to do good testing, related to not being supplanted by buzz grabbing rhetoric about testERS not being needed in Agile because the testING can be accomplished by develpERS at the unit level & users at the system level.

One reason these battles continue is because testING has no unifying community or identity. There is no professional society, there is no educational commonality, there simply isn't any unity. So every time I see the community splinter yet further, I find myself wondering how much longer can the non-unity continue before neither testERS nor testING have a recognized identity, and further wondering what impact that would have on all the individuals in the future who find themselves in a position where they are being asked to test software, but without having any idea where to go to learn.

This happens today. There is an organization called the International Institute for  Business Analysts. About 3 years ago, I was talking to a VP in their org that I ran into at a conference and he said to me, "We've got a bunch of members asking us about software testing, and we don't know what to tell them. Is there a professional society like ours I can point them to?" Of course, I mentioned AST, but the reality is that IIBA is getting these questions about testing because in many Financial Services companies *all* testers of software are called Business Analysts... and most of them don't even realize that what they are doing day in and day out is software testing. They don't know there are books & blogs & conferences & training courses, and they don't understand why *the* professional society for Business Analysts doesn't talk about this thing that they spend most of their time doing. The answer, of course, is simple. It's because the Financial Services industry believes that only individuals with deep knowledge of financial services are capable of understanding (and thus testing) financial software & they further believe that "If someone is smart enough to understand financial software, they are smart enough to figure out how to tell us if it's working right."

Yes, that's a direct quote from a Director who oversees over 100 Business Analysts who's primary task every day is to test software... and when I told him that "tell us if it's working right" is an entire specialty in it's own right called software testing he said...

"Really? I thought that was just something people did when told to. You mean people really study that?!?"

Draw your own conclusions. Decide that I'm "putting the whole world on the end of a pin." That's fine... maybe you're right. Maybe I am. But I've been watching this closely for a long time & I'm *really* good at identifying patterns in data (I am a performance test guy, after all), and I see a worrisome pattern. That doesn't mean that more time or more data won't change that pattern, but this CDT-School thing is one more data point consistent with the pattern."
After receiving several comments via twitter & email encouraging me to make this a blog post of it's own, and giving it some more thought, I decided to comply (instead of doing a client status report, and preparing a slide deck due before I go to bed, and, and, and).

Throughout the day, I've been asking folks in my current client's office, an technology vendor account manager (that bought breakfast for me and "the new test manager" for my client who has picked up responsibility for the performance team I've been working with), a "talent scout" (yes, I believe that is her official title) for an IT staffing agency, the girl at the front desk of the hotel, some guy who wanted to talk at me in an elevator, and can't remember who all else...

"What jobs can you think of with the word 'test' in the title?"

The list below is a complete and comprehensive list of every single answer I received that wasn't a variant of "Software Tester".  Take a deep breath... here we go...
  • Test Pilot
  • Test Administrator (i.e. person who proctors exams)
  • Crash Test Dummy
Whew! I bet you thought you'd never get through that whole list, did you?!? (please note the sarcasm dripping from those words).

Thinking I might be the victim of some kind of sampling bias, I turned to my favorite search engine & every job board I could think of to see what I'd missed. I came up with:
  • Test Technician (mostly for circuit boards requiring a H.S. diploma & no experience)
  • Test Engineer Frictional Material (and a few other similarly obscure science/engineering titles)
  • Experimental & Ground Test Stress Engineer (sounded cool, but *totally* obscure)
  • Manager Economic Capital & Stress Test Analytics, ERPM (I read the job description & I think the only word I understood was "financial")
  • Exceptional Electronics Test Engineer (which read like "Test Technician", but clearly for someone who likes a cool-sounding job)
In all I managed to find "8000+" job postings with "*test*" in the title. Of those, I filtered out ~500 that were associated with "electronics", "electrical", or "circuit". Filtering out "software" and "development" got me down to under 1000. After manually screening 100 of those that remained, and finding 68 variants of software related testing (i.e. Security Test Lead), 11 variants on electronics (i.e. Avionics Test Tech), 3 Test Lab Technicians (what my brother who hand blows custom scientific glass equipment would call "beaker washers"), 5 academic test (i.e. exam) related jobs, the middle three bullets above, and 10 very odd groupings of words that seemed to mostly refer to manufacturing processes, I surrendered.

I'm willing to say, without any fear of being proven dramatically wrong, that over 75% of job postings in English that include "*test*" in the title are directly related to Software -- and most of the rest deal with electronics, materials, or manufacturing.

And yet we wonder why corporate leaders (in non-software product companies), our parents, our neighbors, and the people who work in the next department over have no idea what we do or what makes it "special"?!? Clearly, it's not reasonable to expect anyone other us and the people we've educated, to understand why we make such a big deal about software testing having an identity independent from "any other IT geek that does stuff with computers that I'll never understand."

So here's what's been bouncing around in my head like a 20lb blob of flubber keeping me from getting my "before folks get to work in the morning deadline" tasks accomplished...

If those who specialized in testing software before us fought so hard to gain an identity, and clearly succeeded to some degree (if my experience & impromptu, non-scientific, no-where-near 3 sigma "research" is any indication), and there are clearly a lot of people passionate about doing it better as individuals and as a craft (field, industry, profession... pick your own poison... you get my drift), then

There MUST be SOMETHING distinctive, defining, differentiating, unifying and relevant to people outside of TesterLand that at least a simple majority of the folks in TesterLand can AGREE on, right?

If that's true, why is it that we haven't seemed to identify that SOMETHING in some useful and unifying way?

And if that's NOT true, is there actually anything behind our desire to have an identity beyond the desire to have an identity?

Don't get me wrong, I do believe there are things that make testing unique that are seriously similar in the overwhelming propensity of projects that include testers. I just have wonder if it isn't the same traits that make us good testers that also make us so acutely aware of the hundred subtle, yet crucial to our work, ways in which "my context is different from your context" that we can't seem to realize that pretty much all the non-testers on the planet would see the dozen-or-so obvious ways in which what we do (or would do if we were enabled, or could do if we got more folks trained, or, or, or...) has some non-shallow commonalities.

And if THAT is true, when is someone going swoop in, wearing some fabulous super-hero garb, that we testers *trust*, to tell us what those dozen-or-so commonalities are, so we can get past all this squabbling over schools, titles, affiliations, cliques & clubs and get on with learning how to do our jobs better, how to better educate future generations of testers, how to teach our teams & managers how to best leverage our skills toward achieving project & business success and maybe even help folks make some higher quality software along the way?

{deep breath}

"This has been an unpaid public service announcement. Now, back to your regularly scheduled previous-commitment-driven all-nighter."

Scott Barber
Chief Technologist, PerfTestPlus, Inc.
Director, Computer Measurement Group

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."


James Marcus Bach said...

I am a tester. I light the way. I penetrate illusions. That's what I do. That's what all good testers do. We are applied Epistemologists. We practice all the skills of science.

There you go. Some people comprehend this. Most don't.

Ben Kelly said...

I don't think it's the differences in our contexts that causes fundamental disagreement between software testers. Testers who actually know what they are doing are able to agree on the fundamentals of what software testing is.

The biggest reason I see for disunity in testing is that there is a mass of people who call themselves testers who fundamentally misunderstand what it is to be a software tester because they cleave to the horribly broken, Taylorist 'software development is mass production' metaphor. More to the point, the people that hire and manage programmers and testers believe in this broken metaphor because it is largely in their (immediate) best interests to do so.

In this metaphor, Software Testing is transposed into 'QA' in which we are seen as periodically wandering over to the production line, picking up a widget and inspecting it and declaring that 'quality' is thumbs up or thumbs down.

People who truly get software testing understand how detrimental this model is to software development in general, but it's a simple model to grasp and what we have as an alternative largely is not. The broken model is attractive because it looks like an easily controlled process. The alternative is not. People that turn out widgets are cheaper to pay than craftsmen doing sapient work. When you look at these things, it's no real surprise that people who wrangle programmers and testers are unwilling to let it go.

Until that happens though, people new to testing will be indoctrinated into this broken model and become cheap meatbot automatons like those that indoctrinated them. They won't thank you for telling them they're doing it wrong. Moreover, zombie testers are being churned out faster than we could hope to rehabilitate them.

If testers as a collective want to earn the respect and recognition of non-testers, then we need to educate them about what testing is in a way that matters to them and in a way that helps them shift away from this busted metaphor they cling to now. Anything else is bailing out the pool on the Titanic.

Anonymous said...

What's wrong with "QA" and "QC"? Works for me.

Unknown said...

I typically don't approve anonymous comments, but I think the previous one makes a point... That point being that anonymous is *so* convicted to their love of title that they want to ensure that they never have to explain or defend their position.

Seriously folks?!? Do your homework and be able & willing to support your positions/opinions or do everyone a favor and keep them to yourself. All you are doing is driving polarization and mistrust inside a group that ought to be pulling together over our common ground.

And, for the record, I kinda hope anonymous comes back with a name & an supporting argument, 'cause I welcome the opportunity explain all the ways (again -- I've done so in my writing before) that QA or QC would be even worse for our collective identity & credibility than "trained monkey that bangs on keys and whines about it" would be.

Just sayin'