Wednesday, July 20, 2011

CloudTest Lite - A Game Changer in the Performance Tool Market

Yesterday, SOASTA announced their new product, CloudTest Lite (Press Release). It's not common that I get excited about a tool product release, but this is different. This product has the potential to change the market for the better.
Scratch that. I'll be shocked if it doesn't change the market for the better.

Why is that, you ask? Consider the following attributes of CloutTest Lite:
  • It's a fully featured, easy to learn and use, enterprise class, modern, performance testing tool for web & mobile applications
  • All you need to use it is a reasonably modern machine connected to the internet and a web browser.
    • You don't need to buy, install, configure or maintain load generation machines.
    • The "license" is tied to your personal credentials, so you can design, create, execute, and analyze your tests from any machine you want without needing to figure out how to point to the license server, or how to get onto the corporate network from your favorite internet cafe.
    • You can even do much of the design, test enhancement, and analysis entirely off-line.
  • You can simulate up to 100 virtual users any time you want. No more scheduling time on the controller days or weeks in advance guessing the app will be ready for your test. No more having to wait until your next scheduled time to re-run your test when you see something 'wonky' in your data.
  • It's free.
    • Yes, I said free.
    • As in, you never need to pay a dime. Not today, not when the trial expires, not a year from now to continue your maintenance contract.
    • That's right, it is free from now until the sun explodes (or at least until well beyond when anything we're building or planning to build today is long gone and forgotten)
Imagine the implications:

Tuesday, June 7, 2011

Uruguay surpasses world with professional development program for software testers.

The Centro de Ensayos de Software (CES), a non-profit software testing laboratory in Uruguay, has recently launched a program that is certain to become the new “gold standard” in professional development for software testers.  The program, endorsed by the Universidad de la Republica (Uruguay), the Universidad Castilla La Mancha (Spain), and sanctioned by the Uruguayan IT Chamber (CUTI), is the most comprehensive, affordable, and publicly available training program for software testers on the market.  Based on my market research and comprehensive review of the program, I have no reservation in rating it as market leading.

Software Testing, the software development activity responsible for identifying issues with software and providing a wide variety of quality-related information to stakeholders and decision-makers prior to release, is the primary job of many millions world-wide, yet the majority of software testers learn their craft entirely on the job.  Yes, there are various “take a class or two, pass an information-based (not a skill-based) test, and receive a certification” programs – some more respectable than others and most far more expensive than the CES program.  There is even a new certificate coming to market that involves three, one month, on-line courses where students are taught and assessed by experienced testers and university professors, but none of those rise to the level of the CES’s program.

Monday, April 11, 2011

What being a Context-Driven Tester means to me

I guess it’s that time again.  What time is that, you ask?  It’s the time when discussion/debate flares up over Context-Driven. I’m not going to weigh in on the whole discussion of pros/cons, value/distraction, etc.  I am a consultant.  I am Context-Driven (and not just as a tester, it's simply the way I have operated since long before I was a tester and long before I became aware someone had coined a term and composed a set of principles around how I already operated).  The license plate on my car says “CONTEXT”. It works for me.  But my point isn’t to convince you that it’s right for you.  My point is to address a comment that I frequently hear that *feels* very sad to me.

Where I work, I don’t have the freedom or authority to implement all this Context-Driven stuff, so I guess I don’t get to be part of the club.
I find this sad, because I don’t agree.  It is my opinion that “Where I work, I don’t have the freedom or authority…” *is* a "driving context", making smart decisions about what you are empowered to choose, and appropriately trying to inform/educate those who are "driving your context" that there are other options qualifies as being Context-Driven... at least to me.

What follows is something I drafted for an org that had recently decided that it wanted to adopt the principles of being Context-Driven, but didn’t want to inadvertently offend members whose context was largely dictated by decisions outside of their sphere of influence.  Due to a wide variety of unrelated circumstances, what I wrote never got presented to the org & got lost and forgotten on my hard drive.  I recently found it and wanted to share it with everyone because I think it’s valuable.

Wednesday, May 12, 2010

Testing the Limits... Interviews by uTest

uTest is doing a series of interviews with interesting/well-known testers. I was flattered to be asked to be interviewed recently and found that I enjoyed answering the questions... and didn't hate my answers when I read them as posted (any of you who have ever done that sort of thing know that is a fairly significant statement!)

My interview is "Testing the Limits with Scott Barber": Part I, Part II, Part III

Other folks interviewed include James Bach, Jon Bach, Michael Bolton, Matt Heusser, and Rosie Sherry. If you've got the time, I think they are worth the read.
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Wednesday, September 23, 2009

Testing vs. Checking ... my 2 cents.

I was pleased to see Michael Bolton's series on Testing vs. Checking. If you haven't been following, what I consider to be the central thread of the topic (and the unfortunately inevitable fallout that seems to happen in "testerland" almost any time someone says something that makes sense).
From Michael:
From James Bach:
From Scott Barber:

Monday, September 21, 2009

Thorkil Sonne: Recruit Autistics

Wired.com ran their smart list today. If you aren't familiar with it or don't care, at least check out the great press fellow software tester, entrepreneur, and social innovator Thorkil Sonne is getting for Specialisterne here:

http://www.wired.com/techbiz/people/magazine/17-10/ff_smartlist_sonne

While you're at it, why not digg it!

I know there is a lot that we testers disagree about, but if there is one thing we should be able to agree upon, it's that Thorkil, Specialisterne, and the very special people they serve deserve our support and best wishes. I can only hope that this is the spark that get's this (and other such, responsible programs) moving globally. While that would certainly make me happy for Thorkil, the real winners when this takes off will be those who would finally find themselves filling jobs well suited to their skills, those who are reluctantly (and often poorly) doing those jobs now, and their employers who can reassign those reluctant folks to something *they* are better suited for and will complain about less (we all hope) .

Congratulations Thorkil & Specialisterne!!!
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Saturday, January 3, 2009

A misleading benchmark...

No further commentary needed.

Dilbert.com
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Saturday, December 20, 2008

Thoughts on Performance Testing w/o "Tools"

I was recently asked the following question via the "Ask The Expert" feature of SearchSoftwareQuality.com.
How can we conduct performance testing, stress testing, and load testing of a Web application manually without using any tools?
My commentary is reproduced below -- you'll have to click through to see my actual recommendations.
First, I want to express my sympathy to anyone who finds himself in a position of being asked to create multi-user simulations (i.e. the load part of performance/load/stress testing), requesting a load-generation tool, and being denied. In my experience, the excuse of not being able to afford a load-generation tool is almost always just that -- an excuse.

Only once in my career have I found the only tool capable of generating a production-like load against the application I was testing to be prohibitively expensive. (In that case, the tool cost more to purchase than the application was anticipated to earn in a year.) In every other case, either the risk justified the cost or an adequate, inexpensive, or free tool has been available.

The only really good reason to be in this position is if all available tools were considered and none of them supported the application under test (even with customizations and extensions) and you don't have access to the skills (internally or externally) to build a tool of your own -– and that seems unlikely to me.

The simple truth is that if a company is building an application that is realistically expected to have enough users to justify the expense of performance testing, even if that expense is just the time of an employee, that company ought to be projecting enough revenue from the application, or lose enough credibility by having a poorly performing application, to justify either the cost of a tool or the risk (in some company's eyes) of using a free or open source tool.

Now, after saying all of that, I must admit I have found that the vast majority of value that is gained by quality performance testing comes outside of the load-generation tool. Some of my favorite techniques (assuming you are testing websites):
Read the rest of the article here.
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Friday, December 5, 2008

Latest Column -- The controversy surrounding the schools of software testing

My latest column...

Periodically, discussions break out in various software testing communities around the Web regarding the schools of software testing.

As I write this, there are discussions going in SQAForums, on the Software-Testing Yahoo! group, and various blogs that (at least up to the time I started writing this piece) reside on or are fed to Testing Reflections. In principle, I'm always pleased when these discussions break out. The point of identifying the schools in the first place was to increase the overall awareness of the diversity in ideologies, practices, and values (i.e. schools of thought) in our field and to stimulate discussion about the situational pros and cons of each. That said, the discussions that actually take place tend to drift off in one or more directions that end up being disappointing, unnecessarily confrontational, and generally not useful.

After witnessing this pattern, participating in these recent discussions, and listening to comments from those who followed the discussions for several years, I've identified several areas in which these discussions go awry. Below, I call those out and share my thoughts about each.

Read the rest of the column.
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Friday, November 7, 2008

Latest Column -- Testing training: Disturbing behaviors of students

My latest column...

Drive-by training. Never heard of it? It is exactly what it sounds like. You drive to a training facility (or an instructor drives to you), for a day or three the instructor delivers the pre-packaged training class, then everyone drives back home. It's not the best training model ever invented. There is generally no student assessment, and the only instructor/course provider accountability is reputation. Even so, many good ideas can be shared and lots of students come away feeling that it was well worth "the drive."

As it turns out, I've been delivering a lot of drive-by training to software testers this fall. That in itself isn't particularly noteworthy -- end-of-the-budget year is a popular time for drive-by training -- but something that is noteworthy is that I have noticed a rise in some disturbing behaviors among the individuals and organizations that select and attend drive-by training.
At first, I thought it was just me. But after an informal poll (and some lively discussions) with my employees and trainer friends in the testing realm, I became increasingly convinced that the behaviors I'm noticing are not exclusive to me and that I'm not the only one who thinks they are on the rise.

Read the rest of the column.
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Sunday, October 5, 2008

Latest Column -- Software Testers are not helpless

My latest column...

During a coffee break at a class the other week, I overheard the following comment from one student to another:

Tester: "This stinks! All of my automated test scripts are broken and I can't seem to get the tool to work now that the developers have enabled Secure Sockets Layer. I'm going to have to work through the weekend."

I know that it's generally considered rude to eavesdrop, and ruder still to comment on a conversation you weren't invited to, but I figured that since I was teaching the class I'd be forgiven. Besides, I simply couldn't help myself.

Read the rest of the column.
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Tuesday, August 19, 2008

Latest Column -- Avoid "Center of the Universe Syndrome"

My latest column cautioning testers not to think they are the center of the development team's universe http://searchsoftwarequality.techtarget.com/tip/0,289483,sid92_gci1325828,00.html
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Wednesday, August 6, 2008

5 Questions with Scott Barber by a Braidy Tester

I recently had the honor of being interviewed by Michael Hunter, a Braidy Tester, for Dr. Dobbs Portal. Check it out: 5 Questions with Scott Barber
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Friday, July 4, 2008

Latest Column -- Inspired by taking AST's Bug Advocacy Class

I recently completed (successfully, I might add) the second of the Association for Software Testing's all online, free to members Black Box Software Testing course. Each of these courses is four weeks in length. I've been involved with this program since years before it became a program, and I am an instructor for the first course in the series, called Foundations. For this course, called Bug Advocacy, I was a student.

Bug Advocacy focuses on the skills and concepts needed to compose high-quality, easily understood, appropriately compelling and well organized defect reports. I know, it sounds pretty boring to me too, but it was anything but boring. These classes are designed so that you watch recorded lectures (in this class the lecturer is Cem Kaner), answer some quiz questions (to make sure you watched the lectures), participate in class discussions, do both individual and group projects (in this class the project centered around evaluating and enhancing unconfirmed OpenOffice bug reports), peer reviewing one another's assignments, and taking a far-from-trivial closed-book essay exam. All in all, I spent about 40 hours participating in the class over the four week period.

This approach isn't just about writing a good bug report, it's about making sure you do the right testing after you find a bug.
There was one idea in particular from the class that I found absolutely brilliant and wanted to share with you. Below is actually a very lightly edited version of my answer to one of the exam questions asking us to describe a six-factor approach to bug reporting that Cem remembers using the mnemonic "RIMGEA." If you are a regular reader of mine, you know that I have a fondness for mnemonic devices, but that's not what I thought was so great about the approach. What I think is brilliant is that this approach isn't just about writing a good bug report, it's also about making sure you do the right testing after you find a bug to enable you to write a good bug report. Take a look -- you'll see what I mean.

Click here to read the rest of the column

Click here for more information about AST's free-for-members, online training
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Saturday, June 28, 2008

Testing Lessons From Civil Engineering

Below is the paper I submitted as a prologue to an experience report, discussion, and (hopefully) additional research that I'm presenting for the first time during CAST08:

Engineers don’t look at the world the same way that testers do.  Engineers look at the world with an eye to solving problems.  Testers look at the world with an eye toward finding problems to solve.  This seems logical.  What is less logical is the fact that engineers, and I’m talking about the kind of engineers that deal with physical objects, seem to be much more sophisticated in their testing than testers.  In fact, most of what I know about testing, I learned as a civil engineering student.  We didn’t call most of it testing.  We didn’t even identify it as anything other than “You really want to get this right.” Maybe Civil Engineers test better than software testers because of the motivations to “get it right”.  Consider what happens when a piece of Civil Engineering, like a bridge fails:

Monday, May 5, 2008

Identity crisis or delusions of grandeur?

In this month's installment of "Peak Performance" I discuss the frequently erroneous and often grandiose titles software testers have on their business cards or in their e-mail SIGs. Identity crisis or delusions of grandeur? 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Monday, December 17, 2007

Performance Testing Guidance for Web Applications book

Some time back, I blogged about a book I’d been significantly contributing to being available as a free .pdf download. (see the entry here)

Well, the book quietly appeared in “dead tree format” (as Stuart Moncrieff put it in his blog post about the book) a couple of weeks ago and I’ve been getting light heartedly scolded by some of my friends and readers for not making a big announcement, so here’s my “big announcement.”

Performance Testing Guidance for Web Applications

by: J.D. Meier, Scott Barber, Carlos Farre, Prashant Bansode, and Dennis Rea is now available on Amazon.

Reviewed by: Alberto Savoia, Ben Simo, Cem Kaner, Chris Loosley, Corey Goldberg, Dawn Haynes, Derek Mead, Karen N. Johnson, Mike Bonar, Pradeep Soundararajan, Richard Leeke, Roland Stens, Ross Collard, Steven Woody, Alan Ridlehoover, Clint Huffman, Edmund Wong, Ken Perilman, Larry Brader, Mark Tomlinson, Paul Williams, Pete Coupland, and Rico Mariani.

The best part is that you can buy the book on Amazon, download the PDF, browse the HTML, or do any combination of the above.

Tuesday, October 23, 2007

From The Web: "Noncertified IT pros earn more..."

Stop the presses! Can it be true? The industry wants effective, qualified, multi-dimensional people who are capable of understanding business drivers & risk mitigation and applying that in a sapient way to their job as opposed to folks who paid someone to teach them how to pass a multiple-choice exam?!? Amazing!
 
Noncertified IT pros earn more than certified counterparts: survey
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Thursday, October 18, 2007

From the Mailbox: Software Development: Art or Science?

Here’s a question that I didn’t realize I had much to say about until I read my own response.
 
The Question:
Software Development: Is it an art or a science? An age old question I know, but what do you think and why?
My Response:

Tuesday, October 16, 2007

From the Mailbox: What makes software "good" or "bad"?

I was asked the question below (lightly edited for anonymity, clarity, and length) today and found it intriguing, so I thought I'd post it here.
 
The Question:
This is an attempt to understand how (and why) users, practitioners, and professionals perceive the difference between a good software product and a bad software product, specifically released software products.
My Response: