- This is the first installment of a currently unknown number of posts about heuristics and mnemonics I find valuable when teaching and conducting performance testing.
- Other posts about performance testing heuristics and mnemonics are:
- Installment 2 - Classify Performance Tests: IVECTRAS
- Installment 3 - Model Workloads for Performance Testing: FIBLOTS
- There is not a "one-size-fits-most" approach to performance testing, but I have become rather convinced that there are nine principles that are (almost always) applied (or at least actively considered) in successful performance testing projects. I remember those principles by remembering:
- CCD IS EARI
- Context: Project context is central to successful performance testing.
- Criteria: Business, project, system, & user success criteria.
- Design: Identify system usage, and key metrics; plan and design tests.
- Install: Install and prepare environment, tools, & resource monitors.
- Script: Implement test design using tools.
- Execute: Run and monitor tests. Validate tests, test data, and results.
- Analyze: Analyze the data individually and as a cross-functional team.
- Report: Consolidate and share results, customized by audience.
- Iterate: "Lather, rinse, repeat" as necessary.
- For more see Developing an approach to performance testing -- CCD IS EARI.
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me
Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing
"If you can see it in your mind...
you will find it in your life."
2 comments:
What is the principal difference between 1)performance goals and performance targets 2) between performance requirements and performance thresholds? They sound very close to me. Do we actually need five groups here? For me, first three look enough from the practical point of view. What I am missing?
Regards,
Alex
Requirements/Thresholds & Goals/Targets - Only difference is End-User perspective vs. Component perspective.
So why 5 instead of only 3? It's entirely for teaching purposes. I've found over the years that I've been teaching this that the majority of folks attending a class about performance testing are very heavily biased toward one perspective or the other. Separating the categories seems to help folks recognize their biases and adjust accordingly.
The fact is that once you (or your team) recognize and adjust to whatever bias you may (or may not) have, there is no real reason to have 5 "buckets" instead of 3. The points that matters *most* to me are the following:
- Recognize the difference between negotiable and non-negotiable items (whatever you call them).
- Remember to consider both the system as a whole (i.e. End-User) and the pieces that make up the system.
- Consider that not all of the reasons for conducting performance testing may involve "pass/fail" criteria.
I'm confident that you already get that, Alex, and that this is just reinforcement. Like I say, I'm trying really hard these days to make key concepts easy to both teach and learn for folks who haven't been living and breathing this stuff for (probably too many) years like you and I have.
Thanks for the note - that was a really good question to have answered here!
Post a Comment