Model Workloads for Performance Testing: FIBLOTS

This is the third installment of a currently unknown number of posts about heuristics and mnemonics I find valuable when teaching and conducting performance testing.

 

Other posts about performance testing heuristics and mnemonics are:

 

• Installment 1 - Performance Testing Core Principles: CCD IS EARI
• Installment 2 - Classify Performance Tests: IVECTRAS

 

For years, I have championed the use of production logs to create workload models for performance testing. During the same period, I've been researching and experimenting with methods to quickly create "good enough" workload models without empirical data that increase the value of the performance tests. I recently realized that these two ideas are actually complimentary, not exclusionary, and that with or without empirical usage data from production logs, I do the same thing, I:

 

FIBLOTS.

While the play on words makes this mnemonic particularly memorable, I'm not saying that I just make it up. Rather the acronym represents the following guideword heuristics that have served me well in deciding what to include in my workload models over the years.

 

• Frequent: Common application usage.
• Intensive: i.e. Resource hogging activities.
• Business Critical: Even if these activities are both rare and not risky
• Legal: Stuff that will get you sued or not paid.
• Obvious: Stuff that is likely to earn you bad press
• Technically Risky: New technologies, old technologies, places where it’s failed before, previously under-tested areas
• Stakeholder Mandated: Don’t argue with the boss (too much).

--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

Classify Performance Tests: IVECTRAS

This is the second installment of a currently unknown number of posts about heuristics and mnemonics I find valuable when teaching and conducting performance testing.

 

Other posts about performance testing heuristics and mnemonics are:

 

• Installment 1 - Performance Testing Core Principles: CCD IS EARI
• Installment 3 - Model Workloads for Performance Testing: FIBLOTS

 

I have struggled for over 7 years now with first figuring out and then trying to explain all the different "types" of performance tests. You know the ones:

 

• Performance Test
• Load Test
• Stress Test
• Spike Test
• Endurance Test
• Reliability Test
• Component Test
• Configuration Test
• {insert your favorite word} Test

 

Well, I finally have an alternative.

 

IVECTRAS

IVECTRAS is valuable for classifying performance tests (or test cases if you like that term better) and performance test objectives. Better still, it is easy to map to Criteria, Requirements, Goals, Targets, Thresholds, Milestones, Phases, Project Goals, Risks, Business Requirements, Scripts, Suites, Test Data, etc. Yet even better, you can use it as a heuristic to assist with determining performance testing objectives and performance test design. So what is it?

 

To determine, design or classify a performance test objective or test, ask is this an:

 

INVESTIGATION or VALIDATION

of END-TO-END or COMPONENT

response TIMES and/or RESOURCE consumption

under ANTICIPATED or STRESSFUL conditions

 

For me (and my clients since I came up with this) there is a lot less confusion when one says "We need to INVESTIGATE COMPONENT level RESOURCE consumption for the application server under STRESSFUL conditions" than it is to say "We need to do a unit stress test against the application server." Even if there are still questions to be answered after applying IVECTRAS, at least the questions should be more obvious -- and if nothing else, *that* adds value for me.

--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."