- My Testing Metrics Allergy – Dawn Haynes
- The Math Behind the Lies – Doug Hoffman
- Change the Questions for More Valuable Answers – Scott Barber
- Controversies in Quality Metrics – Yvette Francino
- Telling The Testing Story – Fiona Charles
- More "T" and Less "BS" – Jon Bach
- A New World: Metrics That Really Count – Harry Robinson
- Risk Metrics & Reporting – Julie Gardiner
- Top Take Aways – Scott Barber
- Speaker Panel – Scott Barber, Fiona Charles, Jon Bach, Doug Hoffman, Harry Robinson, Yvette Francino
Scott's Top 10 Take Aways from:
- Not every take away is specific to metrics
- Several take aways are points made by more than one presenter
- Like every good “Top 10 List”, take aways are presented in reverse order (according to me)
- I’ve paraphrased many of the take aways to make them “quippy”:)
- If you are using numbers to help you tell the story, make sure they are helpful, not harmful
- Pri 2 issue does not imply ½ as important as Pri 1 issues
- It might be fun to find the standard deviation, of “defect fairy” interactions per Priority over Severity by release cycle, but it’s not mathematically valid!
- Apply “Significant Digits” rules
- Priority 1.03579 might *seem* cool, but it’s kinda like reporting 1.68 children/household when asked how many children live in *this* home
- Seeing what you expect to see
- Workers do what managers check
- Its not the numbers that need to be optimized, its the item/activity
- Don’t try to win the game, try to avoid the game
- T, B, S is a great example… the numbers are interesting, the conversation is insightful!
- Specifically the question they were designed to answer
- If you don’t know what question a metric is going to be used to answer, find out
- Those questions often relate to product risk
- Dashboards that are customized by project, culture, goals, etc. are best
- The only way to know how well a particular dashboard will work is to test it!
- They all seem to think that different ones add value
- Some recommend the same metrics as indicators of *crazy-different* things
- Unless you have one of those experts consulting on site, your team knows the value better than the expert’s book does
- The #1 story managers and executives want to hear is the risk story
- Numbers are a medium
- So are traffic lights
- And smiley/sad faces
Chief Technologist, PerfTestPlus, Inc.
Director, Computer Measurement Group
Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing
"If you can see it in your mind...
you will find it in your life."