Part 3 in the series on how wisely designed pairwise testing plans can help when faced with a combinatorial explosion (part 1, Combinatorial Explosions Explained (Software Testing Example), part 2, How Many Pairwise Tests Required (In the Face of Combinatorial Explosions)).
Hans Buwalda gave this presentation on Soap Opera Testing at the Hexawise software testing talks during StarEast 2017.
Quote from Hans’ article explaining Soap Opera Testing:
The end users were the people with the most practical knowledge, but they had no IT or QA background. The testers lacked the proper financial background and the day-to-day experience. To solve this, the testers and end users were asked to sit together in small work groups (four to five people each) and come up with stories based on the most extreme examples that had happened, or that could happen in practice. Imagination was invited and exaggeration was welcome. To help the process, I asked the groups to imagine that they were writing soap operas.
Michael Bolton (from the video):
I am so excited by Hans’ presentation… He has touched on this fork, and the fork is, are we going to confirm repeatedly, relentlessly, the same stuff, that we have seen over and over and over again (which I worry sometimes the action word stuff drives us into) or are we going to do really exciting stuff that soap opera is good for. Which is to actually investigate what happens when things are non-routine and things are non-routine far more often than we believe. I love the soap opera concept, I absolutely love it.
Exploratory testing is all about curiosity. If you are not curious you do not find bugs…
Hans also mentions “exploratory test design” where you think about the business (not the User Interface UI).
This idea of planning exploratory testing is often overlooked. It is important to think about the critical business rules and how those ideas should be tested. Planning out areas and concepts that need to be covered during exploratory testing is important. Soap Opera Testing can help with this, as can Hexawise test plans which help you consider interaction effects.
The Hexawise coverage matrix displays pairwise coverage as each test is run in the test plan. The Hexawise created test plans also put the tests in order that provides the most coverage as quickly as possible (so if you have to release before you finish the test plan you have achieved as much coverage as possible with the tests you managed to execute).
Subscribe to the
- RSS feed for the Hexawise.TV blog
- Hexawise YouTube Channel
- RSS feed for main Hexawise software testing blog
Kathleen Poulsen of Fidelity Investments gave a presentation at STAREAST 2017 sharing her experience using Hexawise to improve their software testing performance.
We were able to reduce from over 12,000 test to 600 tests.
Those 12,000 tests provided unknown but less than full pairwise test coverage. Using Hexawise Fidelity generated just 600 test cases that provided complete pairwise testing coverage.
A big gain for Fidelity was that the testing teams using Hexawise were able to compare and coordinate testing coverage. For example, previously the UI and services teams could not effectively communicate test plan coverage and that resulted in a great deal of duplicated testing. Now they can communicate with each other easily and coordinate their testing plans using Hexawise.