In the presentation Dot discuses testautomationpatterns.org; that site provides many useful ideas for software testers.
FYI, the video has a break because video card filled up on the camera that I was using. I switched to my phone to capture the very last little bit of the presentation.
This video shows how well designed test plans can achieve full pairwise testing coverage with a very small number of tests. And as the total number of permutations increase exponentially as more parameters and parameter values are added properly designed test plans can provide full pairwise coverage with very few tests.
This is not possible for a person to manually do but using the right algorithms to create a test plans can provide amazing benefits.
The end users were the people with the most practical knowledge, but they had no IT or QA background. The testers lacked the proper financial background and the day-to-day experience. To solve this, the testers and end users were asked to sit together in small work groups (four to five people each) and come up with stories based on the most extreme examples that had happened, or that could happen in practice. Imagination was invited and exaggeration was welcome. To help the process, I asked the groups to imagine that they were writing soap operas.
Michael Bolton (from the video):
I am so excited by Hans’ presentation… He has touched on this fork, and the fork is, are we going to confirm repeatedly, relentlessly, the same stuff, that we have seen over and over and over again (which I worry sometimes the action word stuff drives us into) or are we going to do really exciting stuff that soap opera is good for. Which is to actually investigate what happens when things are non-routine and things are non-routine far more often than we believe. I love the soap opera concept, I absolutely love it.
Hans:
Exploratory testing is all about curiosity. If you are not curious you do not find bugs…
Hans also mentions “exploratory test design” where you think about the business (not the User Interface UI).
This idea of planning exploratory testing is often overlooked. It is important to think about the critical business rules and how those ideas should be tested. Planning out areas and concepts that need to be covered during exploratory testing is important. Soap Opera Testing can help with this, as can Hexawise test plans which help you consider interaction effects.
It is impractical in many situations for testers to test every possible permutation of software tests.
In this video Justin Hunter, the founder of Hexawise, explains how combinatorial explosions impact software testing. He explains how to quickly calculate the total possible number of permutations exist in simplified systems under test using several examples.
Hexawise sponsored an evening gathering with short talks on software testing by various experts. Kathleen discussed the use of Hexawise at Fidelity Investments, as seen in this video.
We hope you enjoy this video. We plan on added several more from the speakers at the evening sessions.
Kathleen, in the video:
We had more than 20,000 less than 30,000 test cases that were in place. 95% of them were redundant; either with the integration layer below them or themselves because people would come on board for 6 months and they would quit and go somewhere else and someone else would come along and reinvent exactly the same test.
I could see that now. When we went with in Hexawise we could see how many tests it actually took and just so small compared to what we had been testing. The first time I ran the full suite of [new Hexawise designed] tests I went back an hour later and I told the manager “we’re done you can move ahead you’re all right now.” By “move ahead” I meant promote to a new environment. And he said, “what, I thought I would see you again in 2 weeks.
Carrie Puterbaugh’s presentation at the Twin Cities Quality Assurance Association on using Hexawise to improve software testing at her organization (a large bank).
In one example she discusses in the video Carrie’s team used Hexawise to create an optimized test suite and provided that to the software vendor to have them run it prior to delivering the software to her bank. In this example historically they vendor was finding 67% of the defects and Carrie’s bank was finding 33%. Now that the vendor is using the Hexawise test suite the vendor is finding 98.5% of the defects and fixing them prior to delivering the software.
Based on these results her team was able to move staff off of testing this application and onto other testing needs of the organization. They are saving 90% of what they used to spend on QA on this project.
Another project she talked about was a high priority and high risk release that they used Hexawise on and achieved the highest quality software release they have ever had.
It was great…
We were able to go to management and say “we reduced the amount of test cases we ran and we got a better quality application.
How Do You Know You are Executing the Right Tests? This video shows a few tools in Hexawise that help answer this question and share the reasoning behind prioritization decisions in software testing (coverage chart, coverage matrix).
This short video gives a quick overview of the Hexawise software as a service solution and a very high level overview of software testing with a focus on interactions between parameters/factors.
