Pairwise Testing
Recently we had a discussion about testing all possible combinations within a FX Trade Ticket.
To give a context, a trade ticket consists of several parameters like:
- Trade Type (SPOT, FORWARD, SWAP, TIMEOPTION)
- Currency Pair (BASE, TERM)
- Direction (BUY, SELL)
If we combine all possibilities we will have to execute 16 tests.
TYPE CURRENCY DIRECTION:
- SPOT BASE BUY
- SPOT BASE SELL
- SPOT TERM BUY
- SPOT TERM SELL
- FORWARD BASE BUY
- FORWARD BASE SELL
- FORWARD TERM BUY
- FORWARD TERM SELL
- SWAP BASE BUY
- SWAP BASE SELL
- SWAP TERM BUY
- SWAP TERM SELL
- TIMEOPTION BASE BUY
- TIMEOPTION BASE SELL
- TIMEOPTION TERM BUY
- TIMEOPTION TERM SELL
During this discussion I remembered a Tech Talk I went to last year about Pairwise and Combinatorial Testing and I started to do some more research on it.
What is Pairwise Testing?
Pairwise Testing is a software testing approach that can help to test a big number of input parameters which normally results in hundreds/thousands of different permutations.
A good example to show the immense number of test cases is a configuration dialog with 12 checkboxes (checked, unchecked) and a dropdown with 3 values.
If we wanted to cover all possible combinations we would have to execute 12,288 different tests which is calculated by (2 power n) * 3.
Amazingly these 12,288 tests can be reduced to 10 if we apply the 2-way pairwise testing strategy.
See http://www.testingeducation.org/wtst5/PairwisePNSQC2004.pdf
How does it work?
Pairwise testing is a way of testing a big set of parameters by only testing pairs of the given parameters. If we look at the example above we can see that there are 2 tests for SPOT, BASE (SPOT-BASE-BUY, SPOT-BASE-SELL).
One of the inspirations for Pairwise Testing is the fact that bugs occur more often on a pair or triple of parameters rather than on a single input parameter. Therefore it is quite popular and often used for configuration testing but it can also be applied to any other area of software quality assurance.
This sounds all quite magic. So I would not recommend to blindly apply this strategy without analysing the software under test and getting some more information about architecture and risk before using it.
More information can be found on the following pages:
What I personally find quite interesting is the combination of a pairwise analysis with exploratory testing. By getting a good idea of the input parameters it can definitely support exploring the application.
Finally I would like to show the result of the 2-way Pairwise Testing strategy applied to the example above.
2-WAY TYPE CURRENCY DIRECTION:
- SPOT BASE BUY
- SPOT TERM SELL
- FORWARD BASE SELL
- FORWARD TERM BUY
- SWAP BASE BUY
- SWAP TERM SELL
- TIMEOPTION BASE BUY
- TIMEOPTION TERM SELL
