Since the start of my career as a tester, I’ve always been obsessed with testers delivering tangible value to the business. I realized ‘redefining software quality’ was my calling. There were quite a few powerful stories that lead to this realization, and one of the most important ones I presented at the QA&TEST 2019 conference.
This was a story about testers taking responsibility and recognizing how important it is to understand the underlying technology of the product. From this event emerged the three core values I feel which can reshape how we think about software quality.
Award ceremony video here.
Presentation key points
Out of scope
The products being very complex, it was impossible to test everything through black box testing. Areas which the testers were unable to understand how they work, or there was no way to test them, were not considered as the tester’s responsibility.
While quality is everyone’s responsibility, but testers do need to make sure all bases are covered. It isn’t necessary that everything must be tested by a tester, but a tester should make sure that ‘someone’ is testing all the areas that should be covered.
Code coverage is not the only measure to check, but certainly is one of the important ones. Unless we are sure all statements and branches of the code have been executed and tested, we cannot be sure of all the hidden risks. With just black box exploratory testing, it’s almost impossible to figure out the code coverage % even is, let alone having a decent code coverage.
In our story here that was the case, almost zero visibility into the code, some functionality had been going untested. Neither was there any effort in figuring out how to improve that coverage.
In pursuit to improve coverage we came with a method to do fault injection / mutation testing. Instead of controlling the input parameters from outside, alter the control flow within the program to reach the desired state.
Without knowing the technical details of how the product works, it’s very difficult to figure out the risk areas and to learn how to test them efficiently. A good understanding of product architecture and what areas are high risk is key.
Testers MUST put in effort to learn the technology. Without it, IMHO one cannot be a good tester.
Testing is more about thinking and collaborating than doing and checking. Learn to think how to test and learn / develop models on how to test. Unfortunately most testers intuitively learn to test well and don’t follow a systematic way to enhance that skill.
Redefining Software Quality
The most valuable learnings for me were the importance of testers to learn technology, enhanced methods to identify risk and ability to provide value in the context business.