The essential mission of testing is always the same; to answer this question…
Are there problems that threaten the on-time, successful completion of our project?
There have been thousands of books written about testing over the years, and I could list over 100 types of testing that are described in those books. But that is not going to help you understand testing, at least not at this point. The most important thing to understand about testing here at UConn is the concept (or concepts) of testing, and how it all relates to QA.
In the last two posts of this series, we have taken a close look at both Quality Assurance (QA) and Quality itself. And we came up with the following definitions as a baseline.
Quality Assurance is an activity that establishes and evaluates the processes that produce products.
Quality is value to some person, at some time, who matters.
The concept is, if the processes are good, the product (the software, project, or integration), will be good.
That is a great concept. The processes however, relate heavily to how and when the testing is performed during the project. Testing is so tightly connected to quality and QA that it fosters a common misconception about best practices.
The misconception is, if we find and adhere to “best practices,” we can assure the quality of the product.
If you have read the What is QA? page on this site, you know that no single group can assure the quality of any product or project. We need everyone involved for that.
I have been studying the Software Testing industry since I started at UConn and I can tell you that testing has gone through a bit of a revolution in the last several years. The way software is being tested and delivered now has changed dramatically. Mostly due to the way companies are now doing business, which includes more online services and more competition. The bottom line is that best practices are not really best practices for all situations, and they will not assure the quality of a project. Let me explain…
A Little History
For years, in an effort to “assure” the quality of software products, QA and testing organizations have emerged and attempted to come up with a set of processes (best practices) for people in QA and testing roles. These processes were heavy in documentation and took a long time to learn and use.
At the same time, a well-known group of experienced software testers got together, and formed their own group. As they compared notes, they realized that testers were spending too much time writing documentation, and not enough time testing. Interestingly, the Agile development methodology came to the same conclusion. See the diagram below for an interesting look at the ups and downs of changing the way the software testing industry works.
The group quickly realized that “best practices” did not work in all situations, which gave rise to the Context-Driven approach to testing. One of the principles of this approach is, “There are no best practices, only good practices in context.”
We have started using this approach for the projects here at UConn. It fits very well with all of the project methodologies we use, including Agile.
See the What is QA? page of this website for more information on the Context-Driven approach to testing.
Note: The Context-Driven approach to testing is the basis for the new up-and-coming testing methodology called, Rapid Software Testing (RST). Look for more posts on Rapid Software Testing in future posts.
This Context-Driven approach to testing features a style of testing known as, Exploratory Testing. However, the consensus now is that all real testing is exploratory, and anything else is just checking. The founders of Context-Driven testing, after much (and often heated) debate; have come up with the following definitions to make a clear distinction between these two terms.
Testing is the process of evaluating a product by learning about it through experiencing, exploring, and experimenting, which includes to some degree: questioning, study, modeling, observation, inference, etc.
This is true testing. An example would be an instance of Exploratory Testing. Where the tester is allowed to investigate and try new things based on the current results of that investigation. There is no script. This is where you will find brand new, hidden bugs, if there are any.
Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.
Examples of checking include automated scripts and sometimes, test cases/scripts. These are following a step-by-step script that will give the same results every time. They are mostly used for regression testing to “check” if anything has changed. The fact that this is often called testing adds to the confusion between these two words.
“The running of these checks – the stuff that the machine does – does not constitute testing,
just as the compiling of a software product – also the stuff that the machine does – does not constitute programming.”
– James Bach
The Testing Mindset
For me, testing is a mindset, not just a role that needs to be performed. Testing isn’t just a step in the project’s development cycle, it should be ingrained in every stage of the project. Every aspect can be “tested,” whether that be requirements, diagrams, integrations, test scenarios, or user documents.
It helps tremendously if you approach testing with the mindset that the system you are testing has hidden defects waiting to be uncovered.
If I am starting work on a project, I am starting to test from the moment I am assigned. This is because I feel that testing is more than just writing test plans, test scenarios, developing automation, or even exploring the product in an exploratory testing session.
There is so much more to QA and testing than what I have described here, but I hope you got the core concepts. It is my intention to offer some testing workshops in the near future in order to show some good techniques for testing. We often enlist other staff members to help us with the User Acceptance Testing (UAT) of a project, so the workshops will help, and they will be available to anyone who wants to attend.
It is my sincere desire that we will all get to the point where we are developing a tester’s mindset and thinking more about testing and QA during our day-to-day work. Almost everyone is using one of the many software products here at UConn.
If you experience something that does not appear to be working, as it should, do not be afraid to investigate the issue to gain a little more information. If it is a real issue, please report it to the appropriate support group. As more of us are thinking about quality and efficiency, we are getting closer to the idea of continuous improvement.
I know there is so much more we can do to make things better here at UConn. Things that have not even been discovered yet. If you are contemplating new projects, please contact us with your ideas and we will be happy to discuss them with you. At the risk of sounding like a broken record, no one person can assure quality. We need everyone involved or we will not achieve the highest level of quality. I look forward to working with you and improving the quality of our projects as we grow closer to that quality mindset
Thanks for reading!
Mick
Related articles:
Introducing QA Part 1: What is QA?
Introducing QA Part 2: What is Quality?
Introducing QA Part 4: How can QA Help You?