“Quality is free, but only to those who are willing to pay heavily for it.”
– T. DeMarco and T. Lister
In part 1 of this series, What is QA?, we talked about Quality Assurance (QA) and how it means different things to different people. Other factors for these different meanings are the many definitions surrounding the word “quality” by itself. We all know quality is important, and we all have an idea of what it means. But it’s not that easy to define in terms of a project. For some, it means reliability and efficiency. To others, it means fitness of purpose, or usability. As an example, here are just some of the definitions you can find online.
But these are mostly generalizations. The truth is every single one of us can have a different idea of what quality really is, and still be right, depending on the context. However, I think most people will agree on these two things:
- Quality is subjective. It’s determined by whoever is using the product at the time.
- Quality is relative. It changes over time.
As an example, for those of you who remember back in the early days of the internet, we were (somewhat) happy to wait several minutes for the connection through a dial-up modem, or for a page to load. We could go get a cup of coffee, sit back down, and begin to surf the internet. No problem.
Today, the internet connection is always present, and the page or the app had better load within a couple of seconds or we get very annoyed and may never use that site or that app again.
Our collective expectations of quality and our tolerance of issues have shifted over time.
It’s no wonder that quality is so difficult to define. We all define it differently and we continually change that definition over time.
In his annual letter to shareholders in 2017, Amazon founder, Jeff Bezos, talked about continual customer obsession as the key to Amazon’s growth. He stated that customers are always, “divinely discontent.” Their expectations never remain the same; their needs are forever growing, and their point of view is ever changing. They will always want something better than what they have today.
That is the essence of quality. It’s hard to define because nothing about it is constant. That is also the challenge and the need for QA. It’s true that if we can keep up with the increasing customer expectations, we will obtain a higher quality in our projects (software, services, or integrations). But this can also create a conflict within the project because the growing expectations could result in extending the project’s scope. There is a fine line here. We want to continuously improve the product, but not at the expense of the project.
QA can help in this effort by getting involved early in the requirements discovery activity. The project is installing some software or service in order to solve a problem. The requirements activity will uncover requirements that reveal that problem. Without good requirements, we will not know if we are solving the real problem or someone else’s idea of the problem. Time spent uncovering the essence of the real problem is key.
“If I were given one hour to save the planet, I would spend 59 minutes defining the problem…” – Albert Einstein
While that may sound extreme, it does highlight the importance of defining the problem, or problems, to be solved. We will talk much more about the requirements activity in a future post.
So what is Quality?
In order to define quality as it pertains to projects at UConn, we will look at quality in terms of software projects and integrations. Some of the biggest software testers in the industry have been defining and redefining quality over the last several years. One of the first was Gerald M. Weinberg, an acclaimed author, writer, and consultant. Here is his original definition of quality:
“Quality is value to some person”.
By “value,” he means, what are people willing to pay (or do) to have their requirements met? It is simple and to the point. It’s also broad and begs to be further defined, which it was; by another very well-known tester and consultant, James Bach, who refined it to:
“Quality is value to some person who matters”.
Then, after recognizing how quality changes over time, the definition was further refined by both James and another tester, Michael Bolton, (not the singer) to:
“Quality is value to some person, at some time, who matters.”
So there you have it. A current and useful definition of quality. This just highlights quality’s subjective nature and the need to constantly re-evaluate it. Here is a recent example that shows whose idea of quality matters when deciding on what needs to be corrected.
A Quality Example from UConn
While testing early on in the Concur project, I found what I thought was a bug in Concur’s software. I had selected a certain airport from the drop-down list in the Request Module to complete a Travel Request. When I went to complete the expense report, in the Expense Module, I could not find that same airport in the drop-down. The user interface also had a slightly different look and feel. As I looked further, I saw that these were two completely different lists. Why wouldn’t they use the same drop-down list in both modules to avoid confusion and maintain consistency?
Thinking that this was a quality issue, I reported it to Concur thinking other companies and universities would have the same concern. To my surprise, they were not concerned with the issue. They had two separate software teams, working on these two modules. Not surprising, but what was surprising was that these two teams did not even talk to each other. That explained the different look and feel and the different drop-down lists. Weren’t they concerned about the quality of their product?
It turned out that the reason they were not concerned was that we were not using their product the way most of their users use it. Most of Concur’s customers use only the Request Module or the Expense Module, but not both. We are a minority in the way we are using it. I don’t know the exact numbers, but for this example, let’s say 10% use both modules. Does it make sense for Concur to redirect their development effort so only 10% (or fewer) of their customers benefit from it? And the other 90% will not? Of course, we would like to think so if we are the 10%. But in reality, not if there are real problems still affecting the other 90%.
Clearly, in defining quality for the Concur product, any issues that are reported by the 90% are from the “person who matters.” I would like to think that in time, the Concur teams will come together to create the same look and feel for all of their products.
In any case, this is an important consideration for us when adding any new SaaS product to UConn’s processes. In these situations, we are at the mercy of someone else’s product quality decisions, and we may not always be the “person who matters” in the decision making process.
In the next section, we will take a closer look at testing and how it has evolved over the years.
Thanks for stopping by!
Mick
Related articles:
Introducing QA Part 1: What is QA?
Introducing QA Part 3: What is Testing?
Introducing QA Part 4: How can QA Help You?