Wireless and Digital Services

Testing – Is it just a phase? (Part 2)

By Chris George - Last updated: Thursday, May 4, 2017

In Part 1, we looked at how testing is not a synonym for quality and that everyone involved in a project must be responsible for quality. In Part 2, we look at how testers can provide greater value in your projects, and how we have lived this at Cambridge Consultants.

How can testers affect quality?

The question might now be asked “why bother testing if everyone else has responsibility for quality, and if testing it doesn’t necessarily improve the quality, while costing me money?

However the risk is that we’re looking at the problem in a very one dimensional manner. The assumption is that testing can only happen on the product, the built software, the constructed hardware, and this is a view that many in the engineering world (both software and hardware) continue to hold. However, a good tester is not good at testing because they know how to test a website or analyse some hardware, they are a good tester because they have the right mind-set. They are enquiring, investigative, questioning, pedantic, observant, creative, analytical, technical, empathic and pragmatic. We’re great at parties.

One of the current buzzwords in the software testing community is to ‘shift-left’ the testing activities; that is to move testing earlier in the life cycle of a product or feature, rather than leaving it to the end of the process. It has long been identified that to test early is to fail fast and make better decisions as a result. One of the negative consequences of the testing phase being at the end of a project is that any major defects that are found to be due to an architectural flaw or an incorrect/ambiguous requirement will be far more expensive to fix. Indeed a proper fix may not even be feasible because of time or budgetary constraints, instead leading us to ‘paper over the cracks’. So the earlier these issues can be found, the better. Returning to Deming’s quote, by the time it’s ready to be inspected, it’s too late. The quality is either already there or it isn’t.

People with testing skills and that get involved in the initial design/requirement sessions can offer new perspectives, ask questions about what the client wants to achieve and define the ‘what-if’ scenarios. Taking things further, they might encourage the use of a Behaviour-Driven Development (BDD) approach. This then leads to a requirements specification, at which point the testers can help to ensure that all requirements are unambiguous, well defined and testable! By becoming involved at this stage, steps can be made to ensure that testability is designed and built into the product from the start. From this process, user scenarios will have been created that will form the backbone of efficient, effective and valuable exploratory testing sessions.

During development, the testers work closely with the software engineers, doing pairing exercises, questioning approaches and testing new code before it even hits the build system. Approaches such as Test-Driven Development (TDD) can be supported by testers coaching the software engineers as they design their test cases.

By getting the right people involved early in a project, a level of quality can be designed and built into the product, such that any testing that happens at the end can be quicker and more efficient, focusing on delivering value where it’s needed.

Here at Cambridge Consultants, an increasing number of our projects are based on software solutions, using cloud infrastructure, service oriented architecture and web technologies. This in turn shifts us towards an iterative, continuous delivery pipeline. To achieve this whilst maintaining the high levels of quality on which our reputation is based, we recognise that ‘building the right thing’, as well as ‘building the thing right’ are supremely important. Traditionally this would be done by defining and designing everything upfront at the start of the project. However, through development methodologies such as Agile, it has been shown that developing in close collaboration with the client, and iterating the design of the product is a far more productive and efficient approach.

George Dinwiddie’s Three Amigos approach to story definition is one that I’ve sometimes used. This typically involves bringing a product owner, developer and tester together to define a story, the associated acceptance criteria and to identify any risks. By having these three different mind-sets and perspectives looking at the same problem, a far better and more considered result is obtained.

A better definition of testing?

In Part 1, I defined testing like this:

“Testing is traditionally defined as the process of validating and verifying that a software program or application or product meets the business and technical requirements that guided its design and development and works as expected.”

At an absolute minimum, this definition encourages the testing after-the-fact approach, and so narrows the scope of the testing role. Even the definitions of testing suggested by community leaders, such as Cem Kaner, James Bach and Michael Bolton, still allude to testing a program or product, whereas Weinberg defines testing as “a process of gathering information about [the thing under test] with the intent that the information could be used for some purpose”. I prefer this definition, which is a little more generic and could apply to the more ‘shift-left’ activities.

At a recent community workshop on testing, James Thomas from Linguamatics considered many definitions of testing, and offered his own as “testing is the pursuit of actual or potential incongruity”. This was intentionally written to be context free, so that it could apply to many situations. This definition is very difficult to argue with as it is so broad, seeking incongruities in anything, from requirements, to software, to hardware, to documentation, the list goes on. I really like this definition – it’s simple yet powerful, but it’s no panacea. Its generality means that it might be interpreted as “testing is the pursuit of bugs in [a product]”, therefore requiring further explanation.

The Future Tester

By treating testing as a range of activities, rather than a phase, we begin to explore the value that people with great testing skills can offer to a project at all stages.

Alberto Savoia, then at Google, infamously introduced the phrase “test is dead” in his fascinating keynote at the 6th Annual Google Test Automation Conference (GTAC) in 2011. Whilst he was being intentionally divisive, Savoia’s description of the shifting mentality towards testing earlier in a project makes much sense. We need to test that we are building the right thing before testing we are building the thing right. To put it another way, there is no point in making a beautiful, reliable and functional product if it’s not what people want.

Testers at Cambridge Consultants are increasingly applying their investigative, critical and pragmatic skills to initial ideas or designs for a project, therefore having a far greater impact on a project than if they’re simply searching for defects later on. Of course testers are not the only people to have such skills, but as testing fundamentally requires these skills, good testers are often well suited to these activities.

In addition to this, by using exploratory testing techniques such as session based test management, we are breaking free from the traditional pre-defined test cases (where appropriate), and spending less time on test admin, and more time testing the product.

Our primary goal is to create a quality product for the client or user in a timely manner and within budget. This is best achieved by increasing quality throughout the project workflow, from inception to delivery and beyond. By increasing quality earlier, testing on a built product can focus on areas that provide real value to the client, such as usability, scenarios, resilience and performance. These are ‘wins’ of a higher order, compared to finding defects arising from poor process and development practices. Not only does this result in a better product, it also makes the role of the tester a more compelling and interesting one.