When it comes to testing on agile projects it is common practice for agile teams to adopt a "whole team testing" approach where the team itself does its own testing. To accomplish this agile teams will often embed testers in the development team. Programmers will work closely with the testers, often via non-solo development strategies such as pair programming, to pick up their valuable testing skills. The testers will in turn pick up new skills from the programmers, and in effect both groups will move away from being just specialists (testers or programmers) to being what's called generalizing specialists. Whole team testing can be very different from traditional approaches where programmers may do some testing, often unit testing of their own code, and then throw it over the wall to testers and quality assurance (QA) professionals for verification and validation.
My experience is that whole team testing is a very effective strategy, that agile testing and quality strategies in general appear to be far more effective than traditional testing and quality strategies, but that whole team testing isn't the full agile testing picture. At scale, particularly in complex domains, complex technical situations, or in regulatory compliance situations, Disciplined Agile Delivery (DAD) teams will extend whole team testing with a parallel independent test team. Although the development team still does the majority of the testing, the independent test team which is working in parallel to the development team looks for problems which are harder or more difficult for the development team to find and then reports potential defects back to the development team.
The types of testing that the parallel independent test team performs may include:
- Pre-production system integration testing. Does the solution work within your overall organizational ecosystem? Importantly, if this is one of several teams currently developing new solutions, does this team's solution work with what will be in production (including the work in progress of other teams) when they go to release? In mid-to-large organizations the only economical way to do this sort of testing is via an independent, centralized team.
- Usability testing. Although it's possible to do usability testing on the development team, the reality is that usability testing is a specialized skill that few people have (although could pick up via non-solo development). Furthermore, particularly for solutions with many potential users, you may want to invest in a usability testing lab. This is a centralized resource, or an outsourced resource these days, which is shared across many teams.
- Security testing. Security testing is also a specialized skill, albeit one well supported with sophisticated security testing tools such as the Rational Appscan suite which can be included in your continuous integration (CI) strategy. Many organizations will centralize their security testing efforts.
- Exploratory testing. The fundamental goal of exploratory testing is to discover where the solution breaks, as opposed to confirmatory testing which focuses on showing that the solution conforms to the requirements (this is the type of testing the development team typically focuses on). Exploratory testing is also a skill, a good one which everyone should strive to pick up, but exploratory testers are often few in number in many organizations. So, to leverage their skills effectively you may want to have some of them on the independent test team while they mentor others while doing so.
- Non-functional testing. Non-functional requirements have a tendency to fall through the cracks on some development teams. Knowing this the independent test team will often "test to the risk" and focus on non-functional issues.
- And much more. The above points are just exemplars, not an exact list. Please follow some of the links above for greater detail.
I'd like to leave you with several important thoughts:
- The developers still do the majority of the testing. Just because there's an independent test team it doesn't imply that they are the ones doing all the testing. In fact, nothing could be further from the truth. They should be doing the minority of the testing effort, albeit the more difficult forms of it.
- An independent test team will support multiple dev teams. For example, a test team of 5-6 people could support several development teams totalling 70 to 80 people. I typically look for a 15:1 or 20:1 ratio of developers to independent testers, hopefully even higher than that.
- You need to consider better tooling. Although the development team will still be using common agile testing tools such as the xUnit and FIT frameworks the independent test team (ITT) will need more sophisticated tooling. First, the ITT will need to be able to report defects back to the team easily. When the development team is using a Jazz-based tool such as Rational Team Concert (RTC) then this can easily be done using either RTC (the web interface may be sufficient) or another Jazz-enabled product such as Rational Quality Manager (RQM). Second, the ITT will likely need more sophisticated testing tools, such as Rational Appscan for static and dynamic security testing and Rational Performance Tester (RPT) for performance testing (just two of several software quality management tools you should consider).
- Independent testing is economical. Although I listed several tools in my previous point (hey, I do work for a vendor after all) an "unfortunate" implication of my advice (unfortunate for IBM at least) is that you can reduce the number of licenses that you require and still get this critical testing done by centralizing their use.
- It may be a bit more complicated in regulatory environments. In a strict regulatory environment the independent test team may need to repeat, or at least validate, the testing efforts of the development team. In regulatory environments my fundamental advice is always this -- Have practical people, including yourself, read and interpret the regulations. If you leave it to the bureaucrats you'll get a bureaucratic solution.
- This is an important scaling technique. Parallel independent testing, when done in an agile manner, is an important technique which you should consider when scaling agile strategies to meet the uniques needs of the situation that you find yourself in.