Let Someone Else Test It

Dwayne Phillips

Revision History
Revision 1.026 September 2000

People who design or code something shouldn't test it. The classic reason is they may not see problems that others can. I've recently stumbled onto another reason, and that reason can save projects time and money.

I have discovered something about people and design and testing while working on a current project (not to say that others haven't discovered this many times before me). The project work involves hardware, but the lessons apply to software and all other systems work.

We are testing circuits that will soon be integrated with other circuits and software. To save time and money, we are using extra engineers to test the circuits. We have observed that the extra engineers can test the circuits faster than the engineers who designed them. This did not make any sense, it seems the originators should be able to test faster than anyone else -- not always better, but faster. The designers should know the circuit so well that they can run tests, interpret results, and move on much faster than a new engineer who must learn the circuit before running tests.

It turns out that the designers took longer to test because they were modifying their designs based on the testing. Testing was the first time their paper designs made it to silicon and copper. The designers saw what happened in practice instead of on paper. What they saw taught them lessons; they applied the lessons on the spot, and they changed the design and the hardware.

The problem with this learning and redesigning is that the circuits were already good enough. The circuits were passing their tests. The designers, however, could see how to make them better, so they made them better. The extra engineers ran the test, noted that the circuits passed, and moved on. They were much more efficient.

Making the circuits better was costing time and money and was not necessary. The designers were polishing their designs. Their gold plating made the product better, but it didn't need to be better.

Some of you reading this are probably coming out of your seats by now. I have baited you with terms like "good enough," "polishing," and "gold plating." These are all subjective terms, and what I call gold plating might be considered quality by others. The circuits under test met specification, so they were good enough. Making them better made them better, but we didn't need better. Yes, there are engineering and emotional justifications for improving quality whenever possible -- but business runs engineering, and business tries to keep costs down.

This testing and polishing applies to software and IT as well as hardware. We all know that someone other than the programmer should test the software. The someone else can see problems the programmer cannot. Now we can add that the someone else is satisfied when the software passes its tests. The test passes, and the tester moves on. The programmer can take the testing as a chance to improve the software, whether it needs improvement or not.

Beware of this when you have programmers or other designers testing their work. Products have quality standards. Exceeding those standards is good if you can afford it. Driving up cost when it's not needed is not good.

Dwayne Phillips has worked as a software and systems engineer with the US government since 1980. He is the author of *The Software Project Manager's Handbook, Principles that Work at Work* (IEEE Computer Society, 1998). He has a Ph.D. in electrical and computer engineering from Louisiana State University. Dr. Phillips can be reached at d.phillips@computer.org.