Contact Us for a Free Trial Subscription Read a Sample Issue

An IDEA to Satisfy People

by Dwayne Phillips

The objective of testing is to satisfy people. Maybe we should use the word "satisfying" instead of testing. Well-tested software that is free of major defects is more likely to satisfy customers. The paying customers are not the only group of people we should try to satisfy. The people on our project team deserve satisfaction. Well-managed projects with a proper emphasis on testing and adequate resources for testing are satisfying to team members. Another part of satisfying our colleagues is the way we act towards one another during the project.

My desire to satisfy people leads me to three principles about testing. These are (1) test everything, (2) test everyday, and (3) test things not people. I have found that using these principles leads to better products and more enjoyable projects. This is because the people involved are more satisfied.

An Idea for Projects

The accompanying figure shows one view of a project. The circles represent activities that produce things. The stick figures next to the circles show which people perform the activities. The items between the circles represent the products the people produce in the tasks. The final product is in the upper right corner of the figure while all others are intermediate products (they stay with the project team). This does look like a waterfall, but this paper is not advocating the waterfall. Most projects have most of these tasks and products. They may not be in the sequence shown, but they are present. For example, most projects produce a set of requirements. Some use a Military Standard Software Requirements Specification while others put user's stories on 3x5 cards.

The figure shows many roles performed by people. Large projects in large organizations may have specific groups of people to fill these roles. They have full-time analysts, full-time designers, full-time testers, etc. Most organizations today designate a small group of people to work the full duration of a project. In those cases, a person may fill one role one week and another role the next. Some projects may have a person filling two or three roles in one day and repeating these roles the next.

The figure shows many intermediate products. It is possible to test each of these as soon as they are produced. People understand testing software, but not many people understand testing the requirements, high-level design, and everything else. This is possible via Inspection, Demonstration, Execution, and Analysis or IDEA. These methods allow us to test everything and satisfy the consumer of that thing.

Inspection means to read the item under test. One example is reading source code as is commonly done in peer reviews. Inspection also includes reading documents and providing comments to improve the documents.

Demonstration means to use an item. One example is using a word processor to determine if it satisfies the user. Another example is using a payroll system in a simulated environment. The tester is the user.

Execution means to create test software to test an item. This is what most of us associate with the term testing. We write test scripts to test software or write test scripts that run test equipment to test hardware.

Analysis means to check the results that an item produces against known answers. An example is creating a document with words we know are spelled correctly and incorrectly. Spell checking software should label the words spelled incorrectly as such and not label the words spelled correctly.

Test Everything

The first principle of testing I recommend is to test everything. We can and should test every intermediate product shown in the figure. Inspection allows us to test the products on the left side of the figure. We can read the requirements, study them, talk to users, and find defects. Demonstration allows us to test the final product as well as the software system. Users can use the product and comment on their satisfaction. We can write test scripts to run the software units, components, and subsystems on the right side of the figure. This is execution. These scripts drive the pieces of software and help us find defects. Finally, analysis also helps test the software units, components, subsystems, and system. These intermediate products have known answers for known inputs, and we can compare the known answer to that given by the product.

There are two levels of work in testing everything. At the top level, we do test everything. My current project will spend $4M over 18 months and involve two dozen people. Just this week I tested an Integration and Test Plan, a Requirements Verification Test Matrix, and a Trade Analysis using Inspection. Several colleagues are also testing these products. We are finding parts of these documents that have defects and must be corrected.

On the lower level, we don't test every detail of everything. For example, in testing these documents, I read through at about two minutes per page (there is much white space on each page). If I read at one hour per page, that would be in great detail. The same level discussion applies to testing the software. At the top level, we test all the software. At the lower level, we do not test every possible branch of code in every subroutine, we do not test every possible combination of variables input to every subroutine, we do not test every possible data entry error a user could make, etc.

Testing everything helps us satisfy everyone. First, it helps us satisfy our project team. Back to the figure, most of the products in a project go to our own people. The designers use our requirements, the programmers use our designs, and the testers use our software. In my current project, I am using a requirements document prepared by others and working with a contractor to build a product that meets those requirements. The requirements document has low quality. We are trying to use it while we correct it. This is maddening. If only someone had gone through a couple of test-fix cycles before passing it on to me.

Second, testing everything helps us satisfy the users. Defects in the intermediate products pass through to the final product. The user could be stuck with a defect-laden product. Testing everything along way raises the quality of the final product and the satisfaction of the user.

Test Something Everyday

The second principle I recommend is for people in the project to test everyday. While an analyst is recording the requirements, a tester is testing them. While a designer is creating the design, a tester is testing it. My current project is in month two of 15 and we have been testing items since the second week. We will continue to test through the end of the project.

Testing everyday requires a different view of people on a project. Many view people in testing as us and them. The world comprises developers and testers. The testers view the developers are prima donnas, and the developers view the testers as clerks.

I use a human view of people in testing. This is us and us. A tester is a person who did not create a product, so they test it. In the figure, the Designers test the Requirements created by the Analysts, the Programmers test the detailed design created by the Designers, etc. I want someone other than the creator of a product to test it. That extra set of eyes is able to catch mistakes that the creator misses.

A person who is a tester this week may be a designer next week. The third week, they may test a software unit produced by someone else. Each person on the project is a tester and a developer. Which role they fill depends on what day it is. Testers no longer detest developers because they too are developers.

This is not a utopian or socialist view of people on a project. Some people are better than others at testing a product, so they may tend to specialize in testing. This happens more often in a large organization that works large projects. In most organizations, however, smaller projects are the norm and people change roles often. A person who is good at testing requirements may be able to teach analysts how to create requirements with fewer defects, or they may become an expert analyst.

I advocate testing everyday for several reasons. First, we create products everyday. Something is ready for one form of testing by someone else each day. Second, testing everyday helps promote the view of people discussed above. Few organizations are large enough and have enough specialized labor to bring in testers daily. Testing everyday depends on people switching roles and reviewing the work of others. Finally, testing everyday helps detect defects closer to their creation. Defects detected soon after creation are easier and cheaper to correct. The creator still has the product fresh in mind. This familiarity permits rapid recall of the product and usually rapid correction of the defect.

Test Things not People

The final principle of testing that I suggest deals with how people treat one another during all this developing and testing. Work on IT projects is intellectual. People pour their creative energy into their products. When someone finds a defect in a product, the creator can interpret that as a personal attack. Testing and reporting defects may lapse quickly into personal battles where people lose sight of satisfying anyone.

Testing things means finding mistakes and correcting them. Testing people means finding the person who made a mistake and blaming them. People should understand the difference between making a mistake and being a mistake. People are more likely to understand this difference if the project manager creates and maintains a good work environment. What follows are four suggestions for project managers. They deal with how we think of ourselves, how we think of one another, how we will talk to one another, and where we want to go.

How We Think of Ourselves

Early in the project, I gather everyone, ask them to raise their right hands, and repeat the phrase, "I make mistakes."

I like to have people say this publicly because the phrase is true. We all make mistakes, and despite our best intentions we sometimes fail to satisfy others. Some of us have personal rules that say we must be perfect, we cannot make mistakes, and we cannot admit to mistakes. People with those rules should not work on IT projects as they are more apt to slip into blaming wars. Admitting our faults helps create an environment where we shift into defect resolution right after defect detection. There is no need to find the person who made the mistake as that doesn't matter. It was one person today and will probably be someone else tomorrow.

How We Think of One Another

I like to use Norm Kerth's prime directive [1] of project retrospectives. It states:

"Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand."

I keep a large-print copy of this on my desk. I want people who wander through my office and growl in frustration at other people to see it. Maybe they will have a little compassion on their colleagues.

It also stares me in the face every time I want to gripe about something someone else did. I have not met anyone in my office who wants to sabotage projects. I have met many people who make mistakes. I think the vast majority of organizations have a similar situation.

For example, there is a person on my current project who has made many mistakes. My management gave him great authority and freedom from review. No one can test his products. He has made many mistakes and many of us are working hard to correct and work around the effects of those mistakes. We talk about his mistakes often, and I am writing about them now. See how easy it is to violate the directive. See how easy it is to listen to someone who is violating it.

How We Talk to One Another

The words we use when talking to one another have much to do with the work environment. I suggest using words like we and our instead of I, they, them, and their. A healthy project has people saying things like, "We found more errors this week. It seems we made them the last couple of days before the holidays. Our defect tracking system is getting a work out."

An unhealthy project has people saying things like, "I found more of their errors this week. Those developers made a mess before they went on vacation. Some of them have the nerve to claim that my test script was wrong."

Some people may claim, "I don't have time to watch my words and tread lightly. I have work to do here. If he cannot take it, he should get out."

Fear and intimidation work for the short term. I can sprint down stairs if the building is on fire. Running over people may work for a week, a month, maybe even several months. It fails in the long term. I suggest an approach that considers the feelings of people, including your own, to create an environment where people can produce for three months and longer.

Few organizations have extra people who can walk in and contribute fully in the first five minutes. How many people can you chase off your project? How many people can you demoralize on your project?

Where We Want to Go

I want everyone in the project to answer the question, "What do we want to happen on this project?"

Notice the "we" in the question. Another good question is, "What do you as an individual want to happen on this project?"

It may be best for the project if the answer to those two questions are the same. That may not be the case, and it doesn't have to be. I hope that the answer to the first question is something like, "Satisfy all people involved (customers as well as ourselves)."

If we can agree to that, we can work towards it. We can deal with the frustrations, fatigue, and other bothers of each day a little easier. Testing everything everyday can be trying on everyone. We can accomplish those goals if we keep the emphasis on testing things and not people.

Conclusion:

Most IT projects are difficult (it seems that someone else has already done all the easy ones). Testing plays a large role in helping us through the difficulties. I suggest viewing testing as a way of satisfying everyone involved in the project. This includes the development team as well as the customer.

Some principles I have found to help satisfy people include (1) test everything, (2) test everyday, and (3) test things not people. Key to these principles are IDEA (Inspection, Demonstration, Execution, and Analysis), viewing people as us and us vice us and them, and treating one another well.

Most readers are in the middle of at least one difficult project right now. I suggest two things that you can do in the next five minutes. First, place a copy of Norm Kerth's prime directive on your desk, and second practice using the word we instead of I. Your technical problems won't disappear, but you may find what I have found: everyone, especially me, has more energy to work on those problems if we stop working on other people.

References:

[1] "Project Retrospectives," Norm Kerth, Dorset House Publishing, 2000.

Dwayne Phillips has worked as a software and systems engineer with the US government since 1980. He helps people manage software projects and has found that simple metrics do help project managers know where they are and where to shift resources in projects. He has written The Software Project Manager's Handbook: Principles that Work at Work (IEEE Computer Society, 1998). He has a Ph.D. in electrical and computer engineering from Louisiana State University.

Dr. Phillips can be reached at 2315 Ballycairne Court, Reston, VA 20191-1633, USA. Tel: +1 703 476 1951; E-mail: d.phillips@"computer.org.