COTS (commercial off-the-shelf) products are great. Someone else does all the research, development, design, and manufacturing; we just read the spec sheet and buy the products. What could be better? Beware, however, the curse of COTS.

Several years ago I was working on a project to build a small device that used several COTS components. The device was a basic audio recorder. Some of the COTS in use included AAA batteries and SD memory cards. Both of these parts are available at any big-box retailer and both are inexpensive.

A key requirement from the user was that the audio recorder function for so many hours on a single AAA battery. The goal was to limit the power consumed by the recorder -- a worthy goal. The design team worked hard to meet this goal. They limited the number of electronic components in the recorder and studied the parts available for the components. They selected the parts that used the least power. Other engineers reviewed all the calculations and they were all solid.

The first few recorders were built and tested, and you can guess the result: they ran only half as long on a AAA battery as required. The first suspects were the AAA batteries. The engineers quickly ran tests on AAA batteries. They found various lots of batteries that provided far more ampere-hours than other lots. They grabbed a few batteries from the good lots and put them in the recorders. The recorders still didn't operate as long as required.

Everyone reviewed the design, the components selected, and the workmanship in the recorders. Everything was as it should be.

The next step was to run the recorders using a bench power supply and measure the power consumed by the recorders. Each recorder was consuming about twice as much power as desired and designed. No wonder the batteries weren't lasting as long enough.

Then the engineers examined the SD memory cards. These were all made by the same company and purchased at the same big-box retailer. The engineers built a test rig for the SD memory cards, set each card in the test rig, and operated the cards from a bench power supply. The SD memory cards in question all consumed about the same power, but that was at least twice as much as was given in the manufacturer's spec sheets.

The engineers drove around town buying SD memory cards (all priced about US $19.95) made by different manufacturers at all the different big-box stores they passed. While shopping, they also bought AAA batteries in every store. They returned to the lab with a hundred SD memory cards and several hundred AAA batteries.

They learned what they probably should have predicted. The power consumed by SD memory cards varied widely. The power produced by AAA batteries varied widely, too.

The solution was simple: test all the SD memory cards and use only those that consumed the least amount of power. After all, the SD memory cards cost 20 bucks, so we would toss the ones that consumed too much power.

This is the curse of COTS. COTS products -- like SD memory cards -- do not have labels indicating the power consumed (or storage or read/write speeds or anything else) that was measured at the factory. Only by testing in our own labs can we learn the values of the attributes of COTS products.

A problem is that to the purchaser, the SD memory cards cost $19.99. The cost of the SD memory cards used in the recorders, however, was near $1,000. This cost includes the time to test the power consumption of all the SD memory cards purchased, the cost of building a test rig, the cost of the test equipment used, and all the other overhead costs of running a lab.

COTS are inexpensive to buy at the store. That is their allure. COTS are expensive to test to ensure they meet the parameters you need. That is their curse.

There is little any of us can do to eliminate the curse of COTS. Just call a manufacturer and ask it to send you 100 SD memory cards that meet your specs from the 1,000 or 10,000 they make each day. I doubt they will comply.

We can, however, avoid being fooled by the allure of COTS. We can calculate the expense of testing the COTS parts to find the handful that we need and put this expense (not the $19.95) in our budget. Then, when hit by this testing expense, we will be ready.