by Dwayne Phillips
Regardless of what Excel tells us, significant digits (remember that?) tells us otherwise.
Recall something called significant digits from high school math?
Consider calculating something with two numbers. One number has two digits while the other number has three digits. The answer can only have two digits.
For example, 23 x 123 = 2,829. That’s what Excel tells us and what the calculators tell us. Significant digits tells us the answer is 2800. So what? So, plug this overly precise number into a long chain of calculations and the error grows until one day we wake and wonder how we got it all wrong.
Too nerdy? Perhaps, but sometimes math, and other fundamentals, are just that way.
Always, always, always remember the fundamentals.
0 responses so far ↓
There are no comments yet...Kick things off by filling out the form below.
Leave a Comment