AcademiaAgriculture & FarmingHumanitiesSocial SciencesSTEM

Rounding Error and Its Consequences

Updated on March 11, 2017
calculus-geometry profile image

TR Smith is a product designer and former teacher who uses math in her work every day.

The round-off error of a number is the difference between its rounded value and its exact value. Some rounding error is unavoidable in solving real-world math problems that involve irrational numbers, for example, finding the diagonal of a rectangle or circumference of a circle.

However, in multi-step processes, rounding errors can propagate and become worse after each step. Small errors can quickly evolve into enormous errors with severe consequences. What seems insignificant at first can lead to final calculations that way off track, sometimes with disastrous results! Here are some examples of round-off error run amok.

Vancouver Stock Exchange

Long before it was absorbed by the Canadian Venture Exchange, the now-defunct Vancouver Stock Exchange (VSE) was the third largest stock exchange in Canada. It was widely believed to trade many fraudulent stocks and outright scams, but surprisingly this was not its biggest problem.

In January 1982, the Vancouver Stock Exchange index was initialized to a nice round value of 1000 points. The average daily trading volume was about 2,800 trades. However, rather than rounding each trade to 3 decimal places as is standard practice, the VSE truncated each trade to 3 decimal points.

For example, if the index stood at 731.84297, the computer would replace it with 731.842 by dropping the last two digits, rather than round it to 731.843. Imagine the effect of doing this over the course of nearly 2,800 trades per day! The average point loss from truncation works out to 0.0005 points per trade or about 1.4 points per day. By consistently truncating instead of rounding, the posted value of the index gradually declined from its true value.

On Friday, November 25, 1983, the index closed at 574.081. When the VSE reopened on Monday, the index was posted as 1098.892. What happened? Over the weekend, the exchange finally decided to correct the index's accumulated error. Over the course of almost two years, the index had erroneously lost over 500 points!

Patriot missile defense system
Patriot missile defense system

Patriot Missile Fails to Intercept Iraqi Scud

One of the most infamous cases of rounding error was when the Patriot missile defense system failed to intercept an Iraqi Scud missile headed toward a base in Saudi Arabia in 1991. Twenty-eight people were killed in the attack.

What happened? The Patriot missile defense system was calibrated to tenths of a second, but the computer used 24-bit point register to store time. In binary, the fraction 1/10 is represented by the repeating decimal 0.00011001100110011... As a repeating binary decimal, it doesn't terminate, so any truncation of this figure results in an underestimate of the true time elapsed. The Patriot missile computer system truncated it to 24 places, resulting in a loss of about 0.003433 seconds per hour. Seems small, right?

Sometime in February the missile defense system was left on for 100 hours, and over that period of time, it accumulated an error of about 0.3433 seconds. A Scud travels about 1676 meters per second, so in 0.3433 seconds it travels about 575 meters, over half a kilometer. This is a huge discrepancy, for when the Patriot missile defense system thought the scud was out of range, it was actually half a kilometer closer. Having miscalculated the scud's position, the Patriot system failed to intercept it before it detonated in the barracks.

Round-Off Error in Simple Geometry Problems

Rounding error can affect your answers in even the simplest geometry problems when irrational numbers are involved.

In the figure on the right, the largest circle has a radius of 100. The medium sized circle has a radius that is 1/sqrt(2) times the radius of the largest circle. The smallest circle has a radius that is 1/sqrt(2) times the medium sized circle. What is the radius of the smallest circle?

Rounding 1/sqrt(2) to the nearest tenth gives you 0.7. If you take this as the "value" of 1/sqrt(2), then the radius of the medium sized circle is 100*0.7 = 70, and the radius of the smallest circle is 70*0.7 = 49.

However, the correct answer is that the smallest circle has a radius of 50, since

100 * (1/sqrt(2)) * (1/sqrt(2))
= 100 / 2
= 50

The round-off error in this problem decreased the computed answer by 1 unit from the true answer.



    0 of 8192 characters used
    Post Comment

    • tirelesstraveler profile image

      Judy Specht 3 years ago from California

      Always wondered why the patriot missiles didn't work. This was fascinating.

    Click to Rate This Article