Floating point is like that... (was Re: Floating point error)

• Date: Wed, 13 Mar 1996 14:08:43 PST
• From:
  tom@mvision.com (Tom Ritchford)
• Reply-To:
  tom@mvision.com
• In-Reply-To: <1.5.4b11.32.19960313214444.00687304@mailbox.syr.edu> (message from Kipp the Kidd on Wed, 13 Mar 1996 13:44:44 PST)

    If programmers are assuming that they are going to get
"exact" numbers with their floating point calculations,
I fear that they will be sadly disappointed.  It's a really
Bad Idea to test floating point numbers for equality -- you should
always build in a little shrinkage into the tests, or else you
will get a nasty surprise one day.  (ie, write abs( a-b ) < epsilon,
rather than a == b, where epsilon is a small number.  If a and b
can vary greatly in magnitude, try abs( a-b )/b < epsilon ((when
b isn't zero of course!)))

    Donald Knuth discusses this at great length in the Art of Computer
Programming, vol 1.

    /t

Tom Ritchford     tom@mvision.com, tom@weirdos.com

Verge's "Little Idiot" -- Music for the mentally peculiar.
1-800-WEIRDOS           http://www.fly.net/~tom/verge.html

• Re: Floating point is like that... (was Re: Floating point error)
• From: "Robert J. Brown" <rj@eli.wariat.org>

• Re: Floating point error
• From: Kipp the Kidd <mcmudge@mailbox.syr.edu>

• Prev: Re: Floating point error
• Next: Re: Floating point is like that... (was Re: Floating point error)
• Index: MOO-cows
• Thread: MOO-cows