Re: is this a pentium, or a bug

• Date: Thu, 7 Mar 1996 08:34:45 PST
• From:
  mark@ukhayq.cv.com (Mark R. Bowyer)
• Content-Md5: hG9JGOJFY6uuBBZuKHjetg==
• Content-Transfer-Encoding: 7bit
• Content-Type: text/plain; charset=us-ascii

> ;10.00/4.0
> => 2.50000000052154
> [used 4 ticks, 0 seconds.]
> ;1.0/2.0
> => 0.500000000521541
> [used 4 ticks, 0 seconds.]

This looks like our old friend the floating point binary rounding error.  Try
to write 1/2 in binary by hand some time and you'll understand the problem...

On my SPARCstation, it goes:

;10.0/4.0
=> 2.5
;1.0/2.0
=> 0.5

so I have to assume it's a problem with the maths library or hardware on your 
system not correcting for this.

Moredhel/TC.

• Prev: is this a pentium, or a bug
• Next: RE: is this a pentium, or a bug
• Index: MOO-cows
• Thread: MOO-cows