DevX Home Today's Headlines   Articles Archive   Tip Bank   Forums

1. Matthew Guest

Rounding problems

Why does this return false
round(1.015,2) = (round(0.015,2) + 1)

it appears that when the number is less than 1 round behaves differently
0.015 rounds to 0.02 - correct
where as
1.015 rounds to 1.01 - wrong

but to even complicate matters more
0.115 rounds to 0.12 - correct
1.115 rounds to 1.12 - correct
1.015 rounds to 1.01 - wrong
1.555 rounds to 1.56 - correct
1.545 rounds to 1.54 - wrong

Does anybody else have experience with this and can they suggest an alternative.

2. Dan Barclay Guest

Re: Rounding problems

Matthew,

There is more than one issue here. (1) not all decimal numbers lend
themselves to exact representation in binary and (2) there are lots of
ways to round.

Some of the numbers you're showing may not be exactly what you think
(may be a touch under or over the value and not exact "5")

VB uses Bankers Rounding, which avoids bias in the overall. That is,
it rounds, using the normal convention, to the nearest "even". The
last example you show labeled as "wrong" (1.545 rounds to 1.54) is
actually correct.

I didn't go through the entire list, but you may want to look at

http://support.microsoft.com/support.../Q196/6/52.ASP
http://support.microsoft.com/support.../Q194/9/83.ASP
http://support.microsoft.com/support...s/Q42/9/80.ASP

On 14 Aug 2001 19:04:37 -0700, "Matthew"

>
>Why does this return false
>round(1.015,2) = (round(0.015,2) + 1)
>
>it appears that when the number is less than 1 round behaves differently
>0.015 rounds to 0.02 - correct
>where as
>1.015 rounds to 1.01 - wrong
>
>but to even complicate matters more
>0.115 rounds to 0.12 - correct
>1.115 rounds to 1.12 - correct
>1.015 rounds to 1.01 - wrong
>1.555 rounds to 1.56 - correct
>1.545 rounds to 1.54 - wrong

>
>Does anybody else have experience with this and can they suggest an alternative.
>
>

Language Stability is a *feature* I wish VB had!
(#6)

3. Matthew Guest

Re: Rounding problems

Hi Dan,

understanding the difference between bankers rounding and real rounding has

round() uses bankers rounding whereas format(#,"0.00") uses correct mathematical
rounding.

Bankers rounding rounds to the nearest even number eg

round(1.155,2) = 1.16
round(1.165,2) = 1.16

Mathmatically this is incorrect, there is a full cent difference

The format command rounds mathematically

Format(1.155,"0.00") = 1.16
Format(1.165,"0.00") = 1.17

I've heard of bankers hours, Bank fees, now I can add a new one, Bank Rounding,
I know it is designed to even out rounding, but somehow I bet the scales
are tilted in their favour.

Cheers,

Matt

Dan Barclay <Dan@MVPs.org> wrote:
>Matthew,
>
>There is more than one issue here. (1) not all decimal numbers lend
>themselves to exact representation in binary and (2) there are lots of
>ways to round.
>
>Some of the numbers you're showing may not be exactly what you think
>(may be a touch under or over the value and not exact "5")
>
>VB uses Bankers Rounding, which avoids bias in the overall. That is,
>it rounds, using the normal convention, to the nearest "even". The
>last example you show labeled as "wrong" (1.545 rounds to 1.54) is
>actually correct.
>
>I didn't go through the entire list, but you may want to look at
>
> http://support.microsoft.com/support.../Q196/6/52.ASP
> http://support.microsoft.com/support.../Q194/9/83.ASP
> http://support.microsoft.com/support...s/Q42/9/80.ASP
>
>
>On 14 Aug 2001 19:04:37 -0700, "Matthew"
>
>>
>>Why does this return false
>>round(1.015,2) = (round(0.015,2) + 1)
>>
>>it appears that when the number is less than 1 round behaves differently
>>0.015 rounds to 0.02 - correct
>>where as
>>1.015 rounds to 1.01 - wrong
>>
>>but to even complicate matters more
>>0.115 rounds to 0.12 - correct
>>1.115 rounds to 1.12 - correct
>>1.015 rounds to 1.01 - wrong
>>1.555 rounds to 1.56 - correct
>>1.545 rounds to 1.54 - wrong

>
>
>>
>>Does anybody else have experience with this and can they suggest an alternative.
>>
>>

>
>Language Stability is a *feature* I wish VB had!
> (#6)

4. mrfelis Guest

Re: Rounding problems

news:3b7a0d75\$1@news.devx.com...
>
> Hi Dan,
>
> understanding the difference between bankers rounding and real rounding

has
>
> round() uses bankers rounding whereas format(#,"0.00") uses correct

mathematical
> rounding.

VB uses bankers rounding in most places. I'm surprised format() doesn't.
Correct rounding can be done using this formula:

'Move the decimal point d places to the right
x = N * 10^d

'Add 1 if the fractional part of x is .5 or greater
x = int(x) + int((x - int(x))*2)

'Move the decimal point back to the original location
N = x * 10^-d

where:
N = the number to round
d = number of decimal places to round to

> I've heard of bankers hours, Bank fees, now I can add a new one, Bank

Rounding,
> I know it is designed to even out rounding, but somehow I bet the scales
> are tilted in their favour.
>

Yuo bet, in a \$19.95 price tag world.
--
~~~
!ti timda I ,KO
..em deppals nocaeB sivaM
!draH
~~
C'Ya,
mrfelis@yahoo!com

> Cheers,
>
> Matt
>
>
>
>
> Dan Barclay <Dan@MVPs.org> wrote:
> >Matthew,
> >
> >There is more than one issue here. (1) not all decimal numbers lend
> >themselves to exact representation in binary and (2) there are lots of
> >ways to round.
> >
> >Some of the numbers you're showing may not be exactly what you think
> >(may be a touch under or over the value and not exact "5")
> >
> >VB uses Bankers Rounding, which avoids bias in the overall. That is,
> >it rounds, using the normal convention, to the nearest "even". The
> >last example you show labeled as "wrong" (1.545 rounds to 1.54) is
> >actually correct.
> >
> >I didn't go through the entire list, but you may want to look at
> >
> > http://support.microsoft.com/support.../Q196/6/52.ASP
> > http://support.microsoft.com/support.../Q194/9/83.ASP
> > http://support.microsoft.com/support...s/Q42/9/80.ASP
> >
> >
> >On 14 Aug 2001 19:04:37 -0700, "Matthew"
> >
> >>
> >>Why does this return false
> >>round(1.015,2) = (round(0.015,2) + 1)
> >>
> >>it appears that when the number is less than 1 round behaves differently
> >>0.015 rounds to 0.02 - correct
> >>where as
> >>1.015 rounds to 1.01 - wrong
> >>
> >>but to even complicate matters more
> >>0.115 rounds to 0.12 - correct
> >>1.115 rounds to 1.12 - correct
> >>1.015 rounds to 1.01 - wrong
> >>1.555 rounds to 1.56 - correct
> >>1.545 rounds to 1.54 - wrong

> >
> >
> >>
> >>Does anybody else have experience with this and can they suggest an

alternative.
> >>
> >>

> >
> >Language Stability is a *feature* I wish VB had!
> > (#6)

>

Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

 FAQ Latest Articles Java .NET XML Database Enterprise