David sent in a question that I wasn't able to resolve, so I thought I'd share it with everyone else:
I'm confused by a odd rounding type error.. and this is really starting to bud me but I can't explain why or how to get round it...It started whilst trying to get the NumberFormat function to return the correct values for Value Added Tax. The value output is different depending if the input value has been calculated or has been set in the system. The input value appears the same - but isn't..
<cfoutput> <cfset testvalue="34.825"> <cfset testvaluerounded=NumberFormat(testvalue, ".")>
#testvalue# #testvaluerounded#<br />
<!--- Now the reason we found the problem --->
<cfset cost="199"> <cfset vat=(cost * 0.175)> <cfset vatrounded=NumberFormat(vat, ".")>
#vat# #vatrounded#
</cfoutput>
So I ran his code and confirmed it. I figured it was probably due to the typeless nature of ColdFusion. I did a quick test and outputted testvalue.getClass() and vat.getClass(). As I expected, testvalue.getClass returned java.lang.String, and bat returned java.lang.Double. Of course, I would have assumed the String would have rounded wrong, not the Double.
So how to fix? I tried this:
<cfset testvalue= 34.825 + 0>
This was enough to make testvaue also return as a Double, but get this - it still rounded the wrong way. I then tried something dumb:
<cfset vat = round(vat*1000)/1000>
Since the vat had numbers out to the thousands place, this should do nothing - and it did nothing. I still had a vat value of 34.825. However - when I next ran the numberFormat, it rounded the same, although still incorrectly.
So - would this a ColdFusion bug? Or just something we have to live with since ColdFusion is typeless? I even tried this:
<cfset vat = round(vat*100)/100>
and it returned the wrong value. The exact same code run on testvalue worked just fine.
Archived Comments
Its funny you mention this, we just had a (looong) discussion about this on cf-talk just last week I think:
http://www.houseoffusion.co...
It has to do with rounding errors that are inherit to all software languages and the conversion to binary and back again.
This should work as expected i believe: <cfset vat = int(vat*100)/100>
Here's a workaround I used in days of old:
<cfset cost="199">
<cfset vat=(cost * 0.175)>
<cfset vatrounded = int( (vat + 0.005) * 100 ) / 100 />
<cfset vatrounded = NumberFormat(vatrounded, "__.__")>
testvalue: #vat# rounded: #vatrounded#<br />
This'll fix it, too:
<cfset cost = "199">
<cfset vat = (cost * 0.175) >
<cfset vat = javacast("float", vat) />
<cfset vatrounded=NumberFormat(vat, "__.__")>
#vat# #vatrounded#
-Joe
If you're interested, add this line:
<br />#(vat - 34.825) * 100000000000000#
Just after the #vat# #vatrounded# line in the original example. It'll show the error.
Yeah, that's ironic.. I wrote a post about this a few days before it showed-up on CF-Talk:
http://devnulled.com/conten...
Geeze, it must be something in the air then. ;) Well, this is precisely the kind of thing that probably should be repeated. It needs to be something people don't forget about.
Also remember that float/doubles are not 100% accurate in how they hold decimals. See Sean's post
http://www.corfield.org/blo...
and the detailed article he links to in his post.
http://blogs.msdn.com/ericl...
The post written by Beth containing:
<cfset vatrounded = int( (vat + 0.005) * 100 ) / 100 />
has saved my life and job toooooo...
Thank you soooo much.....