« Return to Thread: Floating point imprecision in activerecord-jdbc-adapter?
Re: Floating point imprecision in activerecord-jdbc-adapter?
by Uwe Kubosch
::
Rate this Message:
On Tue, 2009-07-07 at 14:53 -0500, Charles Oliver Nutter wrote:
> On Tue, Jul 7, 2009 at 2:26 PM, Justin Coyne<digger250@...> wrote:
> > Okay, here's the real bug. It's not in jdbc at all. It appears to be deeper
> > in the guts of jruby.
> >
> > jruby -e "require 'java'; puts
> > JRuby.runtime.newFloat(java.lang.Float.new(5.8)).to_s()"
> >
> > 5.80000019073486
>
> It looks like the conversion from single-precision float to
> double-precision produces this effect.
> java.lang.Float.new(5.8)doubleValue produces the same result.
> This starts to look like a "WONTFIX" or maybe a "CANTFIX" since it's
> Java/JDK/JVM float-conversion behavior at play here. But if we're
> getting floats out of the database somewhere in the JDBC adapters,
> perhaps we should ask for Double or double instead?
When using Float/Double, you should always be prepared for small
variations during handling.
If you want to store and retrieve a decimal number without it being
subject to rounding etc, you should use the DECIMAL column type. It
will be converted to a BigDecimal when loaded by ActiveRecord, and it
will not suffer any rounding/conversion changes.
--
With kind regards,
Uwe Kubosch
Kubosch Consulting
Norway
http://kubosch.no/
> On Tue, Jul 7, 2009 at 2:26 PM, Justin Coyne<digger250@...> wrote:
> > Okay, here's the real bug. It's not in jdbc at all. It appears to be deeper
> > in the guts of jruby.
> >
> > jruby -e "require 'java'; puts
> > JRuby.runtime.newFloat(java.lang.Float.new(5.8)).to_s()"
> >
> > 5.80000019073486
>
> It looks like the conversion from single-precision float to
> double-precision produces this effect.
> java.lang.Float.new(5.8)doubleValue produces the same result.
> This starts to look like a "WONTFIX" or maybe a "CANTFIX" since it's
> Java/JDK/JVM float-conversion behavior at play here. But if we're
> getting floats out of the database somewhere in the JDBC adapters,
> perhaps we should ask for Double or double instead?
When using Float/Double, you should always be prepared for small
variations during handling.
If you want to store and retrieve a decimal number without it being
subject to rounding etc, you should use the DECIMAL column type. It
will be converted to a BigDecimal when loaded by ActiveRecord, and it
will not suffer any rounding/conversion changes.
--
With kind regards,
Uwe Kubosch
Kubosch Consulting
Norway
http://kubosch.no/
signature.asc (204 bytes) « Return to Thread: Floating point imprecision in activerecord-jdbc-adapter?
| Free embeddable forum powered by Nabble | Forum Help |
