« Return to Thread: Floating point imprecision in activerecord-jdbc-adapter?

Re: Floating point imprecision in activerecord-jdbc-adapter?

by Charles Oliver Nutter-4 :: Rate this Message:

Reply to Author | View in Thread

On Wed, Jul 8, 2009 at 3:16 PM, Justin Coyne<digger250@...> wrote:

>   1) Failure:
> test_jdbc_precision(TestPrecision) [problem_rails.rb:25]:
> <5.8> expected but was
> <5.80000019073486>.
>
>
> This is because the activerecord-jdbc adapter is casting the value from the
> database to a float (32 bit) and then to a RubyFloat which is backed by a
> java double type.   To be consistent, activerecord-jdbc should just use the
> same representation that jruby uses internally (a double).

Agreed...whenever possible the floating-point representation fed to
JRuby should be as high precision as possible, since we'll always
upcast to double.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


 « Return to Thread: Floating point imprecision in activerecord-jdbc-adapter?