« Return to Thread: Floating point imprecision in activerecord-jdbc-adapter?

Re: Floating point imprecision in activerecord-jdbc-adapter?

by Justin Coyne :: Rate this Message:

Reply to Author | View in Thread

I'm going to assert that this test should pass:

  def test_jdbc_precision
    Sample.delete_all()
    assert_nil Sample.find_by_name('br1991a')
    density = 5.8
    Sample.new(:name=>'br1991a', :density=>density).save!
    sample = Sample.find_by_name('br1991a')
    assert_not_nil sample
    assert_equal density, sample.density
  end


Presently it doesn't:


  1) Failure:
test_jdbc_precision(TestPrecision) [problem_rails.rb:25]:
<5.8> expected but was
<5.80000019073486>.


This is because the activerecord-jdbc adapter is casting the value from the database to a float (32 bit) and then to a RubyFloat which is backed by a java double type.   To be consistent, activerecord-jdbc should just use the same representation that jruby uses internally (a double). 


-Justin











On Wed, Jul 8, 2009 at 12:12 PM, Michael Campbell <michael.campbell@...> wrote:
Uwe Kubosch wrote:

When using Float/Double, you should always be prepared for small
variations during handling.

If you want to store and retrieve a decimal number without it being
subject to rounding etc, you should use the DECIMAL column type.  It
will be converted to a BigDecimal when loaded by ActiveRecord, and it
will not suffer any rounding/conversion changes.

+1.  That MRI and JRuby do different things seems interesting, but beyond that this is pretty standard post-COBOL-BCD numeric computing fare.

For the OP, I think most unit test frameworks provide some sort of assertEquals(floatingpoint_value_1, floatingpoint_value_2, allowed_variance) type of call.  If not, it's trivial to write.




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email



 « Return to Thread: Floating point imprecision in activerecord-jdbc-adapter?