precision in AMPL

[MWOR]

Dear AMPL experts,

I'm running ampl on a unix system and encountered the following problem/phenomena:

option display_precision 30; # all other options as predefined
display 1e-15;

Then I receive the following:

1e-15 = 1.00000000000000007770539987666e-15

Furthermore:
display 1e-20 - 1e-10*1e-10;

-1.50463276905252801019998276764e-36

Since I want to implement a termination criterion saying that two values are the same (or their difference is 0 respectively), errors occur when my tolerance is set to values very close to 0, e.g. 1e-15.

Now, I handled this problem with round/trunc, but it took me quite a long time to see what the problem was.

Is this due to AMPL, my machine, a general problem working with floating point numbers, or ....?

Maybe someone else already encountered this problem?

Thanks in advance and best regards
Martin Weber

[Paul A. Rubin]

On Aug 28, 9:45 am, MWOR <weber...@...> wrote:

> Dear AMPL experts,
>
> I'm running ampl on a unix system and encountered the following
> problem/phenomena:
>
> option display_precision 30; # all other options as predefined
> display 1e-15;
>
> Then I receive the following:
>
> 1e-15 = 1.00000000000000007770539987666e-15

This looks correct to me.  I don't know if AMPL is compiled against a
specific standard for double precision (if so, most likely IEEE's
"decimal64"?) or if it depends on the hardware/OS against which it's
being compiled (in which case perhaps it's "decimal64" on some systems
and "decimal128 on others?).  The "decimal64" standard should give 16
decimal digit accuracy, and it looks like you have that about.  If you
were expecting to see _exactly_ 1.00000... e-15, keep in mind that
1e-15 does not have an exact binary representation.

My general approach to testing equality is abs(this - that) <
small_but_not_too_small.
>
> Furthermore:
> display 1e-20 - 1e-10*1e-10;
>
> -1.50463276905252801019998276764e-36

Again, this is about 16 decimal digits of accuracy, which is what I
would expect (actually slightly better than I would expect, since
you're dealing with both truncation error on the decimal-binary
conversions and rounding error on the arithmetic operations).

/Paul
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "AMPL Modeling Language" group.
To post to this group, send email to ampl@...
To unsubscribe from this group, send email to ampl+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/ampl?hl=en
-~----------~----~----~----~------~----~------~--~---