jhandles truncates large numbers

View: New views
2 Messages — Rating Filter:   Alert me  

jhandles truncates large numbers

by Scott Haynes :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Using jhandles-0.3.3 on both my windows and Linux machines I have the same problem.

At the octave command prompt I type:


plot(floor(now):1/24:floor(now)+1);


This yields a jagged line, however if you type the following:


plot(0:1/24:1);


you will see the line is straight (as it should be).



Re: jhandles truncates large numbers

by Michael Goffioul-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Jul 7, 2008 at 5:57 AM, Scott Haynes <scotth20@...> wrote:

>
> Using jhandles-0.3.3 on both my windows and Linux machines I have the same
> problem.
>
> At the octave command prompt I type:
>
>
> plot(floor(now):1/24:floor(now)+1);
>
>
> This yields a jagged line, however if you type the following:
>
>
> plot(0:1/24:1);
>
>
> you will see the line is straight (as it should be).

This is due to the single-precision computation used in
OpenGL. The current OpenGL renderer in octave also
suffer from this problem. It basically happen when the
ratio between DX and XMIN (or XMAX) becomes too
small.

I think that this kind of problem could be avoided by
normalizing the data in octave (hence with double-precision)
before sending them to OpenGL. The normalized transformation
matrix is much more numerically stable than the
unnormalized one. Although I think that you should then be
careful with lighting effect and how normals are transformed,
so it might not be so easy to fix.

Testing it on the competition, it seems they suffer from the
same problem when the renderer is set to OpenGL
(another reason why I think an alternate simple 2D-only
renderer would be useful, for instance a cairo-based one).

Michael.
_______________________________________________
Bug-octave mailing list
Bug-octave@...
https://www.cae.wisc.edu/mailman/listinfo/bug-octave