very very very slow computing

[Depo]

Hi everybody,

I was just trying making some simple operations on a large matrix (10, 3000) and I faced off o very slow computational time comparing to the simple operation I was attempting to do. Looking for the problem I saw that even copying same values to a new matrix was a computationally huge operation!!!!!!!!!!!!!

That is the code:

clear new_sign_difference;                                        #to ensure it is emty
i=1; k=1;
while (k <= n_rows)                                                  #I have used while statement instead of for because some says it is faster...
while (i <= n_cols)
new_sign_difference(k,i)=sign_difference(k,i)           #reassign same values to a new matrix
disp(i);
disp(k);
i++;
endwhile
k++; i=1;
endwhile  

the matrix sign_difference has size 10x3'000, so this is a 30'000-operation loop. It takes up to 10 minutes!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

If I only visualize on screen matrix' values it takes 1 second............................., here is the code:


clear new_sign_difference;
i=1; k=1;
while (k <= n_rows)
while (i <= n_cols)
sign_difference(k,i)                #only to screen, without writing on a new matrix
disp(i);
disp(k);
i++;
endwhile
k++; i=1;
endwhile  


How is it possible?????????????????????????????????????
Please help me, I'm getting crazy and I can't work at all at these conditions..........................


Tanks everybody

[Ivan Sutoris]

On Fri, Feb 6, 2009 at 5:04 PM, Depo <ste_depo@...> wrote:

>
> Hi everybody,
>
> I was just trying making some simple operations on a large matrix (10, 3000)
> and I faced off o very slow computational time comparing to the simple
> operation I was attempting to do. Looking for the problem I saw that even
> copying same values to a new matrix was a computationally huge
> operation!!!!!!!!!!!!!
>
> That is the code:
>
> clear new_sign_difference;                                        #to ensure
> it is emty
> i=1; k=1;
> while (k <= n_rows)                                                  #I have
> used while statement instead of for because some says it is faster...
> while (i <= n_cols)
> new_sign_difference(k,i)=sign_difference(k,i)           #reassign same
> values to a new matrix
> disp(i);
> disp(k);
> i++;
> endwhile
> k++; i=1;
> endwhile
>
> the matrix sign_difference has size 10x3'000, so this is a 30'000-operation
> loop. It takes up to 10
> minutes!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>
> If I only visualize on screen matrix' values it takes 1
> second............................., here is the code:
>
>
> clear new_sign_difference;
> i=1; k=1;
> while (k <= n_rows)
> while (i <= n_cols)
> sign_difference(k,i)                #only to screen, without writing on a
> new matrix
> disp(i);
> disp(k);
> i++;
> endwhile
> k++; i=1;
> endwhile
>
>
> How is it possible?????????????????????????????????????
> Please help me, I'm getting crazy and I can't work at all at these
> conditions..........................
>
>
> Tanks everybody

Hi

I guess your problem is missing semicolon in line
    new_sign_difference(k,i)=sign_difference(k,i)
which means that whole matrix is printed in every cycle . After I
removed output to screen by adding semicolon and removing both disp()
commands, the assignment takes about 2.5 secs on my computer (for
10*3000 matrix).

Regards
Ivan Sutoris
_______________________________________________
Help-octave mailing list
Help-octave@...
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave

[Rob Mahurin]

On Feb 6, 2009, at 11:04 AM, Depo wrote:

> Hi everybody,
>
> I was just trying making some simple operations on a large matrix  
> (10, 3000) and I faced off o very slow computational time comparing  
> to the simple operation I was attempting to do. Looking for the  
> problem I saw that even copying same values to a new matrix was a  
> computationally huge operation!!!!!!!!!!!!!
>
> That is the code:
>
> clear new_sign_difference;   #to ensure it is emty
> i=1; k=1;
> while (k <= n_rows)   #I have used while statement instead of for  
> because some says it is faster...
> while (i <= n_cols)
> new_sign_difference(k,i)=sign_difference(k,i)    #reassign same  
> values to a new matrix
> disp(i);
> disp(k);
> i++;
> endwhile
> k++; i=1;
> endwhile

This code does the same thing as the statement
        new_sign_difference = sign_difference
which copies the entire matrix at once, very quickly.

If you have some expression you need to do element-by-element, use  
"vectorize" to make sure you haven't missed any *,/,^ operators.

If you have some code that doesn't vectorize (hey, it happens),  
define matrices for your results before starting so that your loop  
doesn't waste time reallocating memory when the matrix grows:
        new_sign_difference  = zeros(size(sign_difference));
        for (i = 1:numel(sign_difference))
                new_sign_difference(i) = [something]

> the matrix sign_difference has size 10x3'000, so this is a 30'000-
> operation loop. It takes up to 10 minutes!
>
> If I only visualize on screen matrix' values it takes 1
> second............................., here is the code:
>
>
> clear new_sign_difference;
> i=1; k=1;
> while (k <= n_rows) while (i <= n_cols)
> sign_difference(k,i) #only to screen, without writing on a new matrix
> [...]

Sounds like your problem was repeatedly re-allocating memory.  
Allocate, clear, allocate a little more every time turns a 30,000  
operation loop to a (30,000)^2 operation loop.

Cheers,
Rob

--
Rob Mahurin
Department of Physics and Astronomy
University of Tennessee 865 207 2594
Knoxville, TN 37996 rob@...



_______________________________________________
Help-octave mailing list
Help-octave@...
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave

[schi]

> -----Original Message-----
> From: Depo [mailto:ste_depo@...]
> Sent: Friday, February 06, 2009 5:05 PM
> To: help-octave@...
> Subject: very very very slow computing
>
>
>
> Hi everybody,
>
> I was just trying making some simple operations on a large
> matrix (10, 3000)
> and I faced off o very slow computational time comparing to the simple
> operation I was attempting to do. Looking for the problem I
> saw that even
> copying same values to a new matrix was a computationally huge
> operation!!!!!!!!!!!!!
>
> That is the code:
>
> clear new_sign_difference;                                    
>     #to ensure
> it is emty
> i=1; k=1;
> while (k <= n_rows)                                          
>        #I have
> used while statement instead of for because some says it is faster...
> while (i <= n_cols)
> new_sign_difference(k,i)=sign_difference(k,i)           #reassign same
> values to a new matrix
> disp(i);
> disp(k);
> i++;
> endwhile
> k++; i=1;
> endwhile  
>
> the matrix sign_difference has size 10x3'000, so this is a
> 30'000-operation
> loop. It takes up to 10
> minutes!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>
> If I only visualize on screen matrix' values it takes 1
> second............................., here is the code:
>
>
> clear new_sign_difference;
> i=1; k=1;
> while (k <= n_rows)
> while (i <= n_cols)
> sign_difference(k,i)                #only to screen, without
> writing on a
> new matrix
> disp(i);
> disp(k);
> i++;
> endwhile
> k++; i=1;
> endwhile  
>
>
> How is it possible?????????????????????????????????????
> Please help me, I'm getting crazy and I can't work at all at these
> conditions..........................
>
>
> Tanks everybody
> --
> View this message in context:
> http://www.nabble.com/very-very-very-slow-computing-tp21875383

p21875383.html
Sent from the Octave - General mailing list archive at Nabble.com.

The way you set up your code, there are 30000 times where the new matrix
requires to allocate new memory and get resized. Please try to allocate one
chunk of memory before you enter the loops, e.g. by

new_sign_difference=zeros(size(sign_difference));

Should help you a lot ;-)

Rolf
_______________________________________________
Help-octave mailing list
Help-octave@...
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave

[Depo]

> I guess your problem is missing semicolon in line
>    new_sign_difference(k,i)=sign_difference(k,i)
> which means that whole matrix is printed in every cycle . After I
> removed output to screen by adding semicolon and removing both disp()
> commands, the assignment takes about 2.5 secs on my computer (for
> 10*3000 matrix).

That was the problem!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Even in my computer works

[John W. Eaton-3]

On  7-Feb-2009, Depo wrote:

|
| > I guess your problem is missing semicolon in line
| >    new_sign_difference(k,i)=sign_difference(k,i)
| > which means that whole matrix is printed in every cycle . After I
| > removed output to screen by adding semicolon and removing both disp()
| > commands, the assignment takes about 2.5 secs on my computer (for
| > 10*3000 matrix).
|
| That was the problem!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Even in my computer
| works

Dude, take it easy on the ! key.  Or does that key on your keyboard
sometimes get stuck?

Please tell us that you did also understand the advice about Octave
providing an array-based language, and that you should not be writing
loops to copy matrices.  All you need to do to copy a matrix is

  mx = rand (3);
  new_mx = mx;

Nice, simple, and fast.  A second copy is not even really made until
you modify some element of mx or new_mx.  Magic!!!!!!!!!!!!! (oops,
stuck key).

jwe
_______________________________________________
Help-octave mailing list
Help-octave@...
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave