[68cat] [core] Problems with "Find in projects"

[Neil Green-3]

I'm having serious problems using the "Find in projects" function in 6.8 beta and in 200910260201. The IDE spends a long time (>5 mins) doing "Preparing data for searching" and sometimes an OOME is eventually thrown. This doesn't happen with all projects I use, but those affected are always affected. Has anyone else run into this problem?

Regards
Neil

[Alexei Mokeev-2]

Hi Neil,

Could you please report it to IZ and attach a sample where it always
happened ?
  There was one report against 6.7 or 6.5 with very similar behavior:
"Preparing data for searching" takes too long, but we were not able to
repeat it.

Thanks,
-Alexei.

Neil Green пишет:
> I'm having serious problems using the "Find in projects" function in 6.8
> beta and in 200910260201. The IDE spends a long time (>5 mins) doing
> "Preparing data for searching" and sometimes an OOME is eventually
> thrown. This doesn't happen with all projects I use, but those affected
> are always affected. Has anyone else run into this problem?
>
> Regards
> Neil

[Neil Green-3]

Unfortunately, the project group where it always happens is code belonging to a client so I can't provide it. I'll report it as a defect and see if I can create a similar project structure that exhibits the same problem.

Regards
Neil

2009/10/27 Alexei Mokeev <Alexei.Mokeev@...>

Hi Neil,

Could you please report it to IZ and attach a sample where it always happened ?
 There was one report against 6.7 or 6.5 with very similar behavior:
"Preparing data for searching" takes too long, but we were not able to repeat it.

Thanks,
-Alexei.

Neil Green пишет:

I'm having serious problems using the "Find in projects" function in 6.8 beta and in 200910260201. The IDE spends a long time (>5 mins) doing "Preparing data for searching" and sometimes an OOME is eventually thrown. This doesn't happen with all projects I use, but those affected are always affected. Has anyone else run into this problem?

Regards
Neil

[Alexei Mokeev-2]

Understood. All the details are appreciated.
The original issue is #167152, please provide details what you can here:
http://www.netbeans.org/issues/show_bug.cgi?id=167152

Thank you !

-Alexei.

Neil Green пишет:

> Unfortunately, the project group where it always happens is code
> belonging to a client so I can't provide it. I'll report it as a defect
> and see if I can create a similar project structure that exhibits the
> same problem.
>
> Regards
> Neil
>
> 2009/10/27 Alexei Mokeev <Alexei.Mokeev@...
> <mailto:Alexei.Mokeev@...>>
>
>     Hi Neil,
>
>     Could you please report it to IZ and attach a sample where it always
>     happened ?
>      There was one report against 6.7 or 6.5 with very similar behavior:
>     "Preparing data for searching" takes too long, but we were not able
>     to repeat it.
>
>     Thanks,
>     -Alexei.
>
>     Neil Green пишет:
>
>         I'm having serious problems using the "Find in projects"
>         function in 6.8 beta and in 200910260201. The IDE spends a long
>         time (>5 mins) doing "Preparing data for searching" and
>         sometimes an OOME is eventually thrown. This doesn't happen with
>         all projects I use, but those affected are always affected. Has
>         anyone else run into this problem?
>
>         Regards
>         Neil
>
>
>

[Tomas Pavek]

Could you possibly generate a histogram or heapdump? So we know what data
structures occupy the memory? E.g. use jmap from JDK:
jmap -histo:live <pid>

In this case not when OOME happens (most of the data can be dropped from
memory when the operation is interrupted), but at some moment close to it.

Creating a profiling snapshot (http://wiki.netbeans.org/FaqProfileMeNow) of
the preparing phase of the search operation would also help.

Thanks,
Tomas


Neil Green wrote:

> Unfortunately, the project group where it always happens is code
> belonging to a client so I can't provide it. I'll report it as a defect
> and see if I can create a similar project structure that exhibits the
> same problem.
>
> Regards
> Neil
>
> 2009/10/27 Alexei Mokeev <Alexei.Mokeev@...
> <mailto:Alexei.Mokeev@...>>
>
>     Hi Neil,
>
>     Could you please report it to IZ and attach a sample where it always
>     happened ?
>      There was one report against 6.7 or 6.5 with very similar behavior:
>     "Preparing data for searching" takes too long, but we were not able
>     to repeat it.
>
>     Thanks,
>     -Alexei.
>
>     Neil Green пишет:
>
>         I'm having serious problems using the "Find in projects"
>         function in 6.8 beta and in 200910260201. The IDE spends a long
>         time (>5 mins) doing "Preparing data for searching" and
>         sometimes an OOME is eventually thrown. This doesn't happen with
>         all projects I use, but those affected are always affected. Has
>         anyone else run into this problem?
>
>         Regards
>         Neil
>
>
>

[Neil Green-3]

I've uploaded the profiling snapshot. I'd already turned on auto heap dump for OOME and I've generated one. I'll try and get another before the OOME.

Regards
Neil

2009/10/27 Tomas Pavek <Tomas.Pavek@...>

Could you possibly generate a histogram or heapdump? So we know what data structures occupy the memory? E.g. use jmap from JDK:
jmap -histo:live <pid>

In this case not when OOME happens (most of the data can be dropped from memory when the operation is interrupted), but at some moment close to it.

Creating a profiling snapshot (http://wiki.netbeans.org/FaqProfileMeNow) of the preparing phase of the search operation would also help.

Thanks,
Tomas


Neil Green wrote:

Unfortunately, the project group where it always happens is code belonging to a client so I can't provide it. I'll report it as a defect and see if I can create a similar project structure that exhibits the same problem.

Regards
Neil

2009/10/27 Alexei Mokeev <Alexei.Mokeev@... <mailto:Alexei.Mokeev@...>>

   Hi Neil,

   Could you please report it to IZ and attach a sample where it always
   happened ?
    There was one report against 6.7 or 6.5 with very similar behavior:
   "Preparing data for searching" takes too long, but we were not able
   to repeat it.

   Thanks,
   -Alexei.

   Neil Green пишет:

       I'm having serious problems using the "Find in projects"
       function in 6.8 beta and in 200910260201. The IDE spends a long
       time (>5 mins) doing "Preparing data for searching" and
       sometimes an OOME is eventually thrown. This doesn't happen with
       all projects I use, but those affected are always affected. Has
       anyone else run into this problem?

       Regards
       Neil

[Neil Green-3]

I can't upload the heap dumps, even compressed they're > 80MB each, is there any other way of sending them other than attaching them to an issue? I do have a sequence of histograms though. I'll upload them now.

Regards
Neil

2009/10/27 Neil Green <neil.r.green@...>

I've uploaded the profiling snapshot. I'd already turned on auto heap dump for OOME and I've generated one. I'll try and get another before the OOME.

Regards
Neil

2009/10/27 Tomas Pavek <Tomas.Pavek@...>

Could you possibly generate a histogram or heapdump? So we know what data structures occupy the memory? E.g. use jmap from JDK:
jmap -histo:live <pid>

In this case not when OOME happens (most of the data can be dropped from memory when the operation is interrupted), but at some moment close to it.

Creating a profiling snapshot (http://wiki.netbeans.org/FaqProfileMeNow) of the preparing phase of the search operation would also help.

Thanks,
Tomas


Neil Green wrote:

Unfortunately, the project group where it always happens is code belonging to a client so I can't provide it. I'll report it as a defect and see if I can create a similar project structure that exhibits the same problem.

Regards
Neil

2009/10/27 Alexei Mokeev <Alexei.Mokeev@... <mailto:Alexei.Mokeev@...>>

   Hi Neil,

   Could you please report it to IZ and attach a sample where it always
   happened ?
    There was one report against 6.7 or 6.5 with very similar behavior:
   "Preparing data for searching" takes too long, but we were not able
   to repeat it.

   Thanks,
   -Alexei.

   Neil Green пишет:

       I'm having serious problems using the "Find in projects"
       function in 6.8 beta and in 200910260201. The IDE spends a long
       time (>5 mins) doing "Preparing data for searching" and
       sometimes an OOME is eventually thrown. This doesn't happen with
       all projects I use, but those affected are always affected. Has
       anyone else run into this problem?

       Regards
       Neil

[Ulf Zibis-2]

Am 27.10.2009 14:44, Neil Green schrieb:
> I can't upload the heap dumps, even compressed they're > 80MB each, is
> there any other way of sending them other than attaching them to an
> issue? I do have a sequence of histograms though. I'll upload them now.

Tipp: use 7-Zip instead of normal Zip. I my case I could compress 330 MB
to 21 MB.

-Ulf

[joshis]

Use RAR instead of ZIP, it works slightly better... but I suppose this
won't help in your case... You can upload the file to any persistent
place (what about a little Kenai downloads feature abuse? :)) and link
it from the issue...

--pd

Ulf Zibis wrote:

> Am 27.10.2009 14:44, Neil Green schrieb:
>> I can't upload the heap dumps, even compressed they're > 80MB each,
>> is there any other way of sending them other than attaching them to
>> an issue? I do have a sequence of histograms though. I'll upload them
>> now.
>
> Tipp: use 7-Zip instead of normal Zip. I my case I could compress 330
> MB to 21 MB.
>
> -Ulf
>
>

[Ulf Zibis-2]

Am 27.10.2009 15:07, Petr Dvorak schrieb:
> Use RAR instead of ZIP

... aren't we an open source community ? ;-)

I tried it: RAR compresses much faster, but it needed 37 MB.

-Ulf

[joshis]

 > ... aren't we an open source community ? ;-)

Ouch - you got me... now everyone in the NetCat team knows I am paid by
the monstrous RARCorp Industries for pushing their products... ;) I
wonder how I get over it...

But the results don't lie... thanks for testing it, Ulf...

--pd

Ulf Zibis wrote:

> Am 27.10.2009 15:07, Petr Dvorak schrieb:
>> Use RAR instead of ZIP
>
> ... aren't we an open source community ? ;-)
>
> I tried it: RAR compresses much faster, but it needed 37 MB.
>
> -Ulf
>
>

[Jiri Kovalsky]

Neil, can you try uploading them to our shared QA location [1] for heapdumps and then providing links in the issue? Just like Ulf Zibis did it earlier today.

[1] http://qa.netbeans.org/servlets/ProjectDocumentList?folderID=367&expandFolder=367&folderID=367

Thanks for your cooperation,
-Jirka

Neil Green wrote:

> I can't upload the heap dumps, even compressed they're > 80MB each, is
> there any other way of sending them other than attaching them to an
> issue? I do have a sequence of histograms though. I'll upload them now.
>
> Regards
> Neil
>
> 2009/10/27 Neil Green <neil.r.green@... wrote:
>
>> I've uploaded the profiling snapshot. I'd already turned on auto
>> heap dump for OOME and I've generated one. I'll try and get another
>> before the OOME.
>>
>> Regards
>> Neil

[snip]