Netbeans and a subset of a JDK workspace

[Brad Wetmore]

I'd recently drunk a bit (lot?) of the netbeans 6.1 Kool-Aid.  Wow, what
an improvement over 4.1.  I swore that I'd rather die using vi after
that initial netbeans experience, but here I am!  ;)

If I open the "jdk/make/netbeans/j2se" project, I might as well walk
away and have lunch.  The internal indexing/compiling of the entire j2se
project simply takes forever on my relatively fast machine, with the
majority of the time being in the internal compilation.  I'd really like
to just limit the sources to specific code of interest (core libs,
networking, security).  (What's this Swing thing I hear people talk
about...  ;) )

A big optimization that seems to be working for me is the following:

0)   Install netbeans 6.1

1)   Clone/build (using gnumake) a jdk workspace (say JSN) as usual.

2)   Go to Tools->Java platforms.

      Add the built platform but do not include links to the sources.
      Call this "jsn-gate:  no sources"  We'll set the source
      locations in the project in the next step.

3)   Create a new project from existing sources (jdk/src/shared,
      jdk/src/solaris, etc).  Include only the JDK
      sources I care about (core libs, networking, security),
      everything else will be excluded.  For
      example:

   =java/lang/, java/math/, java/net/, java/nio/, java/security/,
    java/text/, java/util/, javax/crypto/, javax/net/, javax/security/,
    javax/smartcardio/, javax/xml/crypto/, sun/io/, sun/launcher/,
    sun/misc/, sun/net/, sun/nio/, sun/security/, sun/text/, sun/tools/,
    sun/util/, com/sun/security/, com/sun/crypto/, com/sun/net/

      In the properties menu for this project, link this project to the
      JDK platform just created in step 2 (jsn-gate: no sources).
      This seems to internally compile only the sources I care about, not
      the whole world!

      Use this configuration to do your library development.  When you
      want to compile, do so in a separate window (that is, don't use the
      netbeans ant build, but use gnumake as usual).

4)   For testing:  Like in 2a, create a new jdk platform based on the
      build, but this time include links to all the sources in
      open/shared, open/solaris, etc.  Call this "jsn-gate."

      Now create a new project with a simple test case.  Link with the
      platform "jsn-gate".  Then when you want to debug and step into
      the core libraries, you have *ALL* the sources you need, but
      without them being internally compiled.  Indexing seems to take
      much less time.  This also has all the javadoc as well.

Is there an easier/faster way? This environment comes up in 30 seconds
or so, and drilling into the source packages takes about 10 seconds the
first use.

Note I am *NOT* an expert in projects, maybe someone has already figured
this out?  Looking at a few of these related projects in the
jdk/make/netbeans directories, they look like they're being done in the
same way as j2se.

Thanks,

Brad
Potential Netbeans Convert-  ;)

[daniel.fuchs]

Hi Brad,

You could have a look at how the make/netbeans/jmx project is
defined.
This is a project that only compiles a subset of the jdk sources
(the JMX sources) put them in a jmx.jar and then prepend that jar
to the bootclasspath.

For us using the latest promoted JDK7 build with our work-in-progress
jmx.jar in the bootclasspath is usually sufficient for
developing/testing/debugging purposes.


Hope this helps,

-- daniel
http://blogs.sun.com/jmxetc

Brad Wetmore wrote:

> I'd recently drunk a bit (lot?) of the netbeans 6.1 Kool-Aid.  Wow, what
> an improvement over 4.1.  I swore that I'd rather die using vi after
> that initial netbeans experience, but here I am!  ;)
>
> If I open the "jdk/make/netbeans/j2se" project, I might as well walk
> away and have lunch.  The internal indexing/compiling of the entire j2se
> project simply takes forever on my relatively fast machine, with the
> majority of the time being in the internal compilation.  I'd really like
> to just limit the sources to specific code of interest (core libs,
> networking, security).  (What's this Swing thing I hear people talk
> about...  ;) )
>
> A big optimization that seems to be working for me is the following:
>
> 0)   Install netbeans 6.1
>
> 1)   Clone/build (using gnumake) a jdk workspace (say JSN) as usual.
>
> 2)   Go to Tools->Java platforms.
>
>      Add the built platform but do not include links to the sources.
>      Call this "jsn-gate:  no sources"  We'll set the source
>      locations in the project in the next step.
>
> 3)   Create a new project from existing sources (jdk/src/shared,
>      jdk/src/solaris, etc).  Include only the JDK
>      sources I care about (core libs, networking, security),
>      everything else will be excluded.  For
>      example:
>
>   =java/lang/, java/math/, java/net/, java/nio/, java/security/,
>    java/text/, java/util/, javax/crypto/, javax/net/, javax/security/,
>    javax/smartcardio/, javax/xml/crypto/, sun/io/, sun/launcher/,
>    sun/misc/, sun/net/, sun/nio/, sun/security/, sun/text/, sun/tools/,
>    sun/util/, com/sun/security/, com/sun/crypto/, com/sun/net/
>
>      In the properties menu for this project, link this project to the
>      JDK platform just created in step 2 (jsn-gate: no sources).
>      This seems to internally compile only the sources I care about, not
>      the whole world!
>
>      Use this configuration to do your library development.  When you
>      want to compile, do so in a separate window (that is, don't use the
>      netbeans ant build, but use gnumake as usual).
>
> 4)   For testing:  Like in 2a, create a new jdk platform based on the
>      build, but this time include links to all the sources in
>      open/shared, open/solaris, etc.  Call this "jsn-gate."
>
>      Now create a new project with a simple test case.  Link with the
>      platform "jsn-gate".  Then when you want to debug and step into
>      the core libraries, you have *ALL* the sources you need, but
>      without them being internally compiled.  Indexing seems to take
>      much less time.  This also has all the javadoc as well.
>
> Is there an easier/faster way? This environment comes up in 30 seconds
> or so, and drilling into the source packages takes about 10 seconds the
> first use.
>
> Note I am *NOT* an expert in projects, maybe someone has already figured
> this out?  Looking at a few of these related projects in the
> jdk/make/netbeans directories, they look like they're being done in the
> same way as j2se.
>
> Thanks,
>
> Brad
> Potential Netbeans Convert-  ;)

[Brad Wetmore]

Thanks Daniel.

One of the other current limitations of the default projects was that
the "j2se" project didn't index the build/<arch>/gensrc directory where
java/nio/ByteBuffer and friends live (bug 6628175).  I was able to get
the directories to show up in the file lister, but not hooked into the
auto-completion/compilation mechanism.  I ended up talking to David
Bristor, this group's moderator.  He had started working on the gensrc
problem, but got 1/2 of it finished before he left Sun.  I was able to
get the second half, so it's working now.

I am now hooking in all the closed source directories as well.

Once everything gets auto-compiled (15-20 minutes) and loaded into
memory, it's surprisingly fast.  Restarting/scanning current project is
just under a minute, and expanding the node for the shared sources is
just under two.  :)  Fortunately, it's not crashing all the time like my
first experience with NetBeans 4.1!  I don't really have to restrict to
the files of interest once everything's loaded, it's responsive enough.

Unless I find something that's a showstopper, I'll probably start using
netbeans for my day-day work.

Brad



daniel fuchs wrote:

> Hi Brad,
>
> You could have a look at how the make/netbeans/jmx project is
> defined.
> This is a project that only compiles a subset of the jdk sources
> (the JMX sources) put them in a jmx.jar and then prepend that jar
> to the bootclasspath.
>
> For us using the latest promoted JDK7 build with our work-in-progress
> jmx.jar in the bootclasspath is usually sufficient for
> developing/testing/debugging purposes.
>
>
> Hope this helps,
>
> -- daniel
> http://blogs.sun.com/jmxetc
>
> Brad Wetmore wrote:
>> I'd recently drunk a bit (lot?) of the netbeans 6.1 Kool-Aid.  Wow,
>> what an improvement over 4.1.  I swore that I'd rather die using vi
>> after that initial netbeans experience, but here I am!  ;)
>>
>> If I open the "jdk/make/netbeans/j2se" project, I might as well walk
>> away and have lunch.  The internal indexing/compiling of the entire
>> j2se project simply takes forever on my relatively fast machine, with
>> the majority of the time being in the internal compilation.  I'd
>> really like to just limit the sources to specific code of interest
>> (core libs, networking, security).  (What's this Swing thing I hear
>> people talk about...  ;) )
>>
>> A big optimization that seems to be working for me is the following:
>>
>> 0)   Install netbeans 6.1
>>
>> 1)   Clone/build (using gnumake) a jdk workspace (say JSN) as usual.
>>
>> 2)   Go to Tools->Java platforms.
>>
>>      Add the built platform but do not include links to the sources.
>>      Call this "jsn-gate:  no sources"  We'll set the source
>>      locations in the project in the next step.
>>
>> 3)   Create a new project from existing sources (jdk/src/shared,
>>      jdk/src/solaris, etc).  Include only the JDK
>>      sources I care about (core libs, networking, security),
>>      everything else will be excluded.  For
>>      example:
>>
>>   =java/lang/, java/math/, java/net/, java/nio/, java/security/,
>>    java/text/, java/util/, javax/crypto/, javax/net/, javax/security/,
>>    javax/smartcardio/, javax/xml/crypto/, sun/io/, sun/launcher/,
>>    sun/misc/, sun/net/, sun/nio/, sun/security/, sun/text/, sun/tools/,
>>    sun/util/, com/sun/security/, com/sun/crypto/, com/sun/net/
>>
>>      In the properties menu for this project, link this project to the
>>      JDK platform just created in step 2 (jsn-gate: no sources).
>>      This seems to internally compile only the sources I care about, not
>>      the whole world!
>>
>>      Use this configuration to do your library development.  When you
>>      want to compile, do so in a separate window (that is, don't use the
>>      netbeans ant build, but use gnumake as usual).
>>
>> 4)   For testing:  Like in 2a, create a new jdk platform based on the
>>      build, but this time include links to all the sources in
>>      open/shared, open/solaris, etc.  Call this "jsn-gate."
>>
>>      Now create a new project with a simple test case.  Link with the
>>      platform "jsn-gate".  Then when you want to debug and step into
>>      the core libraries, you have *ALL* the sources you need, but
>>      without them being internally compiled.  Indexing seems to take
>>      much less time.  This also has all the javadoc as well.
>>
>> Is there an easier/faster way? This environment comes up in 30 seconds
>> or so, and drilling into the source packages takes about 10 seconds
>> the first use.
>>
>> Note I am *NOT* an expert in projects, maybe someone has already
>> figured this out?  Looking at a few of these related projects in the
>> jdk/make/netbeans directories, they look like they're being done in
>> the same way as j2se.
>>
>> Thanks,
>>
>> Brad
>> Potential Netbeans Convert-  ;)
>

[Dmitri Trembovetski]

   Hi Brad,

Brad Wetmore wrote:

> Thanks Daniel.
>
> One of the other current limitations of the default projects was that
> the "j2se" project didn't index the build/<arch>/gensrc directory where
> java/nio/ByteBuffer and friends live (bug 6628175).  I was able to get
> the directories to show up in the file lister, but not hooked into the
> auto-completion/compilation mechanism.  I ended up talking to David
> Bristor, this group's moderator.  He had started working on the gensrc
> problem, but got 1/2 of it finished before he left Sun.  I was able to
> get the second half, so it's working now.
>
> I am now hooking in all the closed source directories as well.
>
> Once everything gets auto-compiled (15-20 minutes) and loaded into

   What is this "auto-compilation" is actually doing? I mean, if I have
   a fully built j2se workspace, shouldn't everything be compiled already?

> memory, it's surprisingly fast.  Restarting/scanning current project is
> just under a minute, and expanding the node for the shared sources is
> just under two.  :)  Fortunately, it's not crashing all the time like my
> first experience with NetBeans 4.1!  I don't really have to restrict to
> the files of interest once everything's loaded, it's responsive enough.

   I find that tweaking the default heap settings (doubling them) improves
   things. Otherwise sooner or later you'll run into OOE.

>
> Unless I find something that's a showstopper, I'll probably start using
> netbeans for my day-day work.

   Welcome to to the club =) I've been using netbeans exclusively for
   jdk development for last couple of years. I also setup a project for
   the native code, using C/C++ plugin, works OK, even with autocompletion
   (if it feels like it).

   Thanks,
     Dmitri

>
> Brad
>
>
>
> daniel fuchs wrote:
>> Hi Brad,
>>
>> You could have a look at how the make/netbeans/jmx project is
>> defined.
>> This is a project that only compiles a subset of the jdk sources
>> (the JMX sources) put them in a jmx.jar and then prepend that jar
>> to the bootclasspath.
>>
>> For us using the latest promoted JDK7 build with our work-in-progress
>> jmx.jar in the bootclasspath is usually sufficient for
>> developing/testing/debugging purposes.
>>
>>
>> Hope this helps,
>>
>> -- daniel
>> http://blogs.sun.com/jmxetc
>>
>> Brad Wetmore wrote:
>>> I'd recently drunk a bit (lot?) of the netbeans 6.1 Kool-Aid.  Wow,
>>> what an improvement over 4.1.  I swore that I'd rather die using vi
>>> after that initial netbeans experience, but here I am!  ;)
>>>
>>> If I open the "jdk/make/netbeans/j2se" project, I might as well walk
>>> away and have lunch.  The internal indexing/compiling of the entire
>>> j2se project simply takes forever on my relatively fast machine, with
>>> the majority of the time being in the internal compilation.  I'd
>>> really like to just limit the sources to specific code of interest
>>> (core libs, networking, security).  (What's this Swing thing I hear
>>> people talk about...  ;) )
>>>
>>> A big optimization that seems to be working for me is the following:
>>>
>>> 0)   Install netbeans 6.1
>>>
>>> 1)   Clone/build (using gnumake) a jdk workspace (say JSN) as usual.
>>>
>>> 2)   Go to Tools->Java platforms.
>>>
>>>      Add the built platform but do not include links to the sources.
>>>      Call this "jsn-gate:  no sources"  We'll set the source
>>>      locations in the project in the next step.
>>>
>>> 3)   Create a new project from existing sources (jdk/src/shared,
>>>      jdk/src/solaris, etc).  Include only the JDK
>>>      sources I care about (core libs, networking, security),
>>>      everything else will be excluded.  For
>>>      example:
>>>
>>>   =java/lang/, java/math/, java/net/, java/nio/, java/security/,
>>>    java/text/, java/util/, javax/crypto/, javax/net/, javax/security/,
>>>    javax/smartcardio/, javax/xml/crypto/, sun/io/, sun/launcher/,
>>>    sun/misc/, sun/net/, sun/nio/, sun/security/, sun/text/, sun/tools/,
>>>    sun/util/, com/sun/security/, com/sun/crypto/, com/sun/net/
>>>
>>>      In the properties menu for this project, link this project to the
>>>      JDK platform just created in step 2 (jsn-gate: no sources).
>>>      This seems to internally compile only the sources I care about, not
>>>      the whole world!
>>>
>>>      Use this configuration to do your library development.  When you
>>>      want to compile, do so in a separate window (that is, don't use the
>>>      netbeans ant build, but use gnumake as usual).
>>>
>>> 4)   For testing:  Like in 2a, create a new jdk platform based on the
>>>      build, but this time include links to all the sources in
>>>      open/shared, open/solaris, etc.  Call this "jsn-gate."
>>>
>>>      Now create a new project with a simple test case.  Link with the
>>>      platform "jsn-gate".  Then when you want to debug and step into
>>>      the core libraries, you have *ALL* the sources you need, but
>>>      without them being internally compiled.  Indexing seems to take
>>>      much less time.  This also has all the javadoc as well.
>>>
>>> Is there an easier/faster way? This environment comes up in 30
>>> seconds or so, and drilling into the source packages takes about 10
>>> seconds the first use.
>>>
>>> Note I am *NOT* an expert in projects, maybe someone has already
>>> figured this out?  Looking at a few of these related projects in the
>>> jdk/make/netbeans directories, they look like they're being done in
>>> the same way as j2se.
>>>
>>> Thanks,
>>>
>>> Brad
>>> Potential Netbeans Convert-  ;)
>>

[Brad Wetmore]

>> One of the other current limitations of the default projects was that
>> the "j2se" project didn't index the build/<arch>/gensrc directory
>> where java/nio/ByteBuffer and friends live (bug 6628175).  I was able
>> to get the directories to show up in the file lister, but not hooked
>> into the auto-completion/compilation mechanism.  I ended up talking to
>> David Bristor, this group's moderator.  He had started working on the
>> gensrc problem, but got 1/2 of it finished before he left Sun.  I was
>> able to get the second half, so it's working now.
>>
>> I am now hooking in all the closed source directories as well.
>>
>> Once everything gets auto-compiled (15-20 minutes) and loaded into
>
>   What is this "auto-compilation" is actually doing? I mean, if I have
>   a fully built j2se workspace, shouldn't everything be compiled already?

Someone stop me if I say something wrong, I'll still a relative noob
(i.e. newbie, beginner, hack) in netbeans!  I'm making some best guesses
based on my experiences over the past week.

A little background first.  In normal Java projects, there is a target
platform.  You target your app to a specific JDK, and it uses the built
binaries for the Java class signatures.  As an aside, by default the
source .zip in the JDK is used for the javadoc/sources.  You can specify
an actual workspace:

     <jdk-ws-home>/src/share/classes
     <jdk-ws-home>/src/solaris/classes
     <jdk-ws-home>/src/windows/classes

and get the full javadocs for the internal classes as well.  And when
you are in the debugger, if you go to the debugger's sources window, you
can enable these directories in order to step into these methods.  VERY
HANDY!

 >   What is this "auto-compilation" is actually doing? I mean, if I have
 >   a fully built j2se workspace, shouldn't everything be compiled
 >   already?

That's what I would have thought, but that's not what I'm seeing.  My
guess is that it's because the current netbeans j2se project is set up
as a free-form project, and thus doesn't specify a target platform.
Yes, it does include a destination dir, but not a platform.  Thus, it
needs to go through all of the source directories, and compile all the
classes into signatures into:

     <$HOME>/.netbeans/.../var/cache

To avoid this massive initial first compilation, one of my first
attempts at trying to set up a project using a built JDK binary as the
target platform got really close, but ran into some type of
auto-compilation problem.  Each file I open recompiled the world.  And
it only occurred when I attached sources to other unrelated platforms.
Otherwise, it was working pretty well.

>   I find that tweaking the default heap settings (doubling them) improves
>   things. Otherwise sooner or later you'll run into OOE.

I'm running this on a server-class machine and displaying back to my
desktop, but that's not a bad idea.  I haven't hit an OOME yet.

Brad