Debug source lookup question

[Vladimir Prus-3]

Hello,
I've a couple of questions about how source lookup works with CDI/GDB.

1. In CDebugTarget.initializeSourceLookup, the list of all directories
for the ILaunch's source locator is collected, and then passed to
getCDITarget().setSourcePaths, which eventually results in GDB
-environment-directory command.

Why is this done? If I understand correctly, with properly setup
source locators CDT will be able to find the actual file even if
GDB reports filename without path. So, -environment-directory command
does not add anything. In fact, in hurts, because it causes GDB to
ignore full path information in debug info when it is present.

So, what is the intended purpose of ICDISourceManagement.setSourcePaths
method?

2. As far as I can see, the ICDISourceManagement.getSourcePaths method
is not used anywhere. Why does it exist and what (future?) uses are intended?

Thanks,
Volodya


_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Mikhail Khodjaiants-3]

Hi Volodya,

As far as I remember this functionality was added in attempt to
"synchronize" the external (CDT) source lookup with GDB. I haven't been
aware it has such an effect. What's the purpose of this command then?
Regarding getSourcePaths: it was added to get the source lookup
information from GDB and populate the CDT source lookup. But the idea
has never materialized. I am not aware of any future plans for it.

Regards,
Mikhail
-----Original Message-----
From: cdt-dev-bounces@... [mailto:cdt-dev-bounces@...]
On Behalf Of Vladimir Prus
Sent: Wednesday, November 19, 2008 2:19 PM
To: CDT list.
Subject: [cdt-dev] Debug source lookup question


Hello,
I've a couple of questions about how source lookup works with CDI/GDB.

1. In CDebugTarget.initializeSourceLookup, the list of all directories
for the ILaunch's source locator is collected, and then passed to
getCDITarget().setSourcePaths, which eventually results in GDB
-environment-directory command.

Why is this done? If I understand correctly, with properly setup source
locators CDT will be able to find the actual file even if GDB reports
filename without path. So, -environment-directory command does not add
anything. In fact, in hurts, because it causes GDB to ignore full path
information in debug info when it is present.

So, what is the intended purpose of ICDISourceManagement.setSourcePaths
method?

2. As far as I can see, the ICDISourceManagement.getSourcePaths method
is not used anywhere. Why does it exist and what (future?) uses are
intended?

Thanks,
Volodya


_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

--
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.


_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Vladimir Prus-3]

On Wednesday 19 November 2008 17:52:46 Mikhail Khodjaiants wrote:

Hi Mikhail,

> As far as I remember this functionality was added in attempt to
> "synchronize" the external (CDT) source lookup with GDB. I haven't been
> aware it has such an effect. What's the purpose of this command then?

The answer is somewhat long. Most often, binaries compiled with gcc
and typical build systems have two bits of information for every source
file:

        - the name of the file (without directory)
        - the compilation directory

The GDB search path is a list of directories, and may include special
'$cdir' element. When GDB wants to find the fullname of a file, it
checks for it each directory in the search path. If $cdir is found,
it checks in the compilation directory from the debug info.

The default search path includes $cdir already, so if you don't do
any explicit 'dir' commands only compilation directory and current
directory are searched.

The problem is that paths passed via 'dir' or '-environment-directory'
are added in front of the search path. So, it overrides the compilation
directory. And if you have file name a.c in two different directories,
and you add both those directories via -environment-directory, you're
out of luck. Note that adding $cdir in the front will not work, in
general, for example if the binary is run on a different system from where
it was compiled.  I think this is the fundamental limitation of a system
where N source directories are specified and searched in order. The best
approach is
        - Not used 'dir' at all when application is built and debugged
        on the same system
        - Use GDB path substitution mechanisms otherwise, documented at
        http://sourceware.org/gdb/current/onlinedocs/gdb_9.html#SEC63
        This is only available in recent versions of GDB.

I'm also not quite sure how this all meshes with the CDT model. Probably,
search path should not include any directories by default. Then users
would add it if all the standard mechanisms in GDB fail.

> Regarding getSourcePaths: it was added to get the source lookup
> information from GDB and populate the CDT source lookup. But the idea
> has never materialized. I am not aware of any future plans for it.

Should this maybe be dropped from source code? The only possible use case
for this is detecting when user typed 'dir' in GDB console, which does not
seem like a very important use case.

Thanks,
Volodya
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Elena Laskavaia]

Btw, if you concern is duplicate name the current recommended approach is:

- In the Source tab of launch config remove default mapping and add Absolute File Path mapping - this way gdb won't try to resolve anything
- Enable set breakpoints using full path in Debug tab


Vladimir Prus wrote:

> On Wednesday 19 November 2008 17:52:46 Mikhail Khodjaiants wrote:
>
> Hi Mikhail,
>
>> As far as I remember this functionality was added in attempt to
>> "synchronize" the external (CDT) source lookup with GDB. I haven't been
>> aware it has such an effect. What's the purpose of this command then?
>
> The answer is somewhat long. Most often, binaries compiled with gcc
> and typical build systems have two bits of information for every source
> file:
>
> - the name of the file (without directory)
> - the compilation directory
>
> The GDB search path is a list of directories, and may include special
> '$cdir' element. When GDB wants to find the fullname of a file, it
> checks for it each directory in the search path. If $cdir is found,
> it checks in the compilation directory from the debug info.
>
> The default search path includes $cdir already, so if you don't do
> any explicit 'dir' commands only compilation directory and current
> directory are searched.
>
> The problem is that paths passed via 'dir' or '-environment-directory'
> are added in front of the search path. So, it overrides the compilation
> directory. And if you have file name a.c in two different directories,
> and you add both those directories via -environment-directory, you're
> out of luck. Note that adding $cdir in the front will not work, in
> general, for example if the binary is run on a different system from where
> it was compiled.  I think this is the fundamental limitation of a system
> where N source directories are specified and searched in order. The best
> approach is
> - Not used 'dir' at all when application is built and debugged
> on the same system
> - Use GDB path substitution mechanisms otherwise, documented at
> http://sourceware.org/gdb/current/onlinedocs/gdb_9.html#SEC63
> This is only available in recent versions of GDB.
>
> I'm also not quite sure how this all meshes with the CDT model. Probably,
> search path should not include any directories by default. Then users
> would add it if all the standard mechanisms in GDB fail.
>
>> Regarding getSourcePaths: it was added to get the source lookup
>> information from GDB and populate the CDT source lookup. But the idea
>> has never materialized. I am not aware of any future plans for it.
>
> Should this maybe be dropped from source code? The only possible use case
> for this is detecting when user typed 'dir' in GDB console, which does not
> seem like a very important use case.
>
> Thanks,
> Volodya
> _______________________________________________
> cdt-dev mailing list
> cdt-dev@...
> https://dev.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Mikhail Khodjaiants-3]

Thanks Volodya. Dropping "getSourcePaths" is a long story - it's part of
CDI. There are non-GDB implementations based on CDI. Who knows, maybe
one of them is using this interface for their own purposes.

Mikhail
-----Original Message-----
From: cdt-dev-bounces@... [mailto:cdt-dev-bounces@...]
On Behalf Of Vladimir Prus
Sent: Wednesday, November 19, 2008 6:09 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Debug source lookup question

On Wednesday 19 November 2008 17:52:46 Mikhail Khodjaiants wrote:

Hi Mikhail,

> As far as I remember this functionality was added in attempt to
> "synchronize" the external (CDT) source lookup with GDB. I haven't
> been aware it has such an effect. What's the purpose of this command
then?

The answer is somewhat long. Most often, binaries compiled with gcc and
typical build systems have two bits of information for every source
file:

        - the name of the file (without directory)
        - the compilation directory

The GDB search path is a list of directories, and may include special
'$cdir' element. When GDB wants to find the fullname of a file, it
checks for it each directory in the search path. If $cdir is found, it
checks in the compilation directory from the debug info.

The default search path includes $cdir already, so if you don't do any
explicit 'dir' commands only compilation directory and current directory
are searched.

The problem is that paths passed via 'dir' or '-environment-directory'
are added in front of the search path. So, it overrides the compilation
directory. And if you have file name a.c in two different directories,
and you add both those directories via -environment-directory, you're
out of luck. Note that adding $cdir in the front will not work, in
general, for example if the binary is run on a different system from
where it was compiled.  I think this is the fundamental limitation of a
system where N source directories are specified and searched in order.
The best approach is
        - Not used 'dir' at all when application is built and debugged
        on the same system
        - Use GDB path substitution mechanisms otherwise, documented at
        http://sourceware.org/gdb/current/onlinedocs/gdb_9.html#SEC63
        This is only available in recent versions of GDB.

I'm also not quite sure how this all meshes with the CDT model.
Probably, search path should not include any directories by default.
Then users would add it if all the standard mechanisms in GDB fail.

> Regarding getSourcePaths: it was added to get the source lookup
> information from GDB and populate the CDT source lookup. But the idea
> has never materialized. I am not aware of any future plans for it.

Should this maybe be dropped from source code? The only possible use
case for this is detecting when user typed 'dir' in GDB console, which
does not seem like a very important use case.

Thanks,
Volodya
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

--
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.


_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Vladimir Prus-3]

On Wednesday 19 November 2008 21:38:45 Mikhail Khodjaiants wrote:
> Thanks Volodya. Dropping "getSourcePaths" is a long story - it's part of
> CDI. There are non-GDB implementations based on CDI. Who knows, maybe
> one of them is using this interface for their own purposes.

Surely, there must be *some* deprecation process :-)

- Volodya
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Vladimir Prus-3]

On Wednesday 19 November 2008 21:10:22 Elena Laskavaia wrote:
> Btw, if you concern is duplicate name the current recommended approach is:
>
> - In the Source tab of launch config remove default mapping and add Absolute File Path mapping - this way gdb won't try to resolve anything
> - Enable set breakpoints using full path in Debug tab

I know about both, and will likely force them for new projects. The question I'm trying
to establish -- primarily for our product, but also in general -- is whether CDT
should *ever* sent -environment-directory command.

One case where -environment-directory might still be needed is when user adds a
source directory via CDT, and then uses commands like "list" in GDB console --
which probably check the same paths. I don't know how common this use case is.

- Volodya
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[James Blackburn-2]

> I know about both, and will likely force them for new projects. The question I'm trying
> to establish -- primarily for our product, but also in general -- is whether CDT
> should *ever* sent -environment-directory command.
>
> One case where -environment-directory might still be needed is when user adds a
> source directory via CDT, and then uses commands like "list" in GDB console --
> which probably check the same paths. I don't know how common this use case is.

That's the only case I can think of as well.

For what it's worth, our target's launch only uses the absolute path
source lookup as well.  The reason I did this was the very long launch
startup-time we were seeing when discovering all the project paths to
pass to GDB (we run our Eclipses over NFS filesystems...).  See bug:
https://bugs.eclipse.org/225708

There are a number of peculiarities to do with launch and source
paths.  Previously, for example, the debugger would not attempt to set
breakpoints if the source location of the breakpoint couldn't be
resolved via the source lookup mechanism.  Elena committed a patch to
fix this (breakpoints are now set if the marker's IResource is in the
project) -- bug: https://bugs.eclipse.org/247851 .  Source lookup is
still used as a backup (which, as the absolute path container always
resolves the file, is probably wrong...).  I'm not sure either of use
figure out why the breakpoints are resolved via the source lookup
mechanism in the first place.  Mikhail perhaps you can shed some light
on this?

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Mikhail Khodjaiants-3]

James,

Currently the CDT source lookup mechanism doesn't retrieve the source
information from GDB. Which means that the CDT debugger is unable in
many cases to associate the file paths known to CDT to the paths stored
in the debug information in GDB. As a result we don't use absolute paths
to set breakpoints. (There is a new option added by Elena that allows to
use absolute paths. This option only works if the debug information
contains only absolute paths but will cause more problems in many other
cases.)
As we use only file names we need a mechanism to filter breakpoints that
don't "belong" to the session. Note that CDT also support breakpoints
set on external files. External files have no associated Eclipse
resources and therefore can not be filtered according their project
affiliation.
I am convinced the source lookup and the breakpoint setting mechanism in
CDT should work "in sync". The right approach is to generate the content
of the source lookup from the debug information stored in GDB and then
allow users to map it to the file system. In this case we can translate
absolute paths to the paths known to GDB.
As for -environment-directory command, Volodya is right, CDT shouldn't
use it.

Regards,
Mikhail
-----Original Message-----
From: cdt-dev-bounces@... [mailto:cdt-dev-bounces@...]
On Behalf Of James Blackburn
Sent: Wednesday, November 19, 2008 7:22 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Debug source lookup question

> I know about both, and will likely force them for new projects. The
> question I'm trying to establish -- primarily for our product, but
> also in general -- is whether CDT should *ever* sent
-environment-directory command.
>
> One case where -environment-directory might still be needed is when
> user adds a source directory via CDT, and then uses commands like
> "list" in GDB console -- which probably check the same paths. I don't
know how common this use case is.

That's the only case I can think of as well.

For what it's worth, our target's launch only uses the absolute path
source lookup as well.  The reason I did this was the very long launch
startup-time we were seeing when discovering all the project paths to
pass to GDB (we run our Eclipses over NFS filesystems...).  See bug:
https://bugs.eclipse.org/225708

There are a number of peculiarities to do with launch and source paths.
Previously, for example, the debugger would not attempt to set
breakpoints if the source location of the breakpoint couldn't be
resolved via the source lookup mechanism.  Elena committed a patch to
fix this (breakpoints are now set if the marker's IResource is in the
project) -- bug: https://bugs.eclipse.org/247851 .  Source lookup is
still used as a backup (which, as the absolute path container always
resolves the file, is probably wrong...).  I'm not sure either of use
figure out why the breakpoints are resolved via the source lookup
mechanism in the first place.  Mikhail perhaps you can shed some light
on this?

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

--
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.


_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[James Blackburn-2]

Thanks for the reply Mikhail.

> As we use only file names we need a mechanism to filter breakpoints that
> don't "belong" to the session. Note that CDT also support breakpoints
> set on external files. External files have no associated Eclipse
> resources and therefore can not be filtered according their project
> affiliation.

I get that we need to filter breakpoints by
resource-in-the-project-being-debugged.  My point was that this
feature currently doens't work and I believe there's good reason not
to be using the source locators to work out if a breakpoint should be
enabled.

Eclipse IMarkers are associated with the IResources.  In your example
of an external source file, the breakpoint is set on the IProject (or
for external translation units without context they're set on the
WorkspaceRoot).  Now on launch we can perform a quick check to see if
the breakpoint marker is in the project being launched (see:
CBreakpointManager.isTargetBreakpoint()).  However if this check
fails, i.e. the marker is the workspaceroot, or set on a different
project, we fall back to using the source locators.

Most (if not all?) breakpoints set in CDT will have a marker attribute
ICBreakpoint.SOURCE_HANDLE equal to the source location.  Given that
AbsolutePathSourceContainer is used by the default c source lookup
participant, all breakpoints will automatically be resolved and set
during the launch.

> I am convinced the source lookup and the breakpoint setting mechanism in
> CDT should work "in sync". The right approach is to generate the content
> of the source lookup from the debug information stored in GDB and then
> allow users to map it to the file system. In this case we can translate
> absolute paths to the paths known to GDB.
> As for -environment-directory command, Volodya is right, CDT shouldn't
> use it.

I think that source lookup and breakpoint setting shouldn't be "in
sync" because of the above 'bug'.

The job of the source locator is to show you 'correct' source when CDT
is notified that the debugger has stopped at a particular location.
We don't care if the debugger stops in a file which maps to an
IResource in the debuggee project, a different project, or anywhere
else in the filesystem -- we still want to see some source (+/-
allowing the user to change the source lookup order they see by
default).
In my view this is in contrast to the breakpoints mechanism which
should be attempting to filter breakpoints based on the launch
configuration context and the context of the breakpoint markers.

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[James Blackburn-2]

On Thu, Nov 20, 2008 at 11:55 AM, James Blackburn
<jamesblackburn@...> wrote:
>  Given that
> AbsolutePathSourceContainer is used by the default c source lookup
> participant, all breakpoints will automatically be resolved and set
> during the launch.

You can see this by using a recent (since 4.0?) CDT.  Enable verbose
console mode, set a few breakpoints in your workspace, and launch.  If
you have the AbsolutePathSourContainer enabled, you will see cdt
attempt to set all the breakpoints in the workspace.

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Mikhail Khodjaiants-3]

James,

I can't comment on the latest changes - haven't worked on or used
seriously a CDT/GDB based debugger for more than a year. If there is a
problem with AbsolutePathSourceContainer it should be fixed.
But I still think the synchronization with the source lookup is
required. If executable is compiled with "/foo/../main.c" setting a
breakpoint at "/foo/main.c" wouldn't work. And the source lookup is the
place where we can resolve this type of issues.

Mikhail
-----Original Message-----
From: cdt-dev-bounces@... [mailto:cdt-dev-bounces@...]
On Behalf Of James Blackburn
Sent: Thursday, November 20, 2008 11:56 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Debug source lookup question

Thanks for the reply Mikhail.

> As we use only file names we need a mechanism to filter breakpoints
> that don't "belong" to the session. Note that CDT also support
> breakpoints set on external files. External files have no associated
> Eclipse resources and therefore can not be filtered according their
> project affiliation.

I get that we need to filter breakpoints by
resource-in-the-project-being-debugged.  My point was that this feature
currently doens't work and I believe there's good reason not to be using
the source locators to work out if a breakpoint should be enabled.

Eclipse IMarkers are associated with the IResources.  In your example of
an external source file, the breakpoint is set on the IProject (or for
external translation units without context they're set on the
WorkspaceRoot).  Now on launch we can perform a quick check to see if
the breakpoint marker is in the project being launched (see:
CBreakpointManager.isTargetBreakpoint()).  However if this check fails,
i.e. the marker is the workspaceroot, or set on a different project, we
fall back to using the source locators.

Most (if not all?) breakpoints set in CDT will have a marker attribute
ICBreakpoint.SOURCE_HANDLE equal to the source location.  Given that
AbsolutePathSourceContainer is used by the default c source lookup
participant, all breakpoints will automatically be resolved and set
during the launch.

> I am convinced the source lookup and the breakpoint setting mechanism
> in CDT should work "in sync". The right approach is to generate the
> content of the source lookup from the debug information stored in GDB
> and then allow users to map it to the file system. In this case we can

> translate absolute paths to the paths known to GDB.
> As for -environment-directory command, Volodya is right, CDT shouldn't

> use it.

I think that source lookup and breakpoint setting shouldn't be "in sync"
because of the above 'bug'.

The job of the source locator is to show you 'correct' source when CDT
is notified that the debugger has stopped at a particular location.
We don't care if the debugger stops in a file which maps to an IResource
in the debuggee project, a different project, or anywhere else in the
filesystem -- we still want to see some source (+/- allowing the user to
change the source lookup order they see by default).
In my view this is in contrast to the breakpoints mechanism which should
be attempting to filter breakpoints based on the launch configuration
context and the context of the breakpoint markers.

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

--
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.


_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[James Blackburn-2]

Hi Mikhail,

Not meaning to labour the point, but I think we have two separate issues here.

> I can't comment on the latest changes - haven't worked on or used
> seriously a CDT/GDB based debugger for more than a year. If there is a
> problem with AbsolutePathSourceContainer it should be fixed.

I don't think there is a problem with it.  If execution stops at some
external absolute file path -- for example the user steps into a
library function (where the library has debug symbols) they should see
the source.  It would be overkill to force the user to add all
external source locations to the launch config "Source" tab.  For this
AbsolutePathSourceContainer does what it says on the tin.

> But I still think the synchronization with the source lookup is
> required. If executable is compiled with "/foo/../main.c" setting a
> breakpoint at "/foo/main.c" wouldn't work. And the source lookup is the
> place where we can resolve this type of issues.

I see there is a problem here, but it's not one the source locator solves:
  - If both main.c's were in different projects, marker comparison
would spot this.
  - If they are in the same project, and not both built into the same
executable, your default Project Path Container would also fail --
there's a collision here and both exist in the source container.
  - If both main.c's are in the same project and built into the same
executable then you're scuppered if you don't select "Use full file
path to set breakpoints" as the breakpoints aren't disambiguated
before being set (cdt just uses main.c:xx as discussed previously).

A better solution might be to look in the binary and check whether
there are any source file collisions before attempting to set
breakpoints.

>From my point of view, if you make the source locators too specific
(so breakpoints are filtered) then when the target stops you risk not
finding the source.  If you make it too general -- allowing it to
resolve all source on a best effort basis -- then all breakpoints are
set on launch.

Even though both features are based on sets of source files: the set
of files with breakpoints that will be set on launch,  is contained
within the set of source file locations we can resolve in the IDE, and
not vice versa.  (Obviously the sets are equivalent if we know exactly
what files are built into an executable, but this isn't what the
source locator provides.)

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Mikhail Khodjaiants-3]

Sorry James, my example is not correct. The correct paths are
"/project/foo/../main.c" and "/project/main.c". The point is that both
represent the same file, but from the GDB's point of view there are
different.

> A better solution might be to look in the binary and check whether
there are any source file collisions before attempting to set
breakpoints.

That's what I am proposing, but instead of parsing the binary, we let
GDB to do the work and retrieve the list of source files and their paths
from GDB. In this case we can use absolute paths to set breakpoints. If
we need to set a breakpoint at "/project/main.c" the source lookup will
translate the path to "/project/foo/../main.c" and GDB will understand
it.
With absolute paths we won't have file collisions.

Mikhail

-----Original Message-----
From: cdt-dev-bounces@... [mailto:cdt-dev-bounces@...]
On Behalf Of James Blackburn
Sent: Thursday, November 20, 2008 1:07 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Debug source lookup question

Hi Mikhail,

Not meaning to labour the point, but I think we have two separate issues
here.

> I can't comment on the latest changes - haven't worked on or used
> seriously a CDT/GDB based debugger for more than a year. If there is a

> problem with AbsolutePathSourceContainer it should be fixed.

I don't think there is a problem with it.  If execution stops at some
external absolute file path -- for example the user steps into a library
function (where the library has debug symbols) they should see the
source.  It would be overkill to force the user to add all external
source locations to the launch config "Source" tab.  For this
AbsolutePathSourceContainer does what it says on the tin.

> But I still think the synchronization with the source lookup is
> required. If executable is compiled with "/foo/../main.c" setting a
> breakpoint at "/foo/main.c" wouldn't work. And the source lookup is
> the place where we can resolve this type of issues.

I see there is a problem here, but it's not one the source locator
solves:
  - If both main.c's were in different projects, marker comparison would
spot this.
  - If they are in the same project, and not both built into the same
executable, your default Project Path Container would also fail --
there's a collision here and both exist in the source container.
  - If both main.c's are in the same project and built into the same
executable then you're scuppered if you don't select "Use full file path
to set breakpoints" as the breakpoints aren't disambiguated before being
set (cdt just uses main.c:xx as discussed previously).

A better solution might be to look in the binary and check whether there
are any source file collisions before attempting to set breakpoints.

>From my point of view, if you make the source locators too specific
(so breakpoints are filtered) then when the target stops you risk not
finding the source.  If you make it too general -- allowing it to
resolve all source on a best effort basis -- then all breakpoints are
set on launch.

Even though both features are based on sets of source files: the set of
files with breakpoints that will be set on launch,  is contained within
the set of source file locations we can resolve in the IDE, and not vice
versa.  (Obviously the sets are equivalent if we know exactly what files
are built into an executable, but this isn't what the source locator
provides.)

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

--
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.


_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Pawel Piech]

Hi,

I find this conversation very interesting as sourfilce lookup and breakpoint filtering (or scope) are some of the most persistent usability problems for our customers.  I hope that eventually we can pull together our resources and come up with something that really works.

In the Wind River product, the idea behind source lookup logic is essentially identical to that of CDT.  We have a path substitution source container and if the user maps his sources using this container, we will send a full path to the debugger.  If user configures a search source container (e.g. Project, Folder, etc.), we will send just the filename to the debugger.  Unfortunately, the difference between these two is usually too subtle for users and they get into trouble.  We have a few UI additions to help the user identify when the source lookup is causing breakpoints to be planted on the wrong file, but this also is not enough.  So I'm very curious if there's any new ideas out there on how to deal with this problem.

On breakpoint filtering (or scope as we call it), I've never even considered using project or other information to automatically scope breakpoints to a given debug session.  Instead our breakpoints have an explicit parameter that specifies which launch configurations a given BP applies to.  This requires an explicit user action to specify the scope, but in practice the it is set when the breakpoint is created based on the currently active debug session.  It's a simple solution, but obviously not fool-proof. 

Using source path information in the executable to determine the breakpoint scope sounds like a neat idea, but I don't think it would scale.  Just like parsing through the project structure to generate -environment-directory commands takes a long time, processing all the paths in a 200MB executable image would be very painful.

-Pawel

Mikhail Khodjaiants wrote:

Sorry James, my example is not correct. The correct paths are
"/project/foo/../main.c" and "/project/main.c". The point is that both
represent the same file, but from the GDB's point of view there are
different.

  

A better solution might be to look in the binary and check whether
    

there are any source file collisions before attempting to set
breakpoints.

That's what I am proposing, but instead of parsing the binary, we let
GDB to do the work and retrieve the list of source files and their paths
from GDB. In this case we can use absolute paths to set breakpoints. If
we need to set a breakpoint at "/project/main.c" the source lookup will
translate the path to "/project/foo/../main.c" and GDB will understand
it.
With absolute paths we won't have file collisions.

Mikhail

-----Original Message-----
From: cdt-dev-bounces@... [cdt-dev-bounces@...]
On Behalf Of James Blackburn
Sent: Thursday, November 20, 2008 1:07 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Debug source lookup question

Hi Mikhail,

Not meaning to labour the point, but I think we have two separate issues
here.

  

I can't comment on the latest changes - haven't worked on or used 
seriously a CDT/GDB based debugger for more than a year. If there is a
    

problem with AbsolutePathSourceContainer it should be fixed.
    

I don't think there is a problem with it.  If execution stops at some
external absolute file path -- for example the user steps into a library
function (where the library has debug symbols) they should see the
source.  It would be overkill to force the user to add all external
source locations to the launch config "Source" tab.  For this
AbsolutePathSourceContainer does what it says on the tin.

  

But I still think the synchronization with the source lookup is 
required. If executable is compiled with "/foo/../main.c" setting a 
breakpoint at "/foo/main.c" wouldn't work. And the source lookup is 
the place where we can resolve this type of issues.
    

I see there is a problem here, but it's not one the source locator
solves:
  - If both main.c's were in different projects, marker comparison would
spot this.
  - If they are in the same project, and not both built into the same
executable, your default Project Path Container would also fail --
there's a collision here and both exist in the source container.
  - If both main.c's are in the same project and built into the same
executable then you're scuppered if you don't select "Use full file path
to set breakpoints" as the breakpoints aren't disambiguated before being
set (cdt just uses main.c:xx as discussed previously).

A better solution might be to look in the binary and check whether there
are any source file collisions before attempting to set breakpoints.

>From my point of view, if you make the source locators too specific
(so breakpoints are filtered) then when the target stops you risk not
finding the source.  If you make it too general -- allowing it to
resolve all source on a best effort basis -- then all breakpoints are
set on launch.

Even though both features are based on sets of source files: the set of
files with breakpoints that will be set on launch,  is contained within
the set of source file locations we can resolve in the IDE, and not vice
versa.  (Obviously the sets are equivalent if we know exactly what files
are built into an executable, but this isn't what the source locator
provides.)

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

  

_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Elena Laskavaia]

I guess it is a consensus that we can get rid of sending -environment-directory command by default.
We can add a option to source lookup panel (can we?) that enable this behavior if needed.
Vladimir, do you want to send a patch for it?

James Blackburn wrote:

>> I know about both, and will likely force them for new projects. The question I'm trying
>> to establish -- primarily for our product, but also in general -- is whether CDT
>> should *ever* sent -environment-directory command.
>>
>> One case where -environment-directory might still be needed is when user adds a
>> source directory via CDT, and then uses commands like "list" in GDB console --
>> which probably check the same paths. I don't know how common this use case is.
>
> That's the only case I can think of as well.
>
> For what it's worth, our target's launch only uses the absolute path
> source lookup as well.  The reason I did this was the very long launch
> startup-time we were seeing when discovering all the project paths to
> pass to GDB (we run our Eclipses over NFS filesystems...).  See bug:
> https://bugs.eclipse.org/225708
>
> There are a number of peculiarities to do with launch and source
> paths.  Previously, for example, the debugger would not attempt to set
> breakpoints if the source location of the breakpoint couldn't be
> resolved via the source lookup mechanism.  Elena committed a patch to
> fix this (breakpoints are now set if the marker's IResource is in the
> project) -- bug: https://bugs.eclipse.org/247851 .  Source lookup is
> still used as a backup (which, as the absolute path container always
> resolves the file, is probably wrong...).  I'm not sure either of use
> figure out why the breakpoints are resolved via the source lookup
> mechanism in the first place.  Mikhail perhaps you can shed some light
> on this?
>
> Cheers,
>
> James
> _______________________________________________
> cdt-dev mailing list
> cdt-dev@...
> https://dev.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Mikhail Khodjaiants-3]

> Using source path information in the executable to determine the breakpoint scope sounds like a neat idea, but I don't think it would scale.

 

Yes, the scalability is the only reason why it hasn't been implemented. But I still believe that provided with a better support from GDB it could be done. 

At the same time if GDB could solve the source lookup problems internally we would only need to perform the external source lookup if the debug information was not sufficient.

 

Mikhail

---

From: cdt-dev-bounces@... [mailto:cdt-dev-bounces@...] On Behalf Of Pawel Piech
Sent: Thursday, November 20, 2008 5:24 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Debug source lookup question

Hi,

I find this conversation very interesting as sourfilce lookup and breakpoint filtering (or scope) are some of the most persistent usability problems for our customers.  I hope that eventually we can pull together our resources and come up with something that really works.

In the Wind River product, the idea behind source lookup logic is essentially identical to that of CDT.  We have a path substitution source container and if the user maps his sources using this container, we will send a full path to the debugger.  If user configures a search source container (e.g. Project, Folder, etc.), we will send just the filename to the debugger.  Unfortunately, the difference between these two is usually too subtle for users and they get into trouble.  We have a few UI additions to help the user identify when the source lookup is causing breakpoints to be planted on the wrong file, but this also is not enough.  So I'm very curious if there's any new ideas out there on how to deal with this problem.

On breakpoint filtering (or scope as we call it), I've never even considered using project or other information to automatically scope breakpoints to a given debug session.  Instead our breakpoints have an explicit parameter that specifies which launch configurations a given BP applies to.  This requires an explicit user action to specify the scope, but in practice the it is set when the breakpoint is created based on the currently active debug session.  It's a simple solution, but obviously not fool-proof. 

Using source path information in the executable to determine the breakpoint scope sounds like a neat idea, but I don't think it would scale.  Just like parsing through the project structure to generate -environment-directory commands takes a long time, processing all the paths in a 200MB executable image would be very painful.

-Pawel

Mikhail Khodjaiants wrote:

Sorry James, my example is not correct. The correct paths are
"/project/foo/../main.c" and "/project/main.c". The point is that both
represent the same file, but from the GDB's point of view there are
different.

  

A better solution might be to look in the binary and check whether
    

there are any source file collisions before attempting to set
breakpoints.

That's what I am proposing, but instead of parsing the binary, we let
GDB to do the work and retrieve the list of source files and their paths
from GDB. In this case we can use absolute paths to set breakpoints. If
we need to set a breakpoint at "/project/main.c" the source lookup will
translate the path to "/project/foo/../main.c" and GDB will understand
it.
With absolute paths we won't have file collisions.

Mikhail

-----Original Message-----
From: cdt-dev-bounces@... [cdt-dev-bounces@...]
On Behalf Of James Blackburn
Sent: Thursday, November 20, 2008 1:07 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] Debug source lookup question

Hi Mikhail,

Not meaning to labour the point, but I think we have two separate issues
here.

  

I can't comment on the latest changes - haven't worked on or used 
seriously a CDT/GDB based debugger for more than a year. If there is a
    

problem with AbsolutePathSourceContainer it should be fixed.
    

I don't think there is a problem with it.  If execution stops at some
external absolute file path -- for example the user steps into a library
function (where the library has debug symbols) they should see the
source.  It would be overkill to force the user to add all external
source locations to the launch config "Source" tab.  For this
AbsolutePathSourceContainer does what it says on the tin.

  

But I still think the synchronization with the source lookup is 
required. If executable is compiled with "/foo/../main.c" setting a 
breakpoint at "/foo/main.c" wouldn't work. And the source lookup is 
the place where we can resolve this type of issues.
    

I see there is a problem here, but it's not one the source locator
solves:
  - If both main.c's were in different projects, marker comparison would
spot this.
  - If they are in the same project, and not both built into the same
executable, your default Project Path Container would also fail --
there's a collision here and both exist in the source container.
  - If both main.c's are in the same project and built into the same
executable then you're scuppered if you don't select "Use full file path
to set breakpoints" as the breakpoints aren't disambiguated before being
set (cdt just uses main.c:xx as discussed previously).

A better solution might be to look in the binary and check whether there
are any source file collisions before attempting to set breakpoints.

>From my point of view, if you make the source locators too specific
(so breakpoints are filtered) then when the target stops you risk not
finding the source.  If you make it too general -- allowing it to
resolve all source on a best effort basis -- then all breakpoints are
set on launch.

Even though both features are based on sets of source files: the set of
files with breakpoints that will be set on launch,  is contained within
the set of source file locations we can resolve in the IDE, and not vice
versa.  (Obviously the sets are equivalent if we know exactly what files
are built into an executable, but this isn't what the source locator
provides.)

Cheers,

James
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

  

--

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.

_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Daniel Jacobowitz-2]

On Thu, Nov 20, 2008 at 06:11:05PM -0000, Mikhail Khodjaiants wrote:
> At the same time if GDB could solve the source lookup problems
> internally we would only need to perform the external source lookup if
> the debug information was not sufficient.

Source lookup is an environment issue by definition.  For GDB to get
it right, you have to give it adequate and correct information.
What this means in practice, I think, is (A) not dumping a pile of
-environment-directory commands at GDB, and (B) offering an
interface to the source translation mechanism in recent versions
of GDB (set substitute-path).  I haven't thought about it much,
but it seems to me that the ideal interface for this would require
a file browser - for instance, have the user locate one source file
manually and then figure out how many directory levels up to set
substitute-path (containing directory?  grandparent directory?).

--
Daniel Jacobowitz
CodeSourcery
_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Vladimir Prus-3]

On Thursday 20 November 2008 17:07:10 Mikhail Khodjaiants wrote:
> Sorry James, my example is not correct. The correct paths are
> "/project/foo/../main.c" and "/project/main.c". The point is that both
> represent the same file, but from the GDB's point of view there are
> different.

Not necessary. On Linux, GDB uses realpath, so it does not care much
how you specify the file. On Windows, this is not the case now -- I've posted
a patch to fix it, but it's not in GDB CVS.

>
> > A better solution might be to look in the binary and check whether
>
> there are any source file collisions before attempting to set
> breakpoints.
>
> That's what I am proposing, but instead of parsing the binary, we let
> GDB to do the work and retrieve the list of source files and their paths
> from GDB. In this case we can use absolute paths to set breakpoints. If
> we need to set a breakpoint at "/project/main.c" the source lookup will
> translate the path to "/project/foo/../main.c" and GDB will understand
> it.

I'm getting confused. What problem is solved by getting the list of
source files from GDB? Assuming the GDB issue with full paths on Windows is
fixed, you can just *always* send full path when setting a breakpoint.

Using the list of source paths from GDB to filter breakpoints does not seem
necessary, either -- if GDB does not know about a source file, it won't set
a breakpoint -- there's no need to do such filtering on CDT side. Note also
that in presense of shared libraries, including on-demand loaded ones, the
list of source paths becomes undefined notion.

- Volodya

_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev

[Vladimir Prus-3]

On Thursday 20 November 2008 20:37:57 Elena Laskavaia wrote:
> I guess it is a consensus that we can get rid of sending
> -environment-directory command by default. We can add a option to source
> lookup panel (can we?) that enable this behavior if needed. Vladimir, do
> you want to send a patch for it?

The question, I think, is whether we should *ever* send -environment-
directory. For all MI commands, if GDB does not report full path, CDT can
search for the file itself, so there's no need to send -environment-directory.
For CLI commands, sending -environment-directory might help the user,
but given that it's broken whenever there are two files with the same name,
it's probably best to err on the safe  side. That is, user much prefer to see
a.cpp in CLI command output, than see a wrong full path to a.cpp.

- Volodya

_______________________________________________
cdt-dev mailing list
cdt-dev@...
https://dev.eclipse.org/mailman/listinfo/cdt-dev