Completions in Semantic

[Chong Yidong]

Hi Eric,

The completion code implemented by the semantic/complete package appears
to be quirky.  For instance:

 emacs
 M-x semantic-mode RET
 C-x C-f ~/emacs/src/xdisp.c   [visit xdisp.c in in the Emacs tree]
 C-c , j
 window[TAB]

Semantic now displays a *Completions* window with the following
contents:

 Possible completions are:
 window

But this message is misleading, because that's definitely not the only
completion.  Type:

 _[TAB]

and the *Completions* window is updated with

 Possible completions are:
 window_text_bottom_y window_box_width window_box_height

On a more general note, this is one of the drawbacks to the approach
taken in semantic/complete.el, i.e. trying to reimplement completion.
We've painstakingly ironed out a lot of bugs in the standard Emacs
completion code, and it makes no sense to redo all that for Semantic
unless there's a good reason.  (Not to mention inconsistency issues).

In the commentary to semantic/complete.el, you write that using
`all-completions' and `try-completion' isn't practical because they are
too slow when there are large numbers of tags.  How sure are you that
that's true?  Nowadays, the completions code is very flexible; for
instance, you can pass `all-completions' and `try-completion' a
collector function instead of a flat completion table.  Can't
semantic/complete.el make use of this functionality?

[Miles Bader-2]

Chong Yidong <cyd@...> writes:
> On a more general note, this is one of the drawbacks to the approach
> taken in semantic/complete.el, i.e. trying to reimplement completion.
> We've painstakingly ironed out a lot of bugs in the standard Emacs
> completion code, and it makes no sense to redo all that for Semantic
> unless there's a good reason.  (Not to mention inconsistency issues).

Yes, and it won't properly respect user customizations of the completion
stuff.

[I've often felt that semantic was acting "weird" during completion, but
it's hard to pin down exactly what the problem is (by the time I realize
that it is, I often can't reproduce the offending key sequence).]

-Miles

--
Vote, v. The instrument and symbol of a freeman's power to make a fool of
himself and a wreck of his country.

[Eric M. Ludlam]

Hi,

You are right, it is a bit quirky.  The comment in my code about using
try-completion being slow isn't about try-completion itself.  It is
about Semantic converting its table set into something a more normal
completion prompt can use.  For a large code base I work on, it used to
take about 8 minutes.  (I profiled it to make sure it wasn't hung, and
that also slows things down a bunch, so maybe 1 minute unprofiled?)  The
semantic/completion code takes a shortest path approach with what you
type, keeping track of old completion lists and tables and refining it
as you go.  I don't doubt that there's some funny stuff there that
provides the quirkyness.  In your example, I think it's just because
"window" is an exact match.

If you look at semantic-read-symbol, it does what you suggest, and
builds a table from the current buffer, and passes the whole thing into
completing-read.  That's ok for a single buffer, but not when you want
some symbol from anywhere in your project.  Of course, there is a basic
assumption that the user will do something like your example below,
typing in some prefix, before pressing TAB.  If the prompt comes up and
the user presses TAB right away, then there is no win.

Anyway, semantic-read-symbol works fine unless there are multiple
symbols with the same name, thus, a second thing that made me write my
own completion code is that tags are a little special, since they have
both names, and spacial data.  Sometimes, two tags are only different
based on their spacial data.  For example, when jumping to a tag, if
there are many hits with the same name (such as "window" in your example
below), you can press TAB several times, and it will flash where you
would jump to if you hit return, allowing you to differentiate.

For your particular example in xdisp.c, it seems the contents of xdisp.c
fails to parse properly.  It seems to be related to the P_ macro, and
one of the uses of it involving a line-end in the middle.  I've narrowed
it down and will see if I can fix it.

The long and short of all that is, if someone wants to take on a better
(consistent) completion prompt, that'd be great, and I'd be happy to do
the back-end side of things as needed.  (When writing the semantic
completion system, I discovered that I don't really like writing
completion prompts.)  The API you describe sounds (in brief) sufficient
to do some of what is needed.  In the meantime, the current version
works for me going back to Emacs 21, and XEmacs, so I'll stick with it
till I can get my current release out.  After that I'd want to focus on
more recent Emacsen and simplifying that stuff sounds like a fine idea.

Eric

On Sun, 2009-10-18 at 19:00 -0400, Chong Yidong wrote:

> Hi Eric,
>
> The completion code implemented by the semantic/complete package appears
> to be quirky.  For instance:
>
>  emacs
>  M-x semantic-mode RET
>  C-x C-f ~/emacs/src/xdisp.c   [visit xdisp.c in in the Emacs tree]
>  C-c , j
>  window[TAB]
>
> Semantic now displays a *Completions* window with the following
> contents:
>
>  Possible completions are:
>  window
>
> But this message is misleading, because that's definitely not the only
> completion.  Type:
>
>  _[TAB]
>
> and the *Completions* window is updated with
>
>  Possible completions are:
>  window_text_bottom_y window_box_width window_box_height
>
> On a more general note, this is one of the drawbacks to the approach
> taken in semantic/complete.el, i.e. trying to reimplement completion.
> We've painstakingly ironed out a lot of bugs in the standard Emacs
> completion code, and it makes no sense to redo all that for Semantic
> unless there's a good reason.  (Not to mention inconsistency issues).
>
> In the commentary to semantic/complete.el, you write that using
> `all-completions' and `try-completion' isn't practical because they are
> too slow when there are large numbers of tags.  How sure are you that
> that's true?  Nowadays, the completions code is very flexible; for
> instance, you can pass `all-completions' and `try-completion' a
> collector function instead of a flat completion table.  Can't
> semantic/complete.el make use of this functionality?

[Stefan Monnier]

> If you look at semantic-read-symbol, it does what you suggest, and
> builds a table from the current buffer, and passes the whole thing into
> completing-read.  That's ok for a single buffer, but not when you want
> some symbol from anywhere in your project.  Of course, there is a basic
> assumption that the user will do something like your example below,
> typing in some prefix, before pressing TAB.  If the prompt comes up and
> the user presses TAB right away, then there is no win.

I'm not sure I understand what you're saying, but it seems what you
describe can be done as well with the "standard" completion mechanism,
since the completion-table can be a function.

E.g. Info-complete-menu-item builds the table dynamically, using the
prefix to try and build a smaller table, and reusing the previously
built table if the completion is in the same context as the last one and
the earlier prefix is a prefix of the new one.

> Anyway, semantic-read-symbol works fine unless there are multiple
> symbols with the same name, thus, a second thing that made me write my
> own completion code is that tags are a little special, since they have
> both names, and spacial data.  Sometimes, two tags are only different
> based on their spacial data.  For example, when jumping to a tag, if
> there are many hits with the same name (such as "window" in your example
> below), you can press TAB several times, and it will flash where you
> would jump to if you hit return, allowing you to differentiate.

That would require some custom completion code, indeed, since the
current completion does not offer any hook for such things.  Of course,
we could add such a hook, but we'd need to figure out where/how to
do it.


        Stefan

[Eric M. Ludlam]

On Mon, 2009-10-19 at 09:48 -0400, Stefan Monnier wrote:

> > If you look at semantic-read-symbol, it does what you suggest, and
> > builds a table from the current buffer, and passes the whole thing into
> > completing-read.  That's ok for a single buffer, but not when you want
> > some symbol from anywhere in your project.  Of course, there is a basic
> > assumption that the user will do something like your example below,
> > typing in some prefix, before pressing TAB.  If the prompt comes up and
> > the user presses TAB right away, then there is no win.
>
> I'm not sure I understand what you're saying, but it seems what you
> describe can be done as well with the "standard" completion mechanism,
> since the completion-table can be a function.
>
> E.g. Info-complete-menu-item builds the table dynamically, using the
> prefix to try and build a smaller table, and reusing the previously
> built table if the completion is in the same context as the last one and
> the earlier prefix is a prefix of the new one.

I agree.  It just wasn't available when I wrote that stuff against Emacs
21.  (2003?)

> > Anyway, semantic-read-symbol works fine unless there are multiple
> > symbols with the same name, thus, a second thing that made me write my
> > own completion code is that tags are a little special, since they have
> > both names, and spacial data.  Sometimes, two tags are only different
> > based on their spacial data.  For example, when jumping to a tag, if
> > there are many hits with the same name (such as "window" in your example
> > below), you can press TAB several times, and it will flash where you
> > would jump to if you hit return, allowing you to differentiate.
>
> That would require some custom completion code, indeed, since the
> current completion does not offer any hook for such things.  Of course,
> we could add such a hook, but we'd need to figure out where/how to
> do it.

There are a lot of interesting completing completion engines out there
on Emacswiki.  I would expect that a discussion on this topic would
invite a lot of opinions from different Emacs hackers.

Here are some of the things that could be discussed as worthwhile in the
built-in completion system:

* Differentiate between similarly named entities (as above).
  I called it a 'focus', meaning the user was focusing on 1 of many
  options, but has not chosen it.
* Completion list with different text from what shows up on the prompt
  (another way to differentiate.)
* Inline completion in a buffer.  (ie - keymap, recursive-edit,
  minibuffer like, but in the buffer text with the intention of
  inserting that text..)
* Option for using a CLOS object as an input to these tools.
  (ie - completion classes inherit from some interface.)
  as a way of managing persistent data between completion events.
  Just a favorite of mine, I'm sure.

Thanks
Eric

[Stefan Monnier]

>> E.g. Info-complete-menu-item builds the table dynamically, using the
>> prefix to try and build a smaller table, and reusing the previously
>> built table if the completion is in the same context as the last one and
>> the earlier prefix is a prefix of the new one.

> I agree.  It just wasn't available when I wrote that stuff against Emacs
> 21.  (2003?)

Not that it matters, but the version of minibuf.c from "Sun 1993-05-23
03:32:05 +0000" says:

   DEFUN ("try-completion", Ftry_completion, Stry_completion, 2, 3, 0,
   [...]
   ALIST can also be a function to do the completion itself.\n\
   It receives three arguments: the values STRING, PREDICATE and nil.\n\
   Whatever it returns becomes the value of `try-completion'.\n\

> * Inline completion in a buffer.  (ie - keymap, recursive-edit,
>   minibuffer like, but in the buffer text with the intention of
>   inserting that text..)

You mean like lisp-complete-symbol but modal?

> * Option for using a CLOS object as an input to these tools.
>   (ie - completion classes inherit from some interface.)
>   as a way of managing persistent data between completion events.
>   Just a favorite of mine, I'm sure.

The functions that you can pass to try-completion and friends are
basically objcts: the 3rd argument is the method to call.  So it should
be trivial to write a wrapper that takes a CLOS object obeying
a completion-interface and returns a lambda expression suitable for
try-completion and friends.


        Stefan

[Eric M. Ludlam]

On Mon, 2009-10-19 at 14:29 -0400, Stefan Monnier wrote:

> > * Inline completion in a buffer.  (ie - keymap, recursive-edit,
> >   minibuffer like, but in the buffer text with the intention of
> >   inserting that text..)
>
> You mean like lisp-complete-symbol but modal?

Yes, that's what I meant, but in light of your simplified description, I
suppose it doesn't matter much.  If there was a single function like
'completing-read' that took arguments to control what proposed text
after the cursor was, I suppose it wouldn't matter what happened
underneath.  I used overlays and overlay keymaps.

You could try semantic-complete-analyze-inline on something that needs
completing to see the effect I was going for.  It should work for Lisp
or C code.

I would expect designing such a thing would be challenging since the
rules for what is before the cursor is a bit vague and language
specific, so it might belong in a tool like Semantic.

> > * Option for using a CLOS object as an input to these tools.
> >   (ie - completion classes inherit from some interface.)
> >   as a way of managing persistent data between completion events.
> >   Just a favorite of mine, I'm sure.
>
> The functions that you can pass to try-completion and friends are
> basically objcts: the 3rd argument is the method to call.  So it should
> be trivial to write a wrapper that takes a CLOS object obeying
> a completion-interface and returns a lambda expression suitable for
> try-completion and friends.

I hadn't thought of that.  That sounds like a cool hack idea I'll have
to fiddle with.

Not as nice as an official interface though. ;)

Eric

[Stefan Monnier]

>> You mean like lisp-complete-symbol but modal?

> Yes, that's what I meant, but in light of your simplified description, I
> suppose it doesn't matter much.  If there was a single function like
> 'completing-read' that took arguments to control what proposed text
> after the cursor was, I suppose it wouldn't matter what happened
> underneath.  I used overlays and overlay keymaps.

> You could try semantic-complete-analyze-inline on something that needs
> completing to see the effect I was going for.  It should work for Lisp
> or C code.

> I would expect designing such a thing would be challenging since the
> rules for what is before the cursor is a bit vague and language
> specific, so it might belong in a tool like Semantic.

We have `symbol-complete' although I'm not very happy about its API.
But something along thses lines would be good.  When I rewrite the
minibuffer completion code I tried to make it possible to use it in
non-minibuffer contexts, so for example lisp-complete-symbol nowadays
uses minibuffer-complete internally.

I also intend to consolidate the comint completion this way, tho it's
more difficult because its current UI is slightly different.  Part of
the difficulty is how to determine when the "completion is done" so as
to know when to pop-down the *Completions* buffer (if any).  Maybe the
best option is to pop-down *Completions* eagerly ("all the time").


        Stefan

[Eric M. Ludlam]

On Mon, 2009-10-19 at 16:06 -0400, Stefan Monnier wrote:

> >> You mean like lisp-complete-symbol but modal?
>
> > Yes, that's what I meant, but in light of your simplified description, I
> > suppose it doesn't matter much.  If there was a single function like
> > 'completing-read' that took arguments to control what proposed text
> > after the cursor was, I suppose it wouldn't matter what happened
> > underneath.  I used overlays and overlay keymaps.
>
> > You could try semantic-complete-analyze-inline on something that needs
> > completing to see the effect I was going for.  It should work for Lisp
> > or C code.
>
> > I would expect designing such a thing would be challenging since the
> > rules for what is before the cursor is a bit vague and language
> > specific, so it might belong in a tool like Semantic.
>
> We have `symbol-complete' although I'm not very happy about its API.
> But something along thses lines would be good.  When I rewrite the
> minibuffer completion code I tried to make it possible to use it in
> non-minibuffer contexts, so for example lisp-complete-symbol nowadays
> uses minibuffer-complete internally.

I was unaware of that, but it sounds useful.

> I also intend to consolidate the comint completion this way, tho it's
> more difficult because its current UI is slightly different.  Part of
> the difficulty is how to determine when the "completion is done" so as
> to know when to pop-down the *Completions* buffer (if any).  Maybe the
> best option is to pop-down *Completions* eagerly ("all the time").

For my completion engine, I used an overlay to wrap the text being
completed.  As soon as the cursor leaves the overlay, or if the user
types a character that doesn't belong in the symbol such as SPC, ., or
other punctuation, then it exists.  The post-command-hook is a bit
complex during the semantic inline completion.  Perhaps that style of
interface could be used, if not directly.

I was going for something where the completion part could activate at
idle time without interfering with regular editing commands.  ie, if the
user was typing away or using point motion, nothing would change, but if
they saw the decoration indicating active completion, they would know
that TAB was special.

A side effect is that a *completions* buffer is not always desired, so
there are some experiments on completion showing that might not need a
buffer, such as the tooltip, or ghost-text.  These options have varying
success in this task.

The semantic complete code for inline stuff uses the same completion and
display engines as the minibuffer or inline prompts, so you can mix and
match the pieces.  Hopefully an official API can do something similar,
so you can complete in the minibuffer or inline with the same functions.

Eric

[Eric M. Ludlam]

On Sun, 2009-10-18 at 23:50 -0400, Eric M. Ludlam wrote:
> For your particular example in xdisp.c, it seems the contents of
> xdisp.c
> fails to parse properly.  It seems to be related to the P_ macro, and
> one of the uses of it involving a line-end in the middle.  I've
> narrowed
> it down and will see if I can fix it.

To reply to myself, I checked changes into the CEDET repository for the
C preprocessor handling to fix the bug found here in xdisp.

By way of side effect, the completion problem originally posed is also
fixed, as there is no "window" symbol in the file anymore.

Eric

[Lennart Borgman (gmail)]

On Tue, Oct 20, 2009 at 12:17 AM, Eric M. Ludlam <eric@...> wrote:
>
> The semantic complete code for inline stuff uses the same completion and
> display engines as the minibuffer or inline prompts, so you can mix and
> match the pieces.  Hopefully an official API can do something similar,
> so you can complete in the minibuffer or inline with the same functions.

I think the authors of company-mode and completion-UI (Nikolaj and
Toby) are interested in this too. And Drew, of course.

[Stefan Monnier]

> For my completion engine, I used an overlay to wrap the text being
> completed.

That's also what lisp-complete-symbol uses to tell minibuffer-completion
what is to be completed (more specifically minibuffer-completion
operates on a `field', so I create an overlay with a field property).

> As soon as the cursor leaves the overlay, or if the user
> types a character that doesn't belong in the symbol such as SPC, ., or
> other punctuation, then it exists.

That's the kind of UI I was thinking of, yes.  Can you point me to the
relevant code you use for that (especially things like
post-command-hook)?

> A side effect is that a *completions* buffer is not always desired, so
> there are some experiments on completion showing that might not need a
> buffer, such as the tooltip, or ghost-text.  These options have varying
> success in this task.

Yes, rather than a *completions* buffer, we could have tooltip-like
frames show up.  That will need to be chosen by the caller.


        Stefan

[Eric M. Ludlam]

On Mon, 2009-10-19 at 20:14 -0400, Stefan Monnier wrote:

> > For my completion engine, I used an overlay to wrap the text being
> > completed.
>
> That's also what lisp-complete-symbol uses to tell minibuffer-completion
> what is to be completed (more specifically minibuffer-completion
> operates on a `field', so I create an overlay with a field property).
>
> > As soon as the cursor leaves the overlay, or if the user
> > types a character that doesn't belong in the symbol such as SPC, ., or
> > other punctuation, then it exists.
>
> That's the kind of UI I was thinking of, yes.  Can you point me to the
> relevant code you use for that (especially things like
> post-command-hook)?

It is in the cedet/semantic/complete.el, and the function is
semantic-complete-post-command-hook for the inline completion exit
conditions.

Enjoy
Eric

[Lluis-3]

Just as a side-note, now that you've raised the topic of a completion interface.

I don't know about the features of current code, but I miss some features on
some (if not all) the completion interfaces I've tested. So here's a list of
(my) desired features (all of which should be configurable by the user):

- symbol name
  Of course, this is what already provides every completion UI
- extra information
  This can be filled ub with symbol "metadata". Which metadata appears on the
  completion UI should be configurable by the user:
    - return type
    - arguments (type and/or name)
    - definition location (aka file)
    - short documentation
    - long documentation
  All this metadata should be located anywhere around the symbol name and/or the
  minibuffer (e.g., I think company-mode shows short documentation on
  minibuffer, until user presses F1, when full documentation is shown.
  Some metadata might be shown in the minibuffer after completion selection
  (e.g., prototype).
- result narrowing
  A-la company-mode.
- argument placeholders
  So that argument type and/or name is shown as placeholders, such that the user
  simply TABs (or whatever) to fill-in the blanks.

In any case, I've recently jumped into Emacs, so my elisp coding abilities are
still quite low...

Read you,
    Lluis

--
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

[Eric M. Ludlam]

On Wed, 2009-10-21 at 12:58 +0200, Lluis wrote:

> Just as a side-note, now that you've raised the topic of a completion interface.
>
> I don't know about the features of current code, but I miss some features on
> some (if not all) the completion interfaces I've tested. So here's a list of
> (my) desired features (all of which should be configurable by the user):
>
> - symbol name
>   Of course, this is what already provides every completion UI
> - extra information
>   This can be filled ub with symbol "metadata". Which metadata appears on the
>   completion UI should be configurable by the user:
>     - return type
>     - arguments (type and/or name)
>     - definition location (aka file)
>     - short documentation
>     - long documentation
>   All this metadata should be located anywhere around the symbol name and/or the
>   minibuffer (e.g., I think company-mode shows short documentation on
>   minibuffer, until user presses F1, when full documentation is shown.
>   Some metadata might be shown in the minibuffer after completion selection
>   (e.g., prototype).
> - result narrowing
>   A-la company-mode.
> - argument placeholders
>   So that argument type and/or name is shown as placeholders, such that the user
>   simply TABs (or whatever) to fill-in the blanks.

Hi Lluis,

Just as an FYI about the semantic tag info (which you seem to be
alluding to), is that it provides all that info, though not necessarily
from the completion engine via a UI.

The internal tag table format of semantic includes data type info,
argument lists, positional info, and the doc is available.  You can also
pass a semantic native tag table directly into 'try-completions' or any
other completion prompt in Emacs.

The completion engine in Semantic also takes advantage of this and
allows ways to focus on a tag based on it's positional information, or
distinguish based on arg list or some such.  Since it uses a code
analyzer, you tend not to need too much in the way of narrowing of scope
since it knows what symbols don't fit in a particular context, so it
doesn't provide them as completions.

If core emacs completion prompts were to select some metadata based
approach to completion tables, the Semantic tag format is pretty simple,
and could provide what is needed.  A tag table is a list, with each
element formated like this:

("NAME" CLASS ATTRIBUTES PROPERTIES POSITION)

Where attributes and properties are plists, and POSITION is an optional
element which could be either an overlay, or a vector of two numbers.
Attributes are features of whatever the tag is, such as arg lists, or
datatype.  Properties are features of the tag itself, like the filename
it is in, or other data needed to maintain the tag, like parser
information.  Of course there may not be a use case for this outside of
tag data, so perhaps it is a moot point.

Eric

[Lluis-3]

El Wed, Oct 21, 2009 at 08:35:29AM -0400, Eric M. Ludlam ens deleit� amb les seg�ents paraules:
> Just as an FYI about the semantic tag info (which you seem to be
> alluding to), is that it provides all that info, though not necessarily
> from the completion engine via a UI.

Well, it's been a long time since I played with the completion interfaces
provided in CEDET, but I already knew all that information was available
somewhere in semantic :)

I just didn't know documentation was also available as one of the tag's
attributes, which reminds me that:
    1) One-liner doxygen description shows up with the `*/' at the end
    2) Descriptions of the form: /*< description */
       are not assigned to the correct variable
    3) One more nice feature would be to have the "show long documentation"
       parse doxygen-specific commands such that parameters descriptions are
       shown more clearly

I also tried with `completion-ui', `auto-complete' and `company-mode', which are
the ones I've found that provide completion UIs through overlays (hope I'm using
the correct term), which is the form i feel more comfortable with.

But as far as I've been able to see, none of these three provide the features
I've mentioned, although they do provide semantic-based backends for completion.


[...]
> Since it uses a code analyzer, you tend not to need too much in the way of
> narrowing of scope since it knows what symbols don't fit in a particular
> context, so it doesn't provide them as completions.

Right, that's just a feature I found in company-mode, and thought it was nice
when working with a large amount of possible completions; which is not usually
my case, now that I work mostly in C++.


> If core emacs completion prompts were to select some metadata based
> approach to completion tables, the Semantic tag format is pretty simple,
> and could provide what is needed.

See, my Emacs/elisp knowledge is too scarce to have an opinion on this :)

I once tried diving into semantic to accomplish what I wanted, together with
completion-ui, but back then it was just my first or second week with emacs and
lisp, so I think accomplishing that really was out of my possibilities :)

In any case, I don't know if there is general interest on the features I have
described. If not, I might eventually try to implement that, but would need some
advice on which are the "standard" mechanisms from which to build up so that the
solution is near to the least convoluted outcome.

But this will be "some day"... my first contact is still to be finished with a
small improvement to VC to allow filling the commit report with "C-x 4 a"
without having to fill up a changelog file; which I've been postponing for too
long.

Thanks,
    Lluis

--
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

[Chong Yidong]

"Eric M. Ludlam" <eric@...> writes:

> On Sun, 2009-10-18 at 23:50 -0400, Eric M. Ludlam wrote:
>> For your particular example in xdisp.c, it seems the contents of
>> xdisp.c fails to parse properly.  It seems to be related to the P_
>> macro, and one of the uses of it involving a line-end in the middle.
>> I've narrowed it down and will see if I can fix it.
>
> To reply to myself, I checked changes into the CEDET repository for the
> C preprocessor handling to fix the bug found here in xdisp.
>
> By way of side effect, the completion problem originally posed is also
> fixed, as there is no "window" symbol in the file anymore.

I've merged your changes into the Emacs repository:

  * cedet/semantic/bovine/c.el (semantic-c-debug-mode-init)
  (semantic-c-debug-mode-init-pch): New functions.
  (semantic-c-debug-mode-init-last-mode): New var.
  (semantic-c-parse-lexical-token): Use them.

  * cedet/semantic/lex-spp.el (semantic-lex-spp-anlyzer-do-replace):
  When extracting the argument list, limit only by point-max.

I assume it's the lex-spp change you're referring to.  However, the bug
has not gone away for me: `C-c , j window [TAB]' in xdisp.c still brings
up "window" as the only completion.

[Eric M. Ludlam]

On Wed, 2009-10-21 at 10:07 -0400, Chong Yidong wrote:

> "Eric M. Ludlam" <eric@...> writes:
>
> > On Sun, 2009-10-18 at 23:50 -0400, Eric M. Ludlam wrote:
> >> For your particular example in xdisp.c, it seems the contents of
> >> xdisp.c fails to parse properly.  It seems to be related to the P_
> >> macro, and one of the uses of it involving a line-end in the middle.
> >> I've narrowed it down and will see if I can fix it.
> >
> > To reply to myself, I checked changes into the CEDET repository for the
> > C preprocessor handling to fix the bug found here in xdisp.
> >
> > By way of side effect, the completion problem originally posed is also
> > fixed, as there is no "window" symbol in the file anymore.
>
> I've merged your changes into the Emacs repository:
>
>   * cedet/semantic/bovine/c.el (semantic-c-debug-mode-init)
>   (semantic-c-debug-mode-init-pch): New functions.
>   (semantic-c-debug-mode-init-last-mode): New var.
>   (semantic-c-parse-lexical-token): Use them.
>
>   * cedet/semantic/lex-spp.el (semantic-lex-spp-anlyzer-do-replace):
>   When extracting the argument list, limit only by point-max.
>
> I assume it's the lex-spp change you're referring to.  However, the bug

Correct.  The other is related to a different bug with a users
c-mode-hook that might throw an error, and how  to detect and tell them
about it.

> has not gone away for me: `C-c , j window [TAB]' in xdisp.c still brings
> up "window" as the only completion.

That is probably because the old tagging information is still being
used.  If you run

M-x semantic-clear-toplevel-cache RET

in that buffer, it will flush old parsing data, or you can go into your
~/.semanticdb/ directory, and delete the cache file for emacs/src before
starting emacs, which might be better since multiple files  there could
be affected.

Eric

[Eric M. Ludlam]

On Wed, 2009-10-21 at 15:28 +0200, Lluis wrote:

> > Just as an FYI about the semantic tag info (which you seem to be
> > alluding to), is that it provides all that info, though not
> necessarily
> > from the completion engine via a UI.
>
> Well, it's been a long time since I played with the completion
> interfaces
> provided in CEDET, but I already knew all that information was
> available
> somewhere in semantic :)
>
> I just didn't know documentation was also available as one of the
> tag's
> attributes, which reminds me that:
>     1) One-liner doxygen description shows up with the `*/' at the end
>     2) Descriptions of the form: /*< description */
>        are not assigned to the correct variable
>     3) One more nice feature would be to have the "show long
> documentation"
>        parse doxygen-specific commands such that parameters
> descriptions are
>        shown more clearly

The doc in Semantic tags is only for languages that have explicit doc,
like Emacs Lisp.  For other languages, when doc is requested of a tag,
there is some code that goes and looks for comments.  It doesn't follow
doxygen style rules though.

On the flip side, the srecode tool has a documentation feature that can
do doxygen style comment reading/modification, going so far as to take
boring comments and convert them to be doxygen compatible.  You can use
it with srecode-document-insert-comment, or C-c / C if srecode minor
mode is on.  I think I only created templates for C/C++, and Java.

I suppose it would make sense to merge the comment reader pieces back
together.

For folks hacking Emacs C code, if there is a comment style/behavior
that is repetitive/special in some way, it should be possible to create
srecode template specialization for it.  I've been interested in having
EDE provided project template specializations, but haven't had a use
case since I only use the GNUish or doxygen ones which are language
specific.

> I also tried with `completion-ui', `auto-complete' and `company-mode',
> which are
> the ones I've found that provide completion UIs through overlays (hope
> I'm using
> the correct term), which is the form i feel more comfortable with.
>
> But as far as I've been able to see, none of these three provide the
> features
> I've mentioned, although they do provide semantic-based backends for
> completion.

Those tools are pretty generic completion engines, and thus don't have
any special features that use the extra tagging info.  That I know of
anyway.

Eric

[Stefan Monnier]

> It is in the cedet/semantic/complete.el, and the function is
> semantic-complete-post-command-hook for the inline completion exit
> conditions.

:-(  It's not much prettier than the hacks I've come up with so far.


        Stefan