C syntax highlighting, faster syntax based folding and a bug

[Lech Lorens]

I have recently suffered very much from the problem reported by
Charles Campbell and dman described in todo.txt as:

7   C syntax highlighting gets a lot slower after ":set foldmethod=syntax".
    (Charles Campbell)  Inserting a "{" is very slow. (dman)

The problem stems from the fact that inserting '{' into a C file causes
all the folds from the insertion place to the end of the file to become
invalid.
The same goes for inserting '(' or '['. I tried to fix the problem and
came up with a solution (see attached c.vim.patch). I modified C syntax
highlighting in such a way that inserting '(' or '[' causes a new region
to be created but this time the region does not extend to the end of the
file and as a consequence not so many folds have to be recalculated.

I would appreciate feedback on my modifications a lot and really hope
for some comments. You might want to follow my test procedure:

Enter the directory with Vim sources. Execute Vim:
$ vim -p eval.c eval.c eval.c
:tabdo set fdm=syntax fdc=5 number
:normal 1gt
:820
:normal zO

And then type the following (observe the fold column):
Ofor (i =

After inserting '(' with the official syntax highlighting file I need to
wait a few seconds (sic!) before the screen is updated and "i ="
appears. With my modifications of c.vim Vim responds instantaneously
(again: observe the difference in the fold column).

This comes at a cost, however. While in case of the original syntax
highlighting all the braces from the modification place to the end of
the file would become highlighted as error, after my modifications only
braces inside the current block get such highlighting. This is still
quite visible if you open a parenthesis at line 820, but is not so any
more if you do it at line 843.

I am ready to accept such a trade-off as I rely very much on
C-syntax-based folding and recently more and more often have been
finding myself in situations where I input some code without getting any
visual feedback from Vim (this happens on my new machine at work, even
more on my old computer at home). I presume that there are others who
are bothered by the same problem. Perhaps the behaviour I introduced to
c.vim could be made into an option like c_no_curly_error or
c_no_bracket_error.





I would like also to describe the (erroneous in my opinion) behaviour of
Vim I discovered while trying to improve c.vim. The version of c.vim
that allows the problem to be reproduced can be obtained by applying the
attached patch c.vim.patch.bad.

If I follow the test procedure described in the first part of my email
I get the most desirable result: highlighting of all the braces to the
end of file without the folds being recomputed. This is in my opinion an
error as the cBadBlock region starting with '{' at line 827 introduces
a new fold level. cBadBlock is not closed by '}' at line 843 but the
fold - as indicated by the fold column - ends at this point. You can
verify my claim by going to line 844 and performing the following:

:for id in synstack(line("."), col("."))
: echo synIDattr(id, "name")
:endfor

The output is:
cBlock
cParen
cBadBlock

Is this a correct behaviour?

Thank you in advance for any comments!

--
Cheers,
Lech

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_dev" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

---

diff --git a/vim/vim72/syntax/c.vim b/vim/vim72/syntax/c.vim
index 1cb3d11..98ac37c 100644
--- a/vim/vim72/syntax/c.vim
+++ b/vim/vim72/syntax/c.vim
@@ -72,7 +72,7 @@ endif
 " This should be before cErrInParen to avoid problems with #define ({ xxx })
 if exists("c_curly_error")
   syntax match cCurlyError "}"
-  syntax region cBlock start="{" end="}" contains=ALLBUT,cCurlyError,@cParenGroup,cErrInParen,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell fold
+  syntax region cBlock start="{" end="}" contains=ALLBUT,cBadBlock,cCurlyError,@cParenGroup,cErrInParen,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell fold
 else
   syntax region cBlock start="{" end="}" transparent fold
 endif
@@ -82,29 +82,31 @@ endif
 " But avoid matching <::.
 syn cluster cParenGroup contains=cParenError,cIncluded,cSpecial,cCommentSkip,cCommentString,cComment2String,@cCommentGroup,cCommentStartError,cUserCont,cUserLabel,cBitField,cOctalZero,cCppOut,cCppOut2,cCppSkip,cFormat,cNumber,cFloat,cOctal,cOctalError,cNumbersCom
 if exists("c_no_curly_error")
-  syn region cParen transparent start='(' end=')' contains=ALLBUT,@cParenGroup,cCppParen,cCppString,@Spell
+  syn region cParen transparent start='(' end=')' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cCppParen,cCppString,@Spell
   " cCppParen: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppParen transparent start='(' skip='\\$' excludenl end=')' end='$' contained contains=ALLBUT,@cParenGroup,cParen,cString,@Spell
   syn match cParenError display ")"
   syn match cErrInParen display contained "^[{}]\|^<%\|^%>"
 elseif exists("c_no_bracket_error")
-  syn region cParen transparent start='(' end=')' contains=ALLBUT,@cParenGroup,cCppParen,cCppString,@Spell
+  syn region cParen transparent start='(' end=')' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cCppParen,cCppString,@Spell
   " cCppParen: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppParen transparent start='(' skip='\\$' excludenl end=')' end='$' contained contains=ALLBUT,@cParenGroup,cParen,cString,@Spell
   syn match cParenError display ")"
   syn match cErrInParen display contained "[{}]\|<%\|%>"
 else
-  syn region cParen transparent start='(' end=')' contains=ALLBUT,@cParenGroup,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell
+  syn region cParen transparent start='(' end=')' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell
   " cCppParen: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppParen transparent start='(' skip='\\$' excludenl end=')' end='$' contained contains=ALLBUT,@cParenGroup,cErrInBracket,cParen,cBracket,cString,@Spell
   syn match cParenError display "[\])]"
   syn match cErrInParen display contained "[\]{}]\|<%\|%>"
-  syn region cBracket transparent start='\[\|<::\@!' end=']\|:>' contains=ALLBUT,@cParenGroup,cErrInParen,cCppParen,cCppBracket,cCppString,@Spell
+  syn region cBracket transparent start='\[\|<::\@!' end=']\|:>' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cErrInParen,cCppParen,cCppBracket,cCppString,@Spell
   " cCppBracket: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppBracket transparent start='\[\|<::\@!' skip='\\$' excludenl end=']\|:>' end='$' contained contains=ALLBUT,@cParenGroup,cErrInParen,cParen,cBracket,cString,@Spell
   syn match cErrInBracket display contained "[);{}]\|<%\|%>"
 endif
 
+syntax region cBadBlock keepend extend start="{" end="}" contained containedin=cParen,cBracket,cBadBlock transparent fold
+
 "integer number, or floating point number without a dot and with "f".
 syn case ignore
 syn match cNumbers display transparent "\<\d\|\.\d" contains=cNumber,cFloat,cOctalError,cOctal

---

diff --git a/vim/vim72/syntax/c.vim b/vim/vim72/syntax/c.vim
index 1cb3d11..1709bec 100644
--- a/vim/vim72/syntax/c.vim
+++ b/vim/vim72/syntax/c.vim
@@ -72,7 +72,7 @@ endif
 " This should be before cErrInParen to avoid problems with #define ({ xxx })
 if exists("c_curly_error")
   syntax match cCurlyError "}"
-  syntax region cBlock start="{" end="}" contains=ALLBUT,cCurlyError,@cParenGroup,cErrInParen,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell fold
+  syntax region cBlock start="{" end="}" contains=ALLBUT,cBadBlock,cCurlyError,@cParenGroup,cErrInParen,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell fold
 else
   syntax region cBlock start="{" end="}" transparent fold
 endif
@@ -82,29 +82,34 @@ endif
 " But avoid matching <::.
 syn cluster cParenGroup contains=cParenError,cIncluded,cSpecial,cCommentSkip,cCommentString,cComment2String,@cCommentGroup,cCommentStartError,cUserCont,cUserLabel,cBitField,cOctalZero,cCppOut,cCppOut2,cCppSkip,cFormat,cNumber,cFloat,cOctal,cOctalError,cNumbersCom
 if exists("c_no_curly_error")
-  syn region cParen transparent start='(' end=')' contains=ALLBUT,@cParenGroup,cCppParen,cCppString,@Spell
+  syn region cParen transparent start='(' end=')' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cCppParen,cCppString,@Spell
   " cCppParen: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppParen transparent start='(' skip='\\$' excludenl end=')' end='$' contained contains=ALLBUT,@cParenGroup,cParen,cString,@Spell
   syn match cParenError display ")"
   syn match cErrInParen display contained "^[{}]\|^<%\|^%>"
 elseif exists("c_no_bracket_error")
-  syn region cParen transparent start='(' end=')' contains=ALLBUT,@cParenGroup,cCppParen,cCppString,@Spell
+  syn region cParen transparent start='(' end=')' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cCppParen,cCppString,@Spell
   " cCppParen: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppParen transparent start='(' skip='\\$' excludenl end=')' end='$' contained contains=ALLBUT,@cParenGroup,cParen,cString,@Spell
   syn match cParenError display ")"
   syn match cErrInParen display contained "[{}]\|<%\|%>"
 else
-  syn region cParen transparent start='(' end=')' contains=ALLBUT,@cParenGroup,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell
+  syn region cParen transparent start='(' end=')' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cCppParen,cErrInBracket,cCppBracket,cCppString,@Spell
   " cCppParen: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppParen transparent start='(' skip='\\$' excludenl end=')' end='$' contained contains=ALLBUT,@cParenGroup,cErrInBracket,cParen,cBracket,cString,@Spell
   syn match cParenError display "[\])]"
   syn match cErrInParen display contained "[\]{}]\|<%\|%>"
-  syn region cBracket transparent start='\[\|<::\@!' end=']\|:>' contains=ALLBUT,@cParenGroup,cErrInParen,cCppParen,cCppBracket,cCppString,@Spell
+  syn region cBracket transparent start='\[\|<::\@!' end=']\|:>' end='}'me=s-1 contains=ALLBUT,cBlock,@cParenGroup,cErrInParen,cCppParen,cCppBracket,cCppString,@Spell
   " cCppBracket: same as cParen but ends at end-of-line; used in cDefine
   syn region cCppBracket transparent start='\[\|<::\@!' skip='\\$' excludenl end=']\|:>' end='$' contained contains=ALLBUT,@cParenGroup,cErrInParen,cParen,cBracket,cString,@Spell
   syn match cErrInBracket display contained "[);{}]\|<%\|%>"
 endif
 
+" NOTE: the following region definition works instead of the above one.
+" However, it seems that it is due to a bug that Vim does not create new folds
+" for the cBadBlock regions.
+syntax region cBadBlock start="{" end="}" contained containedin=cParen,cBracket transparent fold
+
 "integer number, or floating point number without a dot and with "f".
 syn case ignore
 syn match cNumbers display transparent "\<\d\|\.\d" contains=cNumber,cFloat,cOctalError,cOctal

[Dominique Pellé]

Lech Lorens wrote:

> I would appreciate feedback on my modifications a lot and really hope
> for some comments. You might want to follow my test procedure:
>
> Enter the directory with Vim sources. Execute Vim:
> $ vim -p eval.c eval.c eval.c
> :tabdo set fdm=syntax fdc=5 number
> :normal 1gt
> :820
> :normal zO
>
> And then type the following (observe the fold column):
> Ofor (i =
>
> After inserting '(' with the official syntax highlighting file I need to
> wait a few seconds (sic!) before the screen is updated and "i ="
> appears.

On my machine, I had to wait 1 minute and 50 seconds (!) before I
could see "i = " (and Vim was using 100% of the CPU during that
time).  That's with Vim-7.2.284 built with -O0 on a Core-2, 1.7Ghz.


> With my modifications of c.vim Vim responds instantaneously
> (again: observe the difference in the fold column).

After applying your patch (c.vim.patch), I could see "i = "
instantaneously.

I normally don't use syntax folding, but if I did, I would most certainly
want to do it with your patch given the major speed up.

> This comes at a cost, however. While in case of the original syntax
> highlighting all the braces from the modification place to the end of
> the file would become highlighted as error, after my modifications only
> braces inside the current block get such highlighting. This is still
> quite visible if you open a parenthesis at line 820, but is not so any
> more if you do it at line 843.
>
> I am ready to accept such a trade-off as I rely very much on
> C-syntax-based folding and recently more and more often have been
> finding myself in situations where I input some code without getting any
> visual feedback from Vim (this happens on my new machine at work, even
> more on my old computer at home). I presume that there are others who
> are bothered by the same problem. Perhaps the behaviour I introduced to
> c.vim could be made into an option like c_no_curly_error or
> c_no_bracket_error.

I would also accept the trade-off, given the dramatic speed up.


> I would like also to describe the (erroneous in my opinion) behaviour of
> Vim I discovered while trying to improve c.vim. The version of c.vim
> that allows the problem to be reproduced can be obtained by applying the
> attached patch c.vim.patch.bad.

I did not check that yet.

Thanks
-- Dominique

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_dev" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

[Lech Lorens]

On 10-Nov-2009 Dominique Pellé <dominique.pelle@...> wrote:
>
> > After inserting '(' with the official syntax highlighting file I need to
> > wait a few seconds (sic!) before the screen is updated and "i ="
> > appears.
>
> On my machine, I had to wait 1 minute and 50 seconds (!) before I
> could see "i = " (and Vim was using 100% of the CPU during that
> time).  That's with Vim-7.2.284 built with -O0 on a Core-2, 1.7Ghz.

Bram, do you have an opinion on the subject?

--
Cheers,
Lech

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_dev" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

[Bram Moolenaar]

Lech Lorens wrote:

> On 10-Nov-2009 Dominique Pellé <dominique.pelle@...> wrote:
> >
> > > After inserting '(' with the official syntax highlighting file I need to
> > > wait a few seconds (sic!) before the screen is updated and "i ="
> > > appears.
> >
> > On my machine, I had to wait 1 minute and 50 seconds (!) before I
> > could see "i = " (and Vim was using 100% of the CPU during that
> > time).  That's with Vim-7.2.284 built with -O0 on a Core-2, 1.7Ghz.
>
> Bram, do you have an opinion on the subject?

I have put a remark in the todo list, I'll look into it later.

Did you try profiling to find out where the time is wasted?

--
DENNIS: Oh, very nice. King, eh!  I expect you've got a palace and fine
        clothes and courtiers and plenty of food.  And how d'you get that?  By
        exploiting the workers! By hanging on to outdated imperialist dogma
        which perpetuates the social and economic differences in our society!
                 "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

 /// Bram Moolenaar -- Bram@... -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\        download, build and distribute -- http://www.A-A-P.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_dev" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

[Dominique Pellé]

Bram Moolenaar wrote:

> Lech Lorens wrote:
>
>> On 10-Nov-2009 Dominique Pellé <dominique.pelle@...> wrote:
>> >
>> > > After inserting '(' with the official syntax highlighting file I need to
>> > > wait a few seconds (sic!) before the screen is updated and "i ="
>> > > appears.
>> >
>> > On my machine, I had to wait 1 minute and 50 seconds (!) before I
>> > could see "i = " (and Vim was using 100% of the CPU during that
>> > time).  That's with Vim-7.2.284 built with -O0 on a Core-2, 1.7Ghz.
>>
>> Bram, do you have an opinion on the subject?
>
> I have put a remark in the todo list, I'll look into it later.
>
> Did you try profiling to find out where the time is wasted?

I profiled with gprof using Vim-7.2.293 built with:
   -O0 -g -pg -DWE_ARE_PROFILING
... and executed the steps described earlier by Lech:

$ vim -p eval.c eval.c eval.c
:tabdo set fdm=syntax fdc=5 number
:normal 1gt
:820
:normal zO

Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls   s/call   s/call  name
 72.81     49.62    49.62 106860477     0.00     0.00  in_id_list
  7.55     54.76     5.14  5585619     0.00     0.00  syn_current_attr
  3.19     56.94     2.17  8395304     0.00     0.00  regmatch
  2.60     58.70     1.77   270449     0.00     0.00  syn_stack_find_entry
  2.44     60.37     1.66   533472     0.00     0.00  syntax_start
  1.48     61.38     1.01 114389042     0.00     0.00  utfc_ptr2len
  1.26     62.23     0.86 60053856     0.00     0.00  regnext
  0.97     62.90     0.66  6124658     0.00     0.00  vim_strchr
  0.44     63.20     0.30  6391651     0.00     0.00  vim_regexec_both
  0.40     63.47     0.28 12652530     0.00     0.00  vim_iswordc_buf
  0.37     63.72     0.25 43505784     0.00     0.00  utf_ptr2char
...

I put the full gprof profiling result at:

  http://dominique.pelle.free.fr/prof-vim-7.2.293.txt.gz

Cheers
-- Dominique

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_dev" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---