open large file with C code: is it realy should be so slow?

[davemilter]

I have problem with emacs responsibility,
I work with large enough C header files,
and when I want to scroll it using mouse's wheel or
page (up|down) keys emacs stop react on any keys, like (ctrl+g),
and eats 100% of CPU's time during long period,
I wonder is this a bug, or expected behaviour?

My cpu frequnce is 2200.000MHz.

elp results are:

Function Name                                              Call Count
Elapsed Time  Average Time
=========================================================  ==========
============  ============
font-lock-fontify-region                                   387
14.909457999  0.0385257312
font-lock-default-fontify-region                           387
14.904738000  0.0385135348
font-lock-fontify-keywords-region                          387
14.469801     0.0373896666
font-lock-fontify-syntactically-region                     384
0.4169809999  0.0010858880
font-lock-unfontify-region                                 387
0.0036489999  9.428...e-06
font-lock-mode                                             17
0.0036209999  0.0002129999
font-lock-set-defaults                                     390
0.0035869999  9.197...e-06
font-lock-default-function                                 17
0.0033780000  0.0001987058
font-lock-mode-internal                                    3
0.00324       0.00108
font-lock-compile-keywords                                 3
0.002872      0.0009573333
font-lock-compile-keyword                                  60
0.0027359999  4.559...e-05
font-lock-extend-region-multiline                          387
0.0010560000  2.728...e-06
font-lock-default-unfontify-region                         387
0.0010289999  2.658...e-06
font-lock-extend-region-wholelines                         387
0.0008740000  2.258...e-06
font-lock-extend-jit-lock-region-after-change              103
0.0007060000  6.854...e-06
font-lock-turn-on-thing-lock                               3
0.00034       0.0001133333
font-lock-change-mode                                      5
0.000144      2.880...e-05
font-lock-eval-keywords                                    4
2.7e-05       6.75e-06
font-lock-value-in-major-mode                              8
9e-06         1.125e-06
cc-imenu-init                                              1
7e-06         7e-06
font-lock-append-text-property                             1
6e-06         6e-06
font-lock-choose-keywords                                  2
4.999...e-06  2.499...e-06


Version:

GNU Emacs 22.3.1 (i686-pc-linux-gnu, GTK+ Version 2.12.8)


I tried Visual C++ 6.0 under wine,
and gvim, and they works without any delays,
highlight syntax and allow scrolling.

If somebody wnat reproduce this behaviour,
here is link to source code with which I working:
http://www.atmel.com/dyn/resources/prod_documents/AT91Bootstrap1.10.zip
there are files in it like
include/AT91SAM9263_inc.h
it contains around 4000 lines of code,
most of which is comments or #define

[Alan Mackenzie]

Hi, Dave!

On Mon, Dec 01, 2008 at 01:01:08PM +0300, Dave Milter wrote:
> I have problem with emacs responsibility,
> I work with large enough C header files,
> and when I want to scroll it using mouse's wheel or
> page (up|down) keys emacs stop react on any keys, like (ctrl+g),
> and eats 100% of CPU's time during long period,
> I wonder is this a bug, or expected behaviour?

It's a bug.

Although C Mode works "properly" here, it doesn't seem to be tuned very
well for files like this one (At91SAM9253_inc.h), which contain a lot of
#defines and comments and nothing else.

The only workaround I can suggest at the moment is to turn font-locking
off in these buffers.  This is clearly unsatisfactory.  Sorry.

> My cpu frequnce is 2200.000MHz.

> elp results are:

[ Thanks for these! ]

> Version:

> GNU Emacs 22.3.1 (i686-pc-linux-gnu, GTK+ Version 2.12.8)


> I tried Visual C++ 6.0 under wine,
> and gvim, and they works without any delays,
> highlight syntax and allow scrolling.

> If somebody wnat reproduce this behaviour,
> here is link to source code with which I working:
> http://www.atmel.com/dyn/resources/prod_documents/AT91Bootstrap1.10.zip
> there are files in it like
> include/AT91SAM9263_inc.h
> it contains around 4000 lines of code,
> most of which is comments or #define

I will try to fix this bug, though this could be quite difficult and take
some time.  Thanks very much indeed for taking the trouble to report it.

--
Alan Mackenzie (Nuremberg, Germany).

[davemilter]

On Mon, Dec 1, 2008 at 3:37 PM, Alan Mackenzie <acm@...> wrote:

>
> On Mon, Dec 01, 2008 at 01:01:08PM +0300, Dave Milter wrote:
>> I have problem with emacs responsibility,
>> I work with large enough C header files,
>> and when I want to scroll it using mouse's wheel or
>> page (up|down) keys emacs stop react on any keys, like (ctrl+g),
>> and eats 100% of CPU's time during long period,
>> I wonder is this a bug, or expected behaviour?
>
> It's a bug.
>
> Although C Mode works "properly" here, it doesn't seem to be tuned very
> well for files like this one (At91SAM9253_inc.h), which contain a lot of
> #defines and comments and nothing else.
>

I made some more testing (to find out problem in "file", or in "large"),
because of really want to see this bug fixed,

I see the same behaviour on file created by

for ((i=0;i<500;++i)); do echo "extern void f${i}(int a${i});"; done

emacs from cvs and emacs 22 show the same behaviour -
eating 100% of cpu, if make fast scrolling.

>
>> elp results are:
>

Function Name                                              Call Count
Elapsed Time  Average Time
=========================================================  ==========
============  ============
font-lock-fontify-region                                   41
230.65321899  5.6256882682
font-lock-default-fontify-region                           41
230.65264900  5.6256743658
font-lock-fontify-keywords-region                          41
230.61167599  5.6246750243
font-lock-fontify-syntactically-region                     41
0.039573      0.0009651951
font-lock-unfontify-region                                 41
0.0005050000  1.231...e-05
font-lock-default-unfontify-region                         41
0.0001930000  4.707...e-06
font-lock-extend-region-wholelines                         41
0.0001080000  2.634...e-06
font-lock-extend-region-multiline                          41
9.800...e-05  2.390...e-06
font-lock-set-defaults                                     41
8.299...e-05  2.024...e-06
font-lock-mode                                             3
7.6e-05       2.533...e-05
font-lock-default-function                                 3
1.400...e-05  4.666...e-06


In fact, with small files, for example the same script but 5000 -> 500,
I see the same situation, but after eating cpu during some period,
it never eating it after, and all works smoothly,
while with big files it eats it every fast scroll.

[Alan Mackenzie]

Hi, Dave,

I'm actually working on this bug at the moment.  With the Yuletide break,
and so on, I've kind of lost the thread a bit and I got bogged down in
the complexities of the code.  I should have got back to you sooner;
sorry!

On Mon, Jan 05, 2009 at 01:07:18AM +0300, Dave Milter wrote:
> On Mon, Dec 1, 2008 at 3:37 PM, Alan Mackenzie <acm@...> wrote:

> > On Mon, Dec 01, 2008 at 01:01:08PM +0300, Dave Milter wrote:
> >> I have problem with emacs responsibility,
> >> I work with large enough C header files,
> >> and when I want to scroll it using mouse's wheel or
> >> page (up|down) keys emacs stop react on any keys, like (ctrl+g),
> >> and eats 100% of CPU's time during long period,
> >> I wonder is this a bug, or expected behaviour?

> > It's a bug.

> > Although C Mode works "properly" here, it doesn't seem to be tuned
> > very well for files like this one (At91SAM9253_inc.h), which contain
> > a lot of #defines and comments and nothing else.

> I made some more testing (to find out problem in "file", or in "large"),
> because of really want to see this bug fixed,

It will be fixed, hopefully within the next few days.  It's too serious a
bug not to fix.

> I see the same behaviour on file created by

> for ((i=0;i<500;++i)); do echo "extern void f${i}(int a${i});"; done

Yes.  The CC Mode function which is to blame is `c-parse-state'.  It
maintains a "cache" of (essentially) brace pairs, using the positions of
these braces to scan from.  In a "brace desert", it scans from the
beginning of the buffer, sometimes many times per (scroll) command, and
this is why it is so slow.

To see this, create an alternative file like this:

    for ((i=0;i<500;++i)); do echo "extern void f${i}(int a${i}) {}"; done
                                                                 ^^

Scrolling in this file will be instantaneous, as it should be.

> emacs from cvs and emacs 22 show the same behaviour -
> eating 100% of cpu, if make fast scrolling.

Yes.

[ .... ]

> In fact, with small files, for example the same script but 5000 -> 500,
> I see the same situation, but after eating cpu during some period,
> it never eating it after, and all works smoothly,
> while with big files it eats it every fast scroll.

If you profile the stuff in cc-engine.el, you'll see c-parse-state right
at the top.

--
Alan Mackenzie (Nuremberg, Germany).

[Richard Stallman]

I too find C mode painfully slow these days.  I wonder what changed to
make it so much slower than it was.  Perhaps the only relevant change
is that Font Lock is now enabled; I never used to enable it.  Even so,
it must make a lot of users unhappy.

[Chong Yidong]

Richard M Stallman <rms@...> writes:

> I too find C mode painfully slow these days.  I wonder what changed to
> make it so much slower than it was.  Perhaps the only relevant change
> is that Font Lock is now enabled; I never used to enable it.  Even so,
> it must make a lot of users unhappy.

You are likely using an exceptionally slow computer.  There are certain
special situations in which a C-mode slowdown is observable, but it
should be fine in normal usage, at least in my experience (I am using a
fairly slow Pentium IV, and do not experience C-mode slowness when
editing the Emacs source code).

[Alan Mackenzie]

Hi, Richard!

On Mon, Jan 05, 2009 at 07:01:26PM -0500, Richard M Stallman wrote:
> I too find C mode painfully slow these days.  I wonder what changed to
> make it so much slower than it was.  Perhaps the only relevant change
> is that Font Lock is now enabled; I never used to enable it.  Even so,
> it must make a lot of users unhappy.

First of all, sorry.

Font lock consumes masses of cycles, yes.  Set
`font-lock-maximum-decoration' to 2; that might make it fast enough.

But no, not too many users seem too unhappy about it - at least, not
those who complain on bug-cc-mode@... or emacs-devel.  Or, more
accurately, most complaints received have been because of specific
pessimisations which only manifest themselves in peculiar files (as in
the current thread) or in very big files.  These are relatively easy to
fix.

What speed of processor have you got?  With my 1.2 GHz Athlon, there is
a brief pause (~ 1 second) every now and then, while caches get checked
(not cheques getting cashed), and suchlike.  When I type, sometimes the
display is quite a few characters behind the keyboard.  But pauses like
that don't bother me too much; I just carry on typing, subconsciously
confident that Emacs will cache up soon enough.  But that's just me.

From the tone of your post, I'm guessing that you're generally irritated
by sluggishness in C Mode rather because of any specific long wait.
There is at least one "peculiar" file in emacs, .../src/lisp.h in which C
Mode is sluggish locating BODs because of the number of #defines in it -
that shows up if you do C-x 4 a at its EOF.  But is there anything in
particular you've noticed which is particularly painful?

I'd guess that C Mode slowed down quite a lot in CC Mode 5.30, when the
font locking was taken into CC Mode, with the default decoration level
(3), prioritising accuracy over speed.  Since then, I've added a bit of
scanning to set syntax-table text properties so that the following sort
of stuff doesn't foul up parse-partial-sexp:

    #define LBRACE {
    #error unbalanced single quotes can't be ruled out.

K&R style declarations (more precisely, having to check for them) causes
quite a slow down.  They're never(?) used in modern C code, so maybe we
could flush them from .../src/*.[ch] and make K&R parsing an option.

--
Alan Mackenzie (Nuremberg, Germany).

[Stefan Monnier]

> (not cheques getting cashed), and suchlike.  When I type, sometimes the
> display is quite a few characters behind the keyboard.  But pauses like
> that don't bother me too much; I just carry on typing, subconsciously
> confident that Emacs will cache up soon enough.  But that's just me.

I can live with them as well, but such files were perfectly editable
without any such delays even on much slower processors, not that
long ago.  So I don't think such delays should be considered acceptable,
except in pathological cases.

> I'd guess that C Mode slowed down quite a lot in CC Mode 5.30, when the
> font locking was taken into CC Mode, with the default decoration level
> (3), prioritising accuracy over speed.  Since then, I've added a bit of

It looks like it indeed.  I'm wondering if it was a good decision: maybe
I've just learned to live with the previous inaccuracies, but I have the
impression that I've seen more font-locking errors with the "new"
font-locking than with the old one: maybe it's more accurate, but only
as long as its cache doesn't get out-of-date/garbled.
More specifically, I still regularly see function definitions that are not
properly highlighted (presumably because the cache indicates that it's
not at top-level, even tho it is).

Also accuracy is basically impossible in the presence of funky CPP
macros, and it's already very difficult in actual code, because of
things like

   #if foo
      {
        blabla
   #else
      blibli;
      {
        toto;
   #endif

Emacs has traditionally taken the "shallow" approach, using lexical cues
rather than full blown syntactic analysis, and this has served it fairly
well.  Both approaches have their downsides.


        Stefan

[Richard Stallman]

    > I too find C mode painfully slow these days.  I wonder what changed to
    > make it so much slower than it was.  Perhaps the only relevant change
    > is that Font Lock is now enabled; I never used to enable it.  Even so,
    > it must make a lot of users unhappy.

    You are likely using an exceptionally slow computer.

I am using a Lemote Yeeloong, which could well be the machine
we will recommend to the public for the coming year or more.

I just did some testing and I am obliged to modify my complaint.
It seems that the slowness is mainly in _loading_ C mode.

Visiting buffer.h, when it is the first C file I visit,
just took 20 seconds.  However, once C mode is loaded,
visiting buffer.h again is pretty fast.

[Chong Yidong]

Richard M Stallman <rms@...> writes:

> I just did some testing and I am obliged to modify my complaint.  It
> seems that the slowness is mainly in _loading_ C mode.
>
> Visiting buffer.h, when it is the first C file I visit, just took 20
> seconds.  However, once C mode is loaded, visiting buffer.h again is
> pretty fast.

Loading cc-mode in Emacs 23 takes 0.33 seconds on my current laptop
(about 0.3 seconds for Emacs 22).

Apparently, the netbook you are using (Lemote Yeeloong) has a 600-800
MHz processor.  That shouldn't be so slow as to yield 20 second loading
times.  There's likely something strange going on.  Could you try with
Emacs 22 and see if the problem recurs there?

[Richard Stallman]

    Apparently, the netbook you are using (Lemote Yeeloong) has a 600-800
    MHz processor.  That shouldn't be so slow as to yield 20 second loading
    times.  There's likely something strange going on.  Could you try with
    Emacs 22 and see if the problem recurs there?

Visiting the small .c file below takes around 2 seconds in Emacs 22.
It takes 15 or 20 seconds in Emacs 23.  In both cases I used -Q
and visited the file first thing after Emacs started.

char *p = 0;

main ()
{
  *p = 1;
}

[Alan Mackenzie]

Hi, Richard,

On Thu, Jan 08, 2009 at 06:37:52AM -0500, Richard M Stallman wrote:
>     Apparently, the netbook you are using (Lemote Yeeloong) has a 600-800
>     MHz processor.  That shouldn't be so slow as to yield 20 second loading
>     times.  There's likely something strange going on.  Could you try with
>     Emacs 22 and see if the problem recurs there?

> Visiting the small .c file below takes around 2 seconds in Emacs 22.
> It takes 15 or 20 seconds in Emacs 23.  In both cases I used -Q
> and visited the file first thing after Emacs started.

Could you try in Emacs-23 with the CC Mode from Emacs-22, and vice
versa?  CC Mode is in .../lisp/progmodes/cc-*.elc

Thanks!

--
Alan Mackenzie (Nuremberg, Germany).

[davemilter]

On Mon, Jan 5, 2009 at 2:31 AM, Alan Mackenzie <acm@...> wrote:
> Hi, Dave,
>
> I'm actually working on this bug at the moment.  With the Yuletide break,
> and so on, I've kind of lost the thread a bit and I got bogged down in
> the complexities of the code.  I should have got back to you sooner;
> sorry!
>

Hi,
was something changed?

I recently give a try emacs 23.1, and the bug still here:
font-lock-fontify-region                                   27
128.543602    4.7608741481
font-lock-default-fontify-region                           27
128.54316500  4.7608579629
font-lock-fontify-keywords-region                          27
128.14446099  4.7460911481