[PATCH] Add 2-pass mode to gtags
|
View:
New views
9 Messages
—
Rating Filter:
Alert me
|
 |
[PATCH] Add 2-pass mode to gtags
by Hideki IWAMOTO
::
Rate this Message:
Hi.
To improve the performance of gtags, this patch adds 2-pass mode. The following three points are improved by moving use of function defined() from gtags-parser to gtags. 1. Decreases the frequency of parsing source file from three times to two times. 2. Decreases reading of GTAGS by keeping cache of GTAGS beyond the duration of parser process. 3. Increases parallelism of gtags-parser and gtags on multiprocessor system. [ 3-pass mode (current implementation) ] Pass 1: +--------------+ +-------+ source files ===>| gtags-parser |===========>| gtags |===> GATGS +--------------+ +-------+ Pass 2: +--------------+ +-------+ source files ===>| gtags-parser |===========>| gtags |===> GRTAGS GTAGS ===>| | +-------+ +--------------+ pass 3: +--------------+ +-------+ source files ===>| gtags-parser |===========>| gtags |===> GSYMS GTAGS ===>| | +-------+ +--------------+ [ 2-pass mode ] Pass 1: +--------------+ +-------+ source files ===>| gtags-parser |===========>| gtags |===> GATGS +--------------+ +-------+ Pass 2: +--------------+ +-------+ source files ===>| gtags-parser |===========>| gtags |===> GRTAGS +--------------+ GTAGS ===>| |===> GSYMS +-------+ Performance improvement: 2-pass mode is 20-30% faster than 3-pass mode on 1-CPU system, 40-50% faster on 2-CPU system. $ foreach cpumask ( 1 3 ) foreach? foreach label ( gtags gtags-2pass ) foreach? echo ==== cpumask:$cpumask label:$label ==== foreach? foreach dir ( linux-2.0.40 linux-2.2.26 linux-2.4.37 linux-2.6.31 ) foreach? rm -fr $dir; tar xfj ~/download/linux/$dir.tar.bz2; sync foreach? (cd $dir; time env GTAGSLABEL=$label taskset $cpumask gtags) foreach? end foreach? end foreach? end ==== cpumask:1 label:gtags ==== 8.360u 0.716s 0:09.83 92.2% 0+0k 0+0io 1pf+0w 23.297u 2.448s 0:28.14 91.4% 0+0k 0+0io 0pf+0w 50.083u 6.136s 0:59.39 94.6% 0+0k 0+0io 0pf+0w 116.895u 16.377s 2:34.86 86.0% 0+0k 0+0io 4pf+0w ==== cpumask:1 label:gtags-2pass ==== 6.532u 0.600s 0:07.75 92.0% 0+0k 0+0io 1pf+0w 18.221u 1.836s 0:22.01 91.0% 0+0k 0+0io 0pf+0w 39.414u 4.344s 0:46.94 93.2% 0+0k 0+0io 0pf+0w 92.857u 11.216s 1:58.56 87.7% 0+0k 0+0io 1pf+0w ==== cpumask:3 label:gtags ==== 8.092u 0.848s 0:08.42 106.0% 0+0k 0+0io 1pf+0w 23.105u 2.268s 0:24.24 104.6% 0+0k 0+0io 0pf+0w 50.067u 5.856s 0:58.19 96.0% 0+0k 0+0io 0pf+0w 114.619u 16.581s 2:16.91 95.8% 0+0k 0+0io 0pf+0w ==== cpumask:3 label:gtags-2pass ==== 6.372u 0.608s 0:05.55 125.5% 0+0k 0+0io 0pf+0w 18.189u 1.636s 0:15.46 128.1% 0+0k 0+0io 0pf+0w 38.886u 4.148s 0:35.89 119.8% 0+0k 0+0io 0pf+0w 90.541u 11.100s 1:38.27 103.4% 0+0k 0+0io 1pf+0w Diffstat: global/global.c | 24 ++++++++++++ gtags-parser/C.c | 82 +++++++++++++++++++++++++++----------------- gtags-parser/Cpp.c | 87 ++++++++++++++++++++++++++++------------------- gtags-parser/asm_parse.y | 32 +++++++++-------- gtags-parser/asm_scan.l | 7 ++- gtags-parser/gctags.c | 10 ++--- gtags-parser/gctags.h | 4 +- gtags-parser/java.c | 15 +++----- gtags-parser/php.l | 13 ++++--- gtags.conf.in | 8 ++++ gtags/gtags.c | 65 ++++++++++++++++++++++++++++------- 11 files changed, 231 insertions(+), 116 deletions(-) ---- Hideki IWAMOTO h-iwamoto@... _______________________________________________ Bug-global mailing list Bug-global@... http://lists.gnu.org/mailman/listinfo/bug-global |
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Shigio YAMAGUCHI-2
::
Rate this Message:
|
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Hideki IWAMOTO
::
Rate this Message:
|
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Shigio YAMAGUCHI-2
::
Rate this Message:
|
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Hideki IWAMOTO
::
Rate this Message:
|
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Hideki IWAMOTO
::
Rate this Message:
|
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Shigio YAMAGUCHI-2
::
Rate this Message:
|
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Hideki IWAMOTO
::
Rate this Message:
Hi.
> It seems that moving use of defined() from gtags-parser to gtags is bad idea. > I will reimplement 2-pass mode in another way. Rewritten version is attached. If an appropriate configuration variable is set, source files are parsed only once in the anchor loading of htags Diffstat: ChangeLog | 74 ++++++++--------- global/global.c | 136 ++++++++++++++++++++----------- gtags-parser/C.c | 188 +++++++++++-------------------------------- gtags-parser/Cpp.c | 203 ++++++++++++----------------------------------- gtags-parser/asm_parse.y | 71 +++++----------- gtags-parser/asm_scan.l | 22 ++--- gtags-parser/gctags.c | 41 +++++++-- gtags-parser/gctags.h | 49 +++++++++-- gtags-parser/java.c | 26 ++---- gtags-parser/manual.in | 8 + gtags-parser/php.l | 67 +++++---------- gtags.conf.in | 11 +- gtags/gtags.c | 120 +++++++++++++++++---------- htags/anchor.c | 96 ++++++++++++++-------- htags/anchor.h | 2 libutil/Makefile.am | 4 libutil/format.h | 29 ++++-- libutil/global.h | 1 libutil/parser.c | 84 +++++++++++++++++++ libutil/parser.h | 38 ++++++++ libutil/pathconvert.c | 18 +++- libutil/pathconvert.h | 7 - 22 files changed, 680 insertions(+), 615 deletions(-) On Mon, 23 Nov 2009 22:49:05 +0900, Hideki IWAMOTO wrote... > Hi. > > > Since there are a lot of changes in the parser, the output should be seen by > > a lot of people before the formal release, I think. > > When bree cache is small, current implementation of 2-pass mode is slower than default mode. > > $ foreach cpumask ( 1 3 ) > foreach? foreach label ( default 2pass ) > foreach? echo ==== cpumask:$cpumask label:$label ==== > foreach? foreach cachesize ( 8388608 16777216 33554432 67108864 ) > foreach? rm -fr linux-2.6.31; tar xfj ~/download/linux/linux-2.6.31.tar.bz2; sync > foreach? (cd linux-2.6.31; time taskset $cpumask env GTAGSLABEL=$label GTAGSCACHE=$cachesize gtags) > foreach? end > foreach? end > foreach? end > ==== cpumask:1 label:default ==== > 114.267u 24.085s 3:09.48 73.0% 0+0k 0+0io 1pf+0w > 115.367u 20.521s 2:55.73 77.3% 0+0k 0+0io 0pf+0w > 116.227u 18.081s 2:32.98 87.7% 0+0k 0+0io 0pf+0w > 116.671u 16.093s 2:24.36 91.9% 0+0k 0+0io 0pf+0w > ==== cpumask:1 label:2pass ==== > 91.353u 19.205s 3:48.79 48.3% 0+0k 0+0io 0pf+0w > 92.317u 16.501s 3:05.40 58.6% 0+0k 0+0io 0pf+0w > 92.797u 13.580s 2:08.52 82.7% 0+0k 0+0io 0pf+0w > 93.929u 11.080s 1:56.03 90.4% 0+0k 0+0io 0pf+0w > ==== cpumask:3 label:default ==== > 112.507u 23.593s 2:48.80 80.6% 0+0k 0+0io 1pf+0w > 112.991u 18.965s 2:30.06 87.9% 0+0k 0+0io 0pf+0w > 113.591u 17.305s 2:13.30 98.1% 0+0k 0+0io 0pf+0w > 114.831u 16.113s 2:11.92 99.2% 0+0k 0+0io 0pf+0w > ==== cpumask:3 label:2pass ==== > 88.773u 18.929s 3:17.01 54.6% 0+0k 0+0io 0pf+0w > 90.085u 15.108s 2:35.38 67.6% 0+0k 0+0io 0pf+0w > 90.877u 12.204s 1:44.62 98.5% 0+0k 0+0io 0pf+0w > 91.217u 10.540s 1:29.74 113.3% 0+0k 0+0io 0pf+0w > > It seems that moving use of defined() from gtags-parser to gtags is bad idea. > I will reimplement 2-pass mode in another way. > Since new 2-pass mode is implemented by adding tag type to output of gtags-parser, > the specifications of parser will be compatible with default mode. > > > On Thu, 22 Oct 2009 15:44:28 +0900, Shigio YAMAGUCHI wrote... > > Hi Hideki, > > > > > To improve the performance of gtags, this patch adds 2-pass mode. > > > > > > The following three points are improved by moving use of function defined() > > > from gtags-parser to gtags. > > > 1. Decreases the frequency of parsing source file from three times to two times. > > > 2. Decreases reading of GTAGS by keeping cache of GTAGS > > > beyond the duration of parser process. > > > 3. Increases parallelism of gtags-parser and gtags on multiprocessor system. > > > > Great!!! I named your code 'Giant steps'. > > > > Since there are a lot of changes in the parser, the output should be seen by > > a lot of people before the formal release, I think. > > How about releasing twice like follows? > > > > include Giant steps? enable Giant steps by default? > > ------------------------------------------------------------------------------ > > GLOBAL-5.7.7(beta) Yes No > > GLOBAL-5.8(formal) Yes Yes > > > > Thank you for your wonderful 'Giant steps'! > > -- > > Shigio YAMAGUCHI <shigio@...> > > PGP fingerprint: D1CB 0B89 B346 4AB6 5663 C4B6 3CA5 BBB3 57BE DDA3 > > ---- > Hideki IWAMOTO h-iwamoto@... > > > _______________________________________________ > Bug-global mailing list > Bug-global@... > http://lists.gnu.org/mailman/listinfo/bug-global Hideki IWAMOTO h-iwamoto@... _______________________________________________ Bug-global mailing list Bug-global@... http://lists.gnu.org/mailman/listinfo/bug-global |
|
|
Re: [PATCH] Add 2-pass mode to gtags
by Shigio YAMAGUCHI-2
::
Rate this Message:
|
20091020-gtags-2pass.patch (51K) | Free embeddable forum powered by Nabble | Forum Help |
