Re: [PATCH] add option --no-anchor-db and configuration variable no_anchor_db

[Shigio Yamaguchi]

I have estimated Hideki's patch using Linux and FreeBSD kernel source code.

0. Comparison of method

Htags load tags from a source file to generate hypertext.
There are two method for doing this job.

Method1:  Using anchor-db (Current GLOBAL)
        (1) Convert tag files into a anchor-db using file name
            for indexing.
        (2) For each source file, read tags from anchor-db.

Method2: No anchor-db (patch by Hideki IWAMOTO)
        (1) For each source file, parse a source file directly
            and extract tags without anchor-db.

1. Performance

                        Comsumption time
Source code     Using anchor-db         No anchor-db    Improvement(%)
----------------------------------------------------------------------
FreeBSD 4.3      3428 seconds            3599 seconds   -4.99%(worse)
Linux-2.4.18    15775 seconds           14499 seconds   +8.09%(better)
----------------------------------------------------------------------

Comment:
Performance has not necessarily improved but in large project it have
clearly improved. Performance is more important in large project, I think.


2. Amount of disk use

Using anchor-db

Source code     Using anchor-db No anchor-db
--------------------------------------------
FreeBSD 4.3      30.1MB         0MB             
Linux-2.4.18    100.1MB         0MB
--------------------------------------------

Comment:
Of course, it is a clear improvement.


3. Problem

There is one problem in 'no anchor-db' method.
The parser to which htags(1) refers is not necessarily the same as
what to gtags(1) refers, because a user can redefine it by configuration
file (gtags.conf).

I think we can solve the problem by adding following disclaimer
to the online manual.

        You must use the same parser for both gtags and htags.
        If you use the default parser, it is not necessary to
        consider for it.

Only gtags had called the parser up to now. But hereafter, if we adoopt
this method, the parser will be called by both gtags and htags.
I think, this is not a considerable problem though is not preferable either.

Besides, is there something problem?

----

I want to adopt Hideki's patch. But I don't come to think of switching
the method using configuration variable (no-anchor-db), because general
user cannot understand the difference. So, I'll replace current method
with Hideki's 'no anchor-db' method completely.

What do you think?

> This method is faster in my environment.
> 
> 
> Index: htags/htags.in
> ===================================================================
> RCS file: /cvsroot/global/global/htags/htags.in,v
> retrieving revision 1.123
> diff -u -r1.123 htags.in
> --- htags/htags.in    24 May 2003 14:55:07 -0000      1.123
> +++ htags/htags.in    25 May 2003 12:57:47 -0000
> @@ -82,6 +82,7 @@
>  $'definition_header='after';                 # {no|after|before}
>  $'other_files = 0;                           # 1: list other files
>  $'map_file = 1;
> +$'use_anchor_db = 1;                         # make anchor db
>  #
>  # tag
>  #
> @@ -225,6 +226,9 @@
>  if (&'getconf('no_map_file')) {
>       $'map_file = 0;
>  }
> +if (&'getconf('no_anchor_db')) {
> +     $'use_anchor_db = 0;
> +}
>  if ($var1 = &'getconf('show_position')) {
>       $'show_position = $var1;
>  }
> @@ -657,6 +661,8 @@
>               shift;          # --gtagslabel is estimated only once.
>       } elsif ($opt =~ /^--no-map-file$/) {
>               $'map_file = 0;
> +     } elsif ($opt =~ /^--no-anchor-db$/) {
> +             $'use_anchor_db = 0;
>       } elsif ($opt =~ /^--line-number$/) {
>               $'nflag = 'n';
>       } elsif ($opt =~ /^--other$/) {
> @@ -1013,8 +1019,12 @@
>  #
>  # (#) make anchor database
>  #
> -print STDERR "[", &'date, "] ", "(#) making temporary database ...\n" if ($'
vflag);
> -&anchor'create();
> +if ($'use_anchor_db) {
> +     print STDERR "[", &'date, "] ", "(#) making anchor database ...\n" if (
$'vflag);
> +     &anchor'create();
> +} else {
> +     print STDERR "[", &'date, "] ", "(#) making anchor database ...(skipped
)\n" if ($'vflag);
> +}
>  #
>  # (9) make HTML files ($SRCS/*)
>  #     USING TAG CACHE, %includes and anchor database.
> @@ -2120,7 +2130,11 @@
>       #
>       # load tags belonging to this file.
>       #
> -     &anchor'load($file);
> +     if ($'use_anchor_db) {
> +             &anchor'load($file);
> +     } else {
> +             &anchor'parse($file) unless ($notsource);
> +     }
>       $command = "$'gtags --expand -$tabs ";
>       $command .= ($'w32) ? "\"$file\"" : "'$file'";
>       open(SRC, "$command |") || &'error("cannot fork.");
> @@ -2612,6 +2626,59 @@
>       }
>       close(ANCH);
>       if ($?) {&'error("'$command' failed."); }
> +     sub compare { $keys[$a] <=> $keys[$b]; }
> +     @ANCHORS = @ANCHORS[sort compare 0 .. $#keys];
> +     local($c);
> +     for ($c = 0; $c < @ANCHORS; $c++) {
> +             local($lno, $type, $tag) = split(/(\D)/, $ANCHORS[$c], 2);
> +             if ($type eq 'D') {
> +                     $FIRST = $lno;
> +                     last;
> +             }
> +     }
> +     for ($c = $#ANCHORS; $c >= 0; $c--) {
> +             local($lno, $type, $tag) = split(/(\D)/, $ANCHORS[$c], 2);
> +             if ($type eq 'D') {
> +                     $LAST = $lno;
> +                     last;
> +             }
> +     }
> +}
> +#
> +# parse: extract anchors by parsing specified file.
> +#
> +#    i)      $file   source file
> +#    go)     FIRST   first definition
> +#    go)     LAST    last definition
> +#
> +sub parse {
> +     local($file) = @_;
> +     local(@keys);
> +
> +     @ANCHORS = ();
> +     $FIRST = $LAST = 0;
> +
> +     foreach $db (&'get_taglist()) {
> +             local($option) = &'get_option($db);
> +             local($pipein) = "global -fn$option ";
> +             $pipein .= ($'w32) ? "\"$file\"" : "'$file'";
> +             open(PIPE, "$pipein |") || &'error("cannot fork.");
> +             while (<PIPE>) {
> +                     local($tag, $lno, $filename, $image) = split(/\s+/, $_,
 4);
> +                     local($type);
> +                     if ($db eq 'GTAGS') {
> +                             $type = ($image =~ /^#[ \t]*(define|undef)/) ? 
'M' : 'D';
> +                     } elsif ($db eq 'GRTAGS') {
> +                             $type = 'R';
> +                     } else {
> +                             $type = 'Y';
> +                     }
> +                     push(@keys, int($lno));
> +                     push(@ANCHORS, "$lno$type$tag");
> +             }
> +             close(PIPE);
> +             if ($?) { &'error("'$pipein' failed."); }
> +     }
>       sub compare { $keys[$a] <=> $keys[$b]; }
>       @ANCHORS = @ANCHORS[sort compare 0 .. $#keys];
>       local($c);
> Index: htags/manual.in
> ===================================================================
> RCS file: /cvsroot/global/global/htags/manual.in,v
> retrieving revision 1.47
> diff -u -r1.47 manual.in
> --- htags/manual.in   24 May 2003 14:55:07 -0000      1.47
> +++ htags/manual.in   25 May 2003 12:57:48 -0000
> @@ -85,6 +85,8 @@
>               Specify the main function name. The default is @code{main}.
>       @address@hidden, @option{--line-number}}
>               Print the line numbers. By default, doesn't print it.
> +     @address@hidden
> +             Doesn't generate anchor database. Instead, htags extracts ancho
rs by parsing each file.
>       @address@hidden
>               Doesn't generate javascript code.
>               By default, @name{htags} generates javascript code.
> @@ -221,6 +223,8 @@
>               List tags using table tag. The default is false.
>       @address@hidden(string)}
>               Suffix for normal html file. The default is 'html'.
> +     @address@hidden(boolean)}
> +             Doesn't generate anchor database. The default is false.
>       @address@hidden(boolean)}
>               Doesn't generate javascript code. By default, @name{htags} gene
rates javascript code.
>       @address@hidden(boolean)}
> 
> ----
> Hideki IWAMOTO  address@hidden
> 
> 
> 
> _______________________________________________
> Bug-global mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/bug-global
> 
> 
--
Shigio Yamaguchi <address@hidden> - Tama Communications Corporation
Spare mail address: <address@hidden>
PGP fingerprint: D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3