Re: Is handwritten faster?

[John R. Levine]

> > I'm wondering how much of that is the parser and how much is the lexer.
>
> Disclaimer: I'm totally new at lexers and parsers; all my other work
> is in other areas of the Ruby VM.
>
> I just tried uninlining the yylex function in Ruby but roughly 1/2 to
> 2/3 of that time is still in yyparse (aka ruby_yyparse) according to
> "perf report" with some informal tests.

A bison parser has semantic code that's called each time it matches a rule 
in the grammar to create the parse tree and symbol table or whatever.  No 
matter how your parser works, it has to do more or less the same semantic 
stuff.  It's hard to pick that out from the generic parsing bit using 
normal perf tools.

> For yyparse, perf sees sees accessing the giant (over 2K) yydefact array
> the main hotspot.  Not sure what could be done about that with a
> language as large as Ruby...

That's referenced once per token, hard to see how you could speed that up 
much.

It sounds to me like the best way to improve the startup speed would be to 
figure out some way to save the parsed version of the program like python 
does so you can slurp the whole thing in without having to lex or parse it 
again.

R's,
John