Re: Transforming XML or other intermediate representation of Texinfo; category system

[Patrice Dumas]

On Sat, Dec 18, 2021 at 06:05:53PM -0800, Robert Dodier wrote:
> Hi Pat, you mentioned it's not clear what I'm trying to do.
> 
> The Texinfo for such items looks like
> 
> @deffn foo (x, y, z)
> 
> Yadda yadda yadda, foo bar baz quux mumble blurf.
> 
> @opencatbox
> @category{Foobar}
> @category{Bazquux}
> @category{Mumbleblurf}
> @closecatbox
> 
> @end deffn
> 
> The easy part is to define @category{Foobar} as a macro which
> substitutes @ref{Category: Foobar, Foobar}. Also easy to define
> @opencatbox / @closecatbox as <div> / </div> .

What do you do for other output formats than HTML instead of
<div> / </div>?

> The hard part, for which I've concocted some scripts, is to extract
> the set of terms named as categories, and create the list of items
> which are tagged with each category. This is the operation for which I
> would like to inspect a parse tree of the Texinfo document.
> 
> The output of this second part is a bunch of stuff that looks like
> 
> @anchor{Foobar}
> @opencatbox
> @b{Category: Random blurfage}
> @ref{Item: foo, foo}
> @ref{Item: bar, bar}
> @ref{Item: baz, baz}
> @closecatbox
> 
> where foo, bar, and baz are @deffn's or @defvr's which contain
> @category{Foobar}.
> 
> So, in my world of dreams, I would get a parse tree for the document
> containing @deffn foo, etc., look for all the @category doodads, and
> build a lookup table to map categories to items. Then loop over all
> the categories, and emit the @anchor{whatever} and @ref{foo},
> @ref{bar}, etc., for each item claiming to be in category whatever.

You also need an @-command like @printindex, (which I call
@printcategories in the following), to print the categories table?
What is the Texinfo code corresponding to the formatting of this
table after expansion?

(the Texinfo code which generates output in a page like:
https://maxima.sourceforge.io/docs/manual/maxima_370.html
)

> In the current implementation, the stuff about looking for
> @category's, building the lookup table, and emitting the made-up
> Texinfo stuff for the categories is handled by the scripts I
> mentioned.

I see two possibilities to do that.  

In both cases, the texinfo code for categories would be protected
by @verbatim, and then the tree would be modified right after the
parsing, using a custom tree transformation (which is not possible
right now, but could easily be added).  The original code would be like

@verbatim
@opencatbox
@category{Foobar}
@category{Bazquux}
@category{Mumbleblurf}
@closecatbox
@end verbatim

@printcategories would also be protected in @verbatim, like

@verbatim
@printcategories
@end verbatim

The tree transformation code would get the @verbatim content parse its
content as Texinfo, expanding the macros (would probably be simpler if
those macros can operate on a Texinfo fragment and do not need to
operate on the whole document).


For the first possibility, the Texinfo tree corresponding to the parsing
of @opencatbox content would then be examined to get the categories
(presumably using the arguments of the @anchor or @ref @-commands
generated by the macros) and generate a Texinfo tree that can substitute
@printcategories.  This Texinfo tree could either directly be a perl
representation of Texinfo, or be some Texinfo code that is parsed into a
tree.


The second possibility would go through XML, the custom tree transformation
that expands the macros would simply replace @verbatim by another
non regular block Texinfo @-command, for instance with name opencatbox and
would also replace @printcategories @verbatim block by an @-command with
name printcategories.  Then the tree would be expanded as TexinfoXML,
with some code set up to have the TexinfoXML converter consider opencatbox
and printcategories to be regular @-commands.  The next step would be
done in an external tool you would do that starts from the augmented
TexinfoXML generated and that would substitute the opencatbox and
printcategories elements with TexinfoXML regular elements.  Then
txixml2texi would convert back the valid TexinfoXML to Texinfo, which
could then be converted as usual.


> 
> Hope that helps clarify what I'm trying to do, thanks for your interest.
> 
> Robert Dodier
> 
> PS. The scripts for this stuff can be found at:
> https://sourceforge.net/p/maxima/code/ci/master/tree/doc/info/
> namely build_html.sh.in, extract_categories1.sed, and extract_categories1.awk.
> However, I expect that stuff is all pretty much incomprehensible.