• Previous chapter
• Table of contents

Chapter 4: Technical information

The following information describes the Yodl package from the point of the system administrator. Issues such as the installation of the package are addressed here.

4.1: Obtaining and installing Yodl

4.1.1: Configuring the yodl program

        configure    # Check out the bin/set-yo.sh script
        make
        make install

4.1.2: Installing the yodl program

    make all

    make install

4.1.2.1: Prerequisites for the installation

To successfully build and install the Yodl package, the following tools must be present:

A C compiler and run-time environment. A POSIX-compliant compiler, libraries and set of header files should work without problems. The GNU gcc compiler 2.7 and above should work without a flaw.

GNU make

Typical building programs, such as, install. Most Unixes will have these.

/bin/sh: a POSIX-compliant shell interpreter. The GNU shell interpreter bash works without problems.

A number of `standard' tools should be present: sed, grep, etc.. These tools must furthermore include the code generators bison and flex (yacc- and lex lookalikes) to genererate the grammar parsers. The GNU implementations of these tools work like a charm.

A command that converts groff input into viewable format. The default setting for this command is troff -Tascii -man.

4.2: Internal workings of the Yodl package

4.2.1: A single-pass interpreter

This is best illustrated with an example. Given the code,

DEFINESYMBOL(sym)

IFDEF(sym)
    (Symbol sym is defined!
     DEFINEMACRO(testmac)(0)(Testmac now stands for one thing.)
    )
    (Symbol sym is not defined!
     DEFINEMACRO(testmac)(0)(Testmac now stands for another thing.)
    )
    
testmac()

yodl will take the following actions:

The DEFINESYMBOL command updates an internal table of symbols, no other action is necessary here.

When the IFDEF command is parsed, yodl eats up all of the necessary parts from the input: the keyword itself, and three parameter lists. The IFDEF test obviously yields `true', and therefore yodl `pushes back' the true-list

Symbol sym is defined!
DEFINEMACRO(testmac)(0)(Testmac now stands for one thing.)

Next, yodl sees normal text (Symbol sym is etc.). This is sent to the output file.

After this, the DEFINEMACRO statement is seen, which defines testmac() as a placeholder for one thing.

Finally, testmac() is seen. The parser recognizes a user-defined macro having no arguments, which conforms to the definition. The text

Testmac now stands for one thing.

is pushed back.

Finally, yodl reads this last string from its input and sends it to the output.

The most complex part of the program is therefore the tokenizer (lexical analyzer): that part is responsible for matching tokens such as identifiers in the input, and for pushing back text. The pushing back is implemented as a relocatable buffer, which is expanded as text is pushed into it and which shrinks as the lexical analyzer processes input. I already achieved about 100% speed gain by optimizing the tokenizer, but this implementation can still be somewhat of a CPU hog. (But hey, premature optimzation is the root of all evil, says D.E. Knuth. So why optimize, as long as you don't know whether you're being premature about it?)

The lexical analyzer is furthermore responsible for the line continuation feature (see section 2.2.2) and for the SUBST mechanism (see section 2.3.30). These mechanisms are also implemented with the push-back method.

4.2.2: Macros of the converters

The place to start looking is the file shared.yo, which is used by all converters. This file defines most commands that are `global' in the sense that all converters implement them: e.g., bf, em and so on. The implementation is per macro as follows, illustrated for e.g., bf (the actual implementation may be slightly different):

DEFINEMACRO(bf)(1)(\ 
        whenlatex(latexcommand({\bf )ARG1+latexcommand(}))\ 
        whenhtml(htmlcommand(<strong>)ARG1+htmlcommand(</strong>))\ 
        whenman(mancommand(\fB)ARG1+mancommand(\fP))\ 
        whensgml(sgmlcommand(<bf>)ARG1+sgmlcommand(</bf>))\ 
        whentxt(ARG1))

The macro expands to a set of commands that are active per output format, using latexcommand, htmlcommand etc..

The file shared.yo is included in its turn by tex.yo, html.yo and so on. These separate files are very short; their function is to:

define the conversion type with an appropriate symbol (latex for latex conversions, html for HTML, and so on),

activate character tables (when necessary),

include shared.dm,

define special macros for the particular conversion (when necessary).

4.2.2.1: Adding a new macro

This section describes how to add a new user-defined macro. For the sake of simplicity, a macro fname(name) is defined to typeset a filename. The macro maps to em() in LaTeX output or to bf() in other formats.

The place to insert the macro is lib/yodl/shared.yo.in in the Yodl source tree. This is the `master file' from which both the final macrofile and relevant documentation is created.

The macro must appear in alphabetical order in the list of all other macros, this is relevant for the documentation. Hence, the fname macro would appear somewhere near footnote() though before it.

The documentation about the hypothetical macro must be set between two tags @startdoc and @enddoc. The make process looks for these tags. The documentation itself must appear as a macro which is (appropriately) called macro, and expects two arguments: the new macro that is being defined, and its description:

@startdoc
macro(fname(file))
(Sets a filename in either boldface or italics,
depending on the output format.)
@enddoc

Following the documentation, the macro itself is defined using whenformat() commands and/or formatcommand():

    DEFINEMACRO(fname)(1)(\ 
        whenlatex(em(ARG1))\ 
        whenhtml(bf(ARG1))\ 
        whenman(bf(ARG1))\ 
        whenms(bf(ARG1))\ 
        whentxt(bf(ARG1))\ 
        whensgml(bf(ARG1)))

The new macro is now in place. A new make job should install it all, and if necessary, make the appropriate new documentation files.

After this, please kindly mail me your addition to the Yodl package. I will include it (if it's worth while), stating full credits too.

• Previous chapter
• Table of contents

Please send Yodl questions and comments to yodl@icce.rug.nl.

Please send comments on these web pages to (address unknown)

Copyright (c) 1997, 1998, 1999 Karel Kubat and Jan Nieuwenhuizen.

Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.