Go up to Top
Go forward to 2 Annotations to be handled uniformly
1 Introduction
Parsers generated by tools like lex/yacc or flex/bison work in two
stages: First the input sequence of characters is converted into a
stream of lexemes, and second the stream of lexemes is converted into
an abstract syntax tree (AST). (This corresponds to the first two
steps of the CASL parsing process, i.e. lexing and context free
parsing.) Using a standard tool like bison to create the context free
parser depends on two assumptions for this process.
- Every part of the source text that should
find its way into the AST needs to be present in the lexeme sequence
which is produced in the first stage.
- Each lexeme in a well formed lexeme sequence
is `consumed' by some terminal symbol on the right hand side of a
rule in the grammar when the lexeme syntax is converted to an AST in
the second stage.
These assumptions are reasonable if the lexer can recognize and ignore
parts of the input character sequence that are not to be represented
in the AST, and if all parts of the text that are to be represented in
the AST are present in the relevant rules of the context free grammar.
However, in [CoF98] annotations and comments are not
represented in the grammar describing the concrete syntax of CASL,
but still required to be represented in the abstract syntax tree.
Depending on whether or not lexemes for comments and annotations are
generated by the lexer this violates one of the assumption above: if
lexemes for annotations are not present it violates (1), otherwise
(2). So given the current specification of CASL's concrete syntax
and the requirements that all comments and annotations must be
represented in the AST, flex/bison like tools cannot be used for
building CASL lexers and context free parsers.
Sections 2 and 3
describe a possibility of handling comments and annotations that
allows CASL lexers/parsers to be built by flex/bison like tools.
Section 4 describes how the INKA frontend is
designed assuming this scheme of handling comments and annotations.
CoFI
Note: T-7 -- Version: 1 -- 28 April 1999.
Comments to schairer@dfki.de