Beaver - a LALR Parser Generator

Working With Scanners

Scanner API

Beaver defines a simple interface that it expect scanners will follow in order to feed it with tokens. A compliant scanner extends beaver.core.Scanner class end behaves according to the "contract" defined by that base class:

Scanner returns the next token it finds in the input when parser calls nextToken method.
It is expected that there may be IO exceptions, and scanner may throw Scanner.Exception to highlight its own critical issues
After the exception scanner should not break into pieces - parser will catch and report an exception and will keep calling nextToken, so in the end at least end-of-file token should be returned.

Most scanner generators are flexible enough to be able to fulfill this contract. For example, this is how JFlex specification may define a Beaver compliant scanner:

import beaver.Symbol;
import beaver.Scanner;

import example.ExampleParser.Terminals;

%%

%class LanguageScanner
%extends Scanner
%function nextToken
%type Symbol
%yylexthrow Scanner.Exception
%eofval{
	return new Symbol(Terminals.EOF, "end-of-file");
%eofval}

// ...

%%