Created by: gwideman, Oct 4, 2012 4:31 pm
Revised by: gwideman, Oct 5, 2012 3:06 am (3 revisions)

Chapter 11 Wielding Lexical Black Magic

11.1a Using different channels

  • Works as per book

11.1b

  • Works as per book

11.2a

  • Works as per book

11.2b maximal munch ambiguity: Java.g4

  • Output differs from book because Java.g4 now sends WS to HIDDEN, which gets reported in output.
  • Otherwise OK

11.2c Python newlines

  • Does not discard newlines, throws three errors:
    • line 2:16 no viable alternative at input '\n'
    • line 4:3 extraneous input ',' expecting NEWLINE
    • line 6:3 extraneous input ')' expecting NEWLINE

11.3a XML Tags.g4

  • Output does not match book due to insertion of large copyright comment.
    • However, main function is OK

11.3b ModeTagsLexer

  • Works as per book

11.4a Tokenizing XML

  • gtest XML document ... XML-inputs/entity.xml
    • Completely hosed by inserted copyright comment
    • Removing the comment allows run which agrees with book output.
  • gtest XML document ... cat.xml
    • Same problem, fixed after removing copyright comment
  • gtest XML ... weekly-euc-jp.xml
    • Same problem, fixed after removing copyright comment

Chapter 12 API

12.2 Recognizers SimpleLexer etc

  • Compile step, page 238
    • SimpleLexer.java:17: error: SimpleLexer is not abstract and does not override abstract method getTokenFactory() in TokenSource
    • public class SimpleLexer implements TokenSource {

12.4 Tokens and Token factories

Same problem: SimpleLexer does not override abstract method getTokenFactory in TokenSource.

12.8 Unbuffered character and token streams CSV

  • Works as per book