Created by: gwideman, May 30, 2012 5:00 pm
Revised by: gwideman, May 31, 2012 1:40 am (9 revisions)

Chapter 5 Building Language Applications

Section 5.1, Decoupling Application Code

Ex 05.01a, PropertyFilePrinter

  • Begins p99
  • No supplied example code. Instead I assembled:
    • Grammar from PDF p99 bottom (with the @members)
    • Code from the PDF p100, "first application".
      • main class TestPropertyFilePrinter, within which I made class PropertyFilePrinter (basically copying the pattern in the supplied TestPropertyFile.java, which must be for a later exercise).
    • PropertyFile-input --> out
  • ANTLR generates code that has an error
    • the prop rule is missing a semicolon on the end of the visitProperty() call. Fixed in grammar
  • IDE reports error on PropertyFilePrinter extends PropertyFileParser
    • no default constructor
    • ... then there's corresponding error at PropertyFilePrinter parser = new PropertyFilePrinter(tokens); PropertyFilePrinter() cannot be applied to CommonTokenStream
    • Had to add a constructor to PropertyFilePrinter

Ex 05.01b, PropertyFileLoader

This is basically the same code, except using a different descendant of PropertyFileParser
  • p100
  • Code:
    • Copied all from 05.01a src
    • Renamed *Printer to *Loader
    • Replaced Printer code with Loader code from p100
    • I did NOT rerun the ANTLR process
    • Built OK
  • Run
    • PropertyFile-input --> out
    • To see result, added to end of main(): System.out.print(parser.props);

Ex 05.01c Implementing Applications with Parse Tree Listeners

  • p101. This is the example to which the provided sample code seems to apply
  • Commands: p103
    • Some weirdness: Commands show producing an input file with cat, and then using < to feed it to he program. Actually a sample file is provided, and also the program understand its first argument to be a filename, so no need to use <.
  • Copy
    • PropertyFile.g4
    • PropertyFileListener.java
    • TestPropertyFile.java
    • PropertyFile-input --> out
  • Builds and runs OK
    • If run with rig, produces tree matching PDF
    • Running TestPropertyFile produces expected output

Ex 05.01d Labeling Rule Alternatives

  • p104
  • Commands: none provide
  • Copy
    • Expr.g shows rules without labels, LExpr.g shows rules with labels, so use the latter
    • TestLExpr.java
    • Expr-input --> out contains: 1+2*3
  • Build
    • failed due to presence of EvalVisitor class in TestLExpr (for later example?), and also launchvisitor and launchprop code in main()
      • I commented out all that, and build succeeded
  • Run
    • Using rig, produces parse tree. Running TestLExpr produces representation of tree, and "stackresult = 7", the correct answer

Section 5.2 Sharing Information among Listener Methods

Ex 05.02a

  • p105
  • Commands: p107
  • Copy
    • LExpr.g shows rules with labels
    • TestLExpr.java
    • Expr-input --> out contains: 1+2*3
  • Build
    • Failed for same reason as previous example.
      • Comment out EvalVisitor class in TestLExpr (for later example?), and also launchvisitor and launchprop code in main()
      • I commented out all that, and build succeeded
  • Run
    • Using rig, produces parse tree. Running TestLExpr produces representation of tree, and "stackresult = 7", the correct answer
  • BUT, we need to get the outputs from the parts commented out, so we need to get ANTLR to produce the LExprBaseVisitor class.
    • Not explained in this section? Ah, see subsequent section on p107. Run ANTLR with -visitor option.
    • OK, builds without error
    • Runs and produces output as in PDF
This example includes both the evalVisitor and evalProp code. I guess that exhibits all the code, but it ends up muddling together the two approaches, I think. (I'll probably end up splitting these apart for clarity.)

Section 5.3 Loading CSV Data

Ex 05.03

  • p113
  • Commands: none provided
  • Copy
    • CSV.g4
    • LoadCSV.java
    • CSV-input --> out
  • Build: OK
  • Run
    • Output is more elaborate than in PDF, probably different string conversion for loader.rows List<Map<String,String>>

Ex 05.04 Translating JSON to XML

  • p115
  • Commands: none supplied
  • Copy
    • JSON.g4
    • JSON2XML*.java
    • JSON-input --> out
  • Build -- OK
  • Run
    • java JSON2XML JSON-input
    • Again '<' not needed.

Ex 05.05 Generating a Call Graph (Cymbol)

  • p119
  • Commands
    • p123, just how to invoke program
  • Copy
    • Cymbol.g4
    • CallGraph.java

Ex 05.06 Validating Program Symbol Usage

  • p123
  • Commands
    • Sample invocation p124
  • Copy
    • Cymbol.g4
    • CheckSymbols.java
    • DefPhase.java, RefPhase.java
    • Symbol.java, MethodSymbol.java, VariableSymbol.java
    • BaseScope.java, Scope.java, GlobalScope.java, LocalScope.java
    • vars.cym, vars2.cym --> out
  • Build: OK
  • Run
    • Invocation as on p124 produces same result as in PDF

Chapter 6 Error Reporting and Recovery

Examples based on E.g4 and TestE.java

6.1 A Parade of Errors

  • Copy
    • E.g4
    • TestE.java
    • Build OK
  • Run p133
    • Same output as PDF

6.2 Altering and redirecting ANTLR Error Messages

  • Copy
    • TestE_Listener.java
  • Build OK
  • Run p136
    • No Ctrl-Z shown
    • Same output, but output lines in different order.
  • Copy
    • TestE_Dialog.java
  • Build OK
  • Run p137
    • OK, but why does error message jam together the 'int int' into 'intint'?
  • Copy
    • TestE_Listener2.java
  • Build
  • Run as in sample p137
    • No Ctrl-Z shown
    • Prints each error message line (such as 'line 1:8...') twice

Ex 06.03 Automatic Error Recovery Strategy: F.g4

  • p140
  • Copy
    • F.g4
  • Build: OK
  • gtest F s -gui
    • produces same tree as PDF p141

Ex 06.03b Recovery from Mismatched Tokens: Back to E.g4

  • p142
  • Copy
    • E.g4
    • TestE_Tree.java
  • Build OK
  • Runs
    • p143: OK error message from parser
    • but error on hard-code postscript output path "/tmp/t.ps"
      • OK fixed that (revised to "./t.ps"), but of course on Windows there's no way to view it. Apparently Ghostscript is the way to go. (You'd think that Adobe Reader could manage to display PS by now, but nooooo.)

Detour to "install Ghostscript"

... back to the regularly scheduled exercises...

  • p144: generated t.ps indeed looks like Fig 24.
  • p144-145 sample input
    • Generated ps diagrams match PDF's Fig 25-27

Ex 06.03c Catching failed semantic predicates: Vec.g4

  • p146
  • Copy
    • Vec.g4, VecMsg.g4
    • TestVec*.java
    • Vec-input --> out
    • ...though this does mess together two examples
  • Build: OK
  • Run
    • Arrrg, dies on same hard-coded path. Changed to "./t.ps"
    • p147: Expected "line 1:10 rule ints failed", but got "line 1:8 no viable alternative at input ',' "
      • Appears to be same error in the input, but different message
    • p147 VecMsg example
      • Expected improved error message, but got same "no viable alternative" message.

Ex 06.03d Example using Pred.g4

  • p147
  • Copy
    • Pred.g4
    • TestPred.java
  • Run
    • java TestPred
      • Enter x = 0; CR Ctrl-Z
        • Result is "line 1:5 rule assign failed predicate: {$v.int>0}?"
        • Improved message in a "simple action" is not provided in the sample code.
      • Enter x = 1; CR Ctrl-Z
        • Result is "assign x to ". probably should be "assign 1 to x". So that looks like the action in the assign rule is wrong.

Error Recovery Failsafe: Reuse Ex 06.01

  • Commands on p148
  • Run TestE
    • Output matches PDF
    • Could step through in debugger
  • gtest E prog -gui
    • Produces trees like Fig 29 p150

Ex 06.04 Error Alternatives: Call.g4

  • p149
  • Copy
    • Call.g4
    • TestCall.java
    • Call-input --> out
  • Build: OK
  • Run as on p150
    • Output matches PDF

Ex 06.05 Altering ANTLR's Error Handling Strategy

  • p151
  • Copy
    • E.g4
    • TestBail.java
    • BailErrorStrategy.java
  • Build: OK
  • Run per p153
    • First invocation OK
    • Invocation: j TestBail fails. Presumably relies on "j" aliased to "java".
      • Revise to java TestBail. Runs OK
  • Adding class MyErrorStrategy as on p153
    • reportNoViableAlternative signature doesn't match ancestor
      • BaseRecognizer --> Parser
    • notifyListeners --> notifyErrorListeners
    • e.offendingToken not accessible here because it's protected --> getOffendingToken()
    • parser.setErrorHandler(new MyErrorStrategy());
    • Builds OK
    • Run: repeat java TestBail on input class T { int i; }
      • Output: Line 1:0 token recognition error at: '#'
      • This is confusing and seems wrong.
        • In the example using BailErrorStrategy, the output shows "token recoognition error", as the output from a NoViableAltException.
        • But then the narrative says the standard message for reportNoViableAlternative() is "no viable alternative at input X", which is indeed in the DefaultErrorStrategy source.
      • So where does the text "token recognition error at" come from? And under what circumstance would "no viable alternative" appear instead?
      • Ah, problem appears to be that this is a LexerNoViableAltException, not a NoViableAltException, and neither exception types is a descendant of the other. At any rate, LexerNoViableAltException is not specifically dispatched by DefaultErrorStrategy.reportError, so never calls reportNoViableAlternative().
        • Furthermore, the string for LexerNoViableAltException.toString returns "NoViableAltException"! Arrrrg!
      • OK, at this point I'm confused. I don't know where the key problems are, whether in the narrative, or the example or the ANTLR code.

End of Chapter 6