Created by: gwideman, Oct 1, 2012 5:41 pm
Revised by: gwideman, Oct 5, 2012 3:29 am (17 revisions)

Heading

This series of pages gathers some notes during the process of installing and testing ANTLR 4 relative to the beta 1 beta 2 2012-10-03 reviewer edition of the ANTLR 4 book. It follows more or less the same progression as ANTLR4 Install and test notes 1, which for most purposes is obsolete at this point.

Preliminaries

  • Page numbers are relative to PDF of 2012-10-01 antlrreference_review_20121003.pdf.
  • Sample code for book: ...\ANTLR4-book-samples\tpantlr2-code.zip (Downloaded 2012-10-04, files dated 2012-10-02)
  • ANTLR 4 executables version: antlr-4.0b2-complete.jar
  • In the following, I plan to isolate the code for each example into its own module, so that each could be examined separately, without the clutter of the many other source files.

Organization of downloaded components

I found that with the proliferation of versions and naming schemes of different bits and pieces, my brain was losing traction. So I adopted the following scheme for downloading and then using. (Specific dirs noted here mostly for my own memory.)
Descr
Download to
[DLdir]\ANTLR
Install or deploy notes
Book pdfs
ANTLR4-book\...
Print; edits in Proj_ANTLR\_Book-edits
Book sample code
ANTLR4-book-examples\...
Proj_ANTLR\book_examples\ ... separate dirs for each version, within which
  • one copy where the orig sample dirs have chapter number prefixed to name (so they sort in order), and
  • another copy where all examples are split into separate dirs -- one dir per example, named by [chap].[num]. It's these examples that I'll actually exercise.
ANTLR 4 tool jar
ANTRL4-exec\...
  • Copy to: C:\usr\local\lib
  • Add C:\usr\local\lib\antlr-4.0b2-complete.jar to System's CLASSPATH
  • Batch file to launch tool (see below).
Source for ANTLR 4 tool
(for debugging and study)
ANTLR4-exec-src\...
[Figure out where best to put this when I get to setting up NetBeans for use with ANTLR examples.]



Chapter 1 Meet ANTLR

Installing Java

DAR2-1.1 says "you've got to have Java installed" (1.6 or above). The narrative could elaborate:
  • Java runtime: The ANTLR tool itself, being written in Java, requires a Java runtime
  • Java SDK: The ANTLR tool can generate the code for parsers in a variety of languages. If your main application code is written in java, then you will presumably want ANTLR to create a parser in Java code, and for the project that it's to be a part of, you'll need a Java SDK (JDK) installed, not just Java runtime.
    • And between the various "editions" of Java, it's Java SE (Standard Edition) that we want.
  • More notes at: ANTLR4 Install and test notes 1
As it happens,I already have Java 1.7.0_04 SDK set up. I quickly checked that using http://www.java.com/en/download/testjava.jsp. However, that failed -- but that turned out to be because the Java plugin in Firefox was set to disabled. Enabled it, and success.

Installing ANTLR

As noted in the table above, I established a common location for Java libraries, and for want of a more obvious spot, earlier I decided to pseudo follow the unix convention and put them at: C:\usr\local\lib.
  • Doh! Why did I decide on that path?? Why not just "C:\JavaLibs" or something equally simple and clear? Was there some other library that needed the unix-ish path, perhaps other subdirs relative to /usr/local?
Anyhow:
  • Place antlr-4.0b2-complete.jar at C:\usr\local\lib
  • Add C:\usr\local\lib\antlr-4.0b2-complete.jar to Java's CLASSPATH
  • Batch file: To make it easier to launch the ANTLR Tool, batch file antlr4.bat (named to correspond to similar shell file in the book):
java -jar C:\usr\local\lib\antlr-4.0b2-complete.jar %*
  • batch file for running "test rig", gtest.bat (equivalent to grun in book):
java org.antlr.v4.runtime.misc.TestRig %*
  • Place the batch files in: C:\_Commands, (which is in the Windows PATH).
Test:
  • In a command window, type: "antlr4".
    • Result: ANTLR runs, shows version number and command options.
Note: I guess there's some obtuseness here.
  • The batch file as written above relies on the jar file having a Manifest which specifies org.antlr.v4.Tool as the default class to run.
  • Since the jar file is in CLASSPATH, the invocation command could reference just the class to run:
    • java org.antlr.v4.Tool %*
  • I probably should decide whether the antlr-4 jar should always be in CLASSPATH, or (as in the book) added to CLASSPATH only temporarily by the batch file that runs the tool. The latter seems cleaner, but at the moment I'm suspecting that other programs, such as example code, will need runtime classes from that jar, and it's not clear how that will happen if the antlr 4 jar is not in CLASSPATH. To revisit later.
    • And actually, the book's script/batch file to run TestRig (grun, which I'm calling gtest) depends on the antlr 4 jar being in CLASSPATH.

Organizing the example code

The organization of the code as supplied (in which there are possibly several examples in each directory) may be fine for most readers, but I wanted to end up with separate directories for each example, so that I could be clear on exactly which files contribute to a particular executable. I expected that I might not know in advance exactly which files were needed for each example directory, and that I'd have to determine that as I attempted to build each example.
As a basis for that exercise, I thought it would be helpful to at least have the supplied directories sort in order of chapter. To that end I created a copy of the supplied example source tree, and prefixed each of the example directories with its chapter number. For example, example directory "install" becomes "20121001_beta_bychap\01_install", and so on.
I then created a second example source directory whose separate subdirectories for each example are named by the book's section number to which the example code corresponds. So example "20121001_beta_bychap\03_tour/Expr.g4" will form the basis of code in directory "20121001_beta_sep\03.01\".

Chapter 1 Cont'd

1.2 Executing ANTLR and Testing Recognizers

  • gtest Hello r -tokens
    • Failed with token recognition error. Fixed in 2012-10-02 examples
    • Hello.g4 still has same error as Ex-01.5 in alpha sample code: ANTLR4 Install and test notes 1. ie: must revise Hello.g4 to add \r (Carriage Return) to the WS (whitespace) rule.
    • With that fix, runs as book describes.
  • gtest Hello r -print
    • No output. Fixed in 2012-10-02 examples
    • This is because "-print" is not a valid option. Should be "-tree" (ie: book is in error).
    • gtest Hello r -tree
      • Works as in book
  • gtest Hello r -gui
    • Shows GUI window with graphic of parse tree, as in book.

Chapter 2 The Big Picture

2.6 A Starter ANTLR Project

02.6a

  • gtest ArrayInit init -tokens
    • Fails with token recognition error. STILL IN 2012-10-02 examples
    • Add /r to WS rule
    • OK, fine.
  • gtest ArrayInit init -tree
    • OK

02.6b

  • (Assuming /r fix to WS rule)
  • java Test with good input
    • Results as per book
  • java Test with bad input
    • Note: book shows superfluous '$' preceding user input. STILL IN 2012-10-03reviewer pdf
    • Otherwise, result as per book

02.6c

  • (Assuming /r fix to WS rule)
  • Book instructions omit compiling ShortToUnicodeString, but this seems to be ok as that gets compiled as side-effect of compiling Translate.java. Not sure why.
  • java Translate
    • Results as per book

Chapter 3 A Quick Tour

3.1 Matching an Expression Language

03.1a

  • javac Expr*.java would inadvertently compile ExprJoyRide.java, not part of this example
  • gtest Expr prog -gui t.expr
    • Works as per book

03.1b

  • javac ExprJoyRide.java Expr*.java -- listing ExprJoyRide.java separately is redundant.
  • java ExprJoyRide t.expr
    • Results per book, except no line breaks (on Windows).
      • Is this a problem in tree.toStringTree()? Or is no-line-breaks the norm, and the output was hand-formatted for the book?

03.1c

  • javac *.java (re)compiles all java files, not just this example
  • gtest LibExpr prog -tree with good input
    • Results as per book (except result shows additional \r, which is expected on Windows.)
  • gtest LibExpr prog -tree with bad input
    • Results as per book
  • gtest LibExpr prog -gui
    • Results as per book
    • Using [X] to quit the GUI causes console window to hang. See "GUI output" note below.
    • (Quitting GUI using [Close] button returns control to console window.)

Note on the GUI output window, and also the -ps option

All the -gui output examples suffer from the problem that if the user closes the window with the [X], then the window closes, but java continues to run, and doesn't return control to the console window. On Windows, one can invoke the Task Manager and kill java, thereby regaining control of the console window, but that's not obvious. Is there some way to get the GUI window to exit Java when it's closed? If the user uses the Close button then indeed the window closes and exits Java.
As a second point, there are a number of examples that specify the -ps output. That is interesting, but I think doesn't particularly improve the example versus the -gui option, since the main point is to be able to quickly see the graphical tree to confirm results. On Windows, the -ps output is not immediately useful unless one has separately obtained a Postscript viewer. In addition, if one neglects to specify an output file for the -ps option, the input file name gets used as the output file, so the input file gets overwritten.

Sections 3.2 - 3.5

03.2

  • Works as per book

03.3 ExtractInterfaceTool

  • antlr4 Java.g4 fails. Posted issue to antlr list, 2012-10-03.
    • Replace instances of $type=Identifier; with setType(Identifier);
    • Fixed in Java.g4 included in 2012-10-02 examples
  • ... generates, compiles and runs as per book.

03.4a Rows

  • Works as per book

03.4b Data

  • Works as per book

03.5a

  • gtest XML tokens -tokens t.xml
    • Succeeds, but file position numbers are way off due to insertion of large initial comment in the data file.

03.5b Java InsertSerialID

  • Again, problem with the Java.g4 grammar.
  • Also, the book shows skip where I think it intends to show channel(HIDDEN)
  • Works as per book

...