[IMPALA-7881] Visualize AST for easier debugging

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 3.0
• Fix Version/s: Impala 3.2.0
• Component/s: Frontend
• Epic Color: ghx-label-7
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Created: 21/Nov/18 23:49
• Updated: 14/Mar/19 14:55
• Resolved: 01/Mar/19 17:52

Description

The parser creates a "raw" AST (abstract syntax tree), which is then "decorated" by the analyzer. Often, when debugging the analyzer, one wants to see the state of the tree. At present, doing so using an IDE's debugger is tedious as one has to slowly navigate within the tree.

 Provide a debug tool that visualizes the tree. For example:

<root> (SelectStmt): {
. isExplain: false
. analyzer: <Skip Analyzer>
. withClause: <null>
. orderByElements: [
. . 0 (OrderByElement): {
. . . expr (SlotRef): {
...
. selectList (SelectList): {
. . planHints: []
. . isDistinct: false
. . items: [
. . . 0 (SelectListItem): {
. . . . expr (SlotRef): {
...
. . . . . rawPath: [
. . . . . . 0: "id"
. . . . . ]
. . . . . label: "id"

Many improvements can be made. (Format as JSON, export to a nice JSON visualizer, etc.) The purpose here is to just get started.

To avoid the need to write code for every AST node class (of which there are many), use Java introspection to walk fields directly. The result may be overly verbose, but it is a quick way to get started.

The idea is to use the visualizer in conjunction with a unit test:

  @Test
  public void test() {
    String stmt =
        "SELECT id, int_col + 10 AS c" +
        " FROM functional.alltypestiny" +
        " WHERE id > 10" +
        " ORDER BY c";
    ParseNode root = AnalyzesOk(stmt);
    AstPrinter.printTree(root);
  }

When debugging an issue, create a test. If things are not working, temporarily insert a call to the visualizer to see what's what. Remove the call when done.

Poking at the AST from outside a unit test (perhaps from the Impala shell) is a larger project, beyond the scope of this ticket.