Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7881

Visualize AST for easier debugging

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: Impala 3.0
    • Fix Version/s: Impala 3.2.0
    • Component/s: Frontend
    • Labels:
      None
    • Epic Color:
      ghx-label-7

      Description

      The parser creates a "raw" AST (abstract syntax tree), which is then "decorated" by the analyzer. Often, when debugging the analyzer, one wants to see the state of the tree. At present, doing so using an IDE's debugger is tedious as one has to slowly navigate within the tree.

       Provide a debug tool that visualizes the tree. For example:

      <root> (SelectStmt): {
      . isExplain: false
      . analyzer: <Skip Analyzer>
      . withClause: <null>
      . orderByElements: [
      . . 0 (OrderByElement): {
      . . . expr (SlotRef): {
      ...
      . selectList (SelectList): {
      . . planHints: []
      . . isDistinct: false
      . . items: [
      . . . 0 (SelectListItem): {
      . . . . expr (SlotRef): {
      ...
      . . . . . rawPath: [
      . . . . . . 0: "id"
      . . . . . ]
      . . . . . label: "id"
      

      Many improvements can be made. (Format as JSON, export to a nice JSON visualizer, etc.) The purpose here is to just get started.

      To avoid the need to write code for every AST node class (of which there are many), use Java introspection to walk fields directly. The result may be overly verbose, but it is a quick way to get started.

      The idea is to use the visualizer in conjunction with a unit test:

        @Test
        public void test() {
          String stmt =
              "SELECT id, int_col + 10 AS c" +
              " FROM functional.alltypestiny" +
              " WHERE id > 10" +
              " ORDER BY c";
          ParseNode root = AnalyzesOk(stmt);
          AstPrinter.printTree(root);
        }
      

      When debugging an issue, create a test. If things are not working, temporarily insert a call to the visualizer to see what's what. Remove the call when done.

      Poking at the AST from outside a unit test (perhaps from the Impala shell) is a larger project, beyond the scope of this ticket.

        Attachments

          Activity

            People

            • Assignee:
              Paul.Rogers Paul Rogers
              Reporter:
              Paul.Rogers Paul Rogers
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: