Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8023

Fix PlannerTest to handle error lines consistently

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: Impala 3.1.0
    • Fix Version/s: Impala 3.2.0
    • Component/s: Frontend
    • Labels:
      None
    • Epic Color:
      ghx-label-8

      Description

      PlannerTest works by running a query from a .test file, generating a plan, and comparing that plan to a "golden" expected result. It work well for most cases. We can use Eclipse's diff tools to compare the actual with expected files, and to copy across any expected changes that result from changes to the planner code.

      Once case that does not work are exceptions. When PlannerTest indicates encounters failure, it emits a line such as the following to the actual results file:

      org.apache.impala.common.NotImplementedException: Scan of table 't' in format 'RC_FILE' is not supported because the table has a column 's' with a complex type 'STRUCT<f1:STRING,f2:INT>'.
      

      Yet, in order for the comparison to pass, the golden file must contain the error in the following form:

      NotImplementedException: Scan of table 'functional.complextypes_fileformat' in format 'TEXT' is not supported because the table has a column 's' with a complex type 'STRUCT<f1:STRING,f2:INT>'.
      

      Note that the actual output includes the package prefix, the expected error must not include that prefix.

      The result is that:

      • When comparing files, one must learn to ignore the differences between these lines: the differences are not the reason why a test might fail, and
      • When "rebasing" a file, one must copy all expected changes except the error lines.

      In short, this is a real nuisance. Use a filter mechanism to fix this once and for all.

      The problem is that the text appended to the "actual output" is not the same as that used for comparison. A simple two-line fix will eliminate this issue.

      Current code in PlanerTestBase.handleException():

          actualOutput.append(e.toString() + "\n");
      ...
              String actualErrorMsg = e.getClass().getSimpleName() + ": " + e.getMessage();
      

      Proposed:

          String actualErrorMsg = e.getClass().getSimpleName() + ": " + e.getMessage();
          actualOutput.append(actualErrorMsg).append("\n");
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Paul.Rogers Paul Rogers
                Reporter:
                Paul.Rogers Paul Rogers
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: