Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-28064

Add cause to ParseException for diagnosability purposes

    XMLWordPrintableJSON

Details

    Description

      The ParseException contains high level information about problems encountered during parsing but currently the stacktrace is pretty shallow.

      select * from author where  true > fname in ('Alex','Rob') 
      

      Currently the query above will fail with the above stacktrace:

      org.apache.hadoop.hive.ql.parse.ParseException: line 4:41 cannot recognize input near 'in' '(' ''Alex'' in expression specification
              at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
              at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:97)
              at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:89)
              at org.apache.hadoop.hive.ql.Compiler.parse(Compiler.java:172)
              at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:105)
              at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
              at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
              at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
              at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
              at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
              at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
              at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
              at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
              at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
      

      The end-user gets a hint about what the error might be but the developer has no way to tell how far we went into parsing the given statement and which grammar rule failed to pass.

      In some cases we are catching the RecognitionException which shows exactly at which point the parser failed to advance so we could pass it as a cause inside the ParseException. With this change the stacktrace would be similar to the one below:

      org.apache.hadoop.hive.ql.parse.ParseException: line 4:41 cannot recognize input near 'in' '(' ''Alex'' in expression specification
              at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
              at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:97)
              at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:89)
              at org.apache.hadoop.hive.ql.Compiler.parse(Compiler.java:172)
              at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:105)
              at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
              at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
              at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
              at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
              at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
              at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
              at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
              at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
              at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
      Caused by: NoViableAltException(173@[()* loopback of 739:5: (equal= precedenceEqualOperator p= precedenceSimilarExpression -> ^( $equal $p) |dist= precedenceDistinctOperator p= precedenceSimilarExpression -> ^( KW_NOT["not"] ^( EQUAL_NS["<=>"] $p) ) )*])
              at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
              at org.antlr.runtime.DFA.predict(DFA.java:116)
              at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceEqualExpression(HiveParser_IdentifiersParser.java:11812)
              at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnarySuffixExpression(HiveParser_IdentifiersParser.java:12370)
              at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceNotExpression(HiveParser_IdentifiersParser.java:12560)
              at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceAndExpression(HiveParser_IdentifiersParser.java:12669)
              at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceOrExpression(HiveParser_IdentifiersParser.java:12810)
              at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.expression(HiveParser_IdentifiersParser.java:8566)
              at org.apache.hadoop.hive.ql.parse.HiveParser.expression(HiveParser.java:45010)
              at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.searchCondition(HiveParser_FromClauseParser.java:6622)
              at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.whereClause(HiveParser_FromClauseParser.java:6540)
              at org.apache.hadoop.hive.ql.parse.HiveParser.whereClause(HiveParser.java:45058)
              at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:38576)
              at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:38849)
              at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:38442)
              at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:37704)
              at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:37592)
              at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2763)
              at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1640)
              at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:123)
              ... 59 more
      

      End-users still get the same high-level error message and developers can see exactly at which point the parser crashed as well as which rule might be worth investigating.

      Attachments

        Issue Links

          Activity

            People

              zabetak Stamatis Zampetakis
              zabetak Stamatis Zampetakis
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: