Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22963

HiveParser misinterpretes quotes in parameters of built-in functions or UDFs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.3.6, 3.1.1
    • Fix Version/s: None
    • Component/s: Parser
    • Labels:
      None

      Description

      Parsing of query fails when we use single or double quotes in from/to string of translate function in 2.3*/3.1.1 version of hive. Parsing of the same query is successful in 2.1.1 version of hive.

      Steps to reproduce:

       

      CREATE TABLE test_table (data string);
      INSERT INTO test_table VALUES("d\"a\"t\"a");
      select translate(data, '"', '') from test_table;
      

       

       

      Parsing fails with the following exception:

       NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=> partitionedTableFunction | tableSource | subQuerySource | virtualTableSource )])NoViableAltException(355@[157:5: ( ( Identifier LPAREN )=> partitionedTableFunction | tableSource | subQuerySource | virtualTableSource )]) at org.antlr.runtime.DFA.noViableAlt(DFA.java:158) at org.antlr.runtime.DFA.predict(DFA.java:116) at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource0(HiveParser_FromClauseParser.java:2942) at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:2880) at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1451) at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1341) at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45811) at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:39699) at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:39951) at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:39597) at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:38786) at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:38674) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2340) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1369) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:223) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)FAILED: ParseException line 1:40 cannot recognize input near 'tt' ';' '<EOF>' in from source 0org.apache.hadoop.hive.ql.parse.ParseException: line 1:40 cannot recognize input near 'tt' ';' '<EOF>' in from source 0 at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:211) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:507) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1388) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1528) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1308) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1298) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:276) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:465) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:992) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:916) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:795) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:223) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

       

      Parsing is successful when double quote is escaped. 

      The following query works:

       

      SELECT translate(data, '\"', ' ') from test_table;
      

       

      But the behaviour which looks buggy is when the double quote is not escaped in from/to string of translate function, it seems to be interpreting the double quote in from/to string of next translate function as ending quote. 

      Parsing is successful for the following queries.

       

      select translate(data, '"', ''), translate(data, '"', '') from test_table;
      select translate(data, '"', ''), translate(data, ' ', '"') from test_table;
      select translate(data, "'", ""), translate(data, "'", "") from test_table;
      select translate(data, "'", ""), translate(data, " ", "'") from test_table;
      select translate(data, '"', '"') from test_table;
      select translate(data, "'", "'") from test_table; 
      

       

      The same behaviour exists when we use any other built-in functions or UDFs.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ganeshas Ganesha Shreedhara
                Reporter:
                ganeshas Ganesha Shreedhara
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: