Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-3495

For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.10.0
    • Component/s: Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters having ConstantObjectInspectors will be replaced by ExprNodeColumnDescs. A UDFArgumentTypeException will be thrown if a UDAF need to checkout parameters' types.

      An example used to reply the error is

      set hive.map.aggr=false;
      SELECT percentile_approx(cast(substr(src.value,5) AS double), 0.5) FROM src;
      

      .

      Here is the log

      2012-09-20 12:36:06,947 DEBUG exec.FunctionRegistry (FunctionRegistry.java:getGenericUDAFResolver(849)) - Looking up GenericUDAF: percentile_approx
      2012-09-20 12:36:06,952 ERROR ql.Driver (SessionState.java:printError(400)) - FAILED: UDFArgumentTypeException The second argument must be a constant, but double was passed instead.
      org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: The second argument must be a constant, but double was passed instead.
      	at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileApprox.getEvaluator(GenericUDAFPercentileApprox.java:149)
      	at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:774)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:2389)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanGroupByOperator(SemanticAnalyzer.java:2561)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlan1MR(SemanticAnalyzer.java:3341)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6140)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6903)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7484)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:245)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:903)
      	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
      	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
      	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:713)
      	at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_replay(TestCliDriver.java:125)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at junit.framework.TestCase.runTest(TestCase.java:168)
      	at junit.framework.TestCase.runBare(TestCase.java:134)
      	at junit.framework.TestResult$1.protect(TestResult.java:110)
      	at junit.framework.TestResult.runProtected(TestResult.java:128)
      	at junit.framework.TestResult.run(TestResult.java:113)
      	at junit.framework.TestCase.run(TestCase.java:124)
      	at junit.framework.TestSuite.runTest(TestSuite.java:232)
      	at junit.framework.TestSuite.run(TestSuite.java:227)
      	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520)
      	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
      	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
      

        Attachments

        1. HIVE-3495.4.patch.txt
          17 kB
          Yin Huai
        2. HIVE-3495.3.patch.txt
          17 kB
          Yin Huai
        3. HIVE-3495.2.patch.txt
          10 kB
          Yin Huai
        4. HIVE-3495.1.patch.txt
          6 kB
          Yin Huai

          Activity

            People

            • Assignee:
              yhuai Yin Huai
              Reporter:
              yhuai Yin Huai
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: