Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5279

Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how previous serialization mechanism solved this but, kryo complaints that it's not Serializable and fails the query.

      The log below is the example,

      java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class cannot be created (missing no-arg constructor): org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
      Serialization trace:
      inputOI (org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
      genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
      aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
      conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
      childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
      childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
      aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
      	at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
      	at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
      	at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
      	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
      	at org.apache.h
      

      If this cannot be fixed in somehow, some UDAFs should be modified to be run on hive-0.13.0

        Attachments

        1. D12963.5.patch
          28 kB
          Phabricator
        2. D12963.4.patch
          24 kB
          Phabricator
        3. D12963.3.patch
          12 kB
          Phabricator
        4. D12963.2.patch
          12 kB
          Phabricator
        5. D12963.1.patch
          10 kB
          Phabricator
        6. 5279.patch
          6 kB
          Ashutosh Chauhan

          Activity

            People

            • Assignee:
              navis Navis
              Reporter:
              navis Navis
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: