Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-1895

Class cast exception while projecting udf result

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0, 0.8.0, 0.9.0
    • Fix Version/s: 0.8.1
    • Component/s: impl
    • Labels:
      None

      Description

      Class cast exception is thrown when I try to project the result from my udf. The udf has a defined schema DataType.BAG,DataType.LONG and DataType.INTEGER

      The below is my script

      Data = load 'file:/home/pvivek/Desktop/input' using PigStorage() as ( i: int );
      AllData = group Data all parallel 1;
      SampledData = foreach AllData generate org.vivek.TestEvalFunc(Data, 5) as rs;
      SampledData1 = foreach SampledData generate rs.sampled;
      

      Even though the output schema defines "sampled" as a data bag, while processing, instead of sending only the data bag generated from the UDF , the entire tuple was sent to the projection as result.

      Exception recieved :
      java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.pig.data.DataBag
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.processInputBag(POProject.java:484)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:197)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.processInputBag(POProject.java:480)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:197)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:339)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:1)
      	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
      	at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
      	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
      	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
      

      This issue is happening with 0.9/0.8 and 0.7

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                daijy Jianyong Dai
                Reporter:
                vivekp Vivek Padmanabhan
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: