Pig
  1. Pig
  2. PIG-2351

Casting Exception in Illustrate for POForEach

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.9.0, 0.10.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      When running illustrate on this script:

       
      S1 = LOAD 'excite.log.bz2' AS (user_id:chararray, timestamp:chararray, query:chararray);
      S2 = LOAD 'excite.log.bz2' AS (user_id:chararray, timestamp:chararray, query:chararray);
      
      C = cogroup S1 BY user_id INNER, S2 by user_id INNER;
      
      D = foreach C 
          generate group,
                   flatten(S1.timestamp) as t1, 
                   flatten(S2.timestamp) as t2; 
      STORE D INTO 'output';
      

      I get this exception:

      2011-11-03 20:49:13,577 [main] ERROR org.apache.pig.PigServer - java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.Tuple
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.illustratorMarkup2(POForEach.java:683)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.createTuple(POForEach.java:458)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:404)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POOptimizedForEach.getNext(POOptimizedForEach.java:124)
      	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POJoinPackage.getNext(POJoinPackage.java:222)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:416)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:399)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:1)
      	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
      	at org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:215)
      	...
      

      The casting error itself is easy to fix (by checking for type) - but that results in the tracked lineages missing tuples. The method POForEach.illustratorMarkup2 seems to be based on the incorrect assumption that Object[] is an array of tuples instead of an array of fields for the output tuple and its not clear to me if this function needs to be modified or if there's a bigger problem somewhere else in the code.

        Activity

        Jeremy Karn made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Duplicate [ 3 ]
        Jeremy Karn made changes -
        Field Original Value New Value
        Attachment PIG-2351-test.patch [ 12502259 ]
        Jeremy Karn created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Jeremy Karn
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development