Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-1813

Pig 0.8 throws ERROR 1075 while trying to refer a map in the result of eval udf.Works with 0.7

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      register myudf.jar;
      A = load 'input' MyZippedStorage('\u0001') as ($inputSchema);
      B = foreach A generate id , value ;
      C = foreach B generate id , org.myudf.ExplodeHashList( (chararray)value, '\u0002', '\u0004', '\u0003') as value;
      D = FILTER C by value is not null;
      E = foreach D generate id , flatten(org.myudf.GETFIRST(value)) as hop;
      F = foreach E generate id , hop#'rmli' as rmli:bytearray ;
      store F into 'output.bz2' using PigStorage();

      The above script fails when run with Pig 0.8 but runs fine with Pig 0.7 or if pig.usenewlogicalplan=false.
      The below is the exception thrown in 0.8 :

      org.apache.pig.backend.executionengine.ExecException: ERROR 1075: Received a bytearray from the UDF. Cannot determine how to convert the bytearray to map.
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:952)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.processInput(POMapLookUp.java:87)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:98)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:117)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:346)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1062)
      at org.apache.hadoop.mapred.Child.main(Child.java:211)

        Attachments

        1. PIG-1813-0.patch
          4 kB
          Jianyong Dai
        2. PIG-1813-1.patch
          8 kB
          Jianyong Dai
        3. PIG-1813-2.patch
          9 kB
          Jianyong Dai
        4. PIG-1813-3.patch
          5 kB
          Jianyong Dai

          Activity

            People

            • Assignee:
              daijy Jianyong Dai
              Reporter:
              vivekp Vivek Padmanabhan
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: