Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5333

LoadCaster sometimes not set for complex type

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TypeCasting a bytearray field in the nested foreach below

      A = load 'input.txt' as (a0:bytearray, a1:bytearray);
      B = group A by (a0,a1);
      C = FOREACH B GENERATE SIZE(A) as sizeofgroup, A;
      SPLIT C into D if sizeofgroup > 2, Z OTHERWISE;
      
      F = FOREACH D {
          F1 = FOREACH A generate (chararray) a0;
          GENERATE F1;
      }
      
      store F into '/tmp/output/F';
      store Z into '/tmp/output/Z';
      

      Fails with

      Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1075: Received a bytearray from the UDF or Union from two different Loaders. Cannot determine how to convert the bytearray to string for [a0[7,29]]
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNextString(POCast.java:1126)
      

        Attachments

        1. pig-5333-v01.patch
          11 kB
          Koji Noguchi

          Activity

            People

            • Assignee:
              knoguchi Koji Noguchi
              Reporter:
              knoguchi Koji Noguchi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: