Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2077

Project UDF output inside a non-foreach statement fail on 0.8

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.1
    • Fix Version/s: 0.8.1
    • Component/s: impl
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The following script fail on 0.8:

      A = load '1.txt' as (tracking_id, day:chararray);
      B = load '2.txt' as (tracking_id, timestamp:chararray);
      C = JOIN A by (tracking_id, day) LEFT OUTER, B by (tracking_id,  STRSPLIT(timestamp, ' ').$0);
      explain C;
      

      Error stack:
      Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
      at java.util.ArrayList.get(ArrayList.java:324)
      at org.apache.pig.newplan.logical.expression.ProjectExpression.findReferent(ProjectExpression.java:207)
      at org.apache.pig.newplan.logical.expression.ProjectExpression.getFieldSchema(ProjectExpression.java:121)
      at org.apache.pig.newplan.logical.optimizer.FieldSchemaResetter.execute(SchemaResetter.java:193)
      at org.apache.pig.newplan.logical.expression.AllSameExpressionVisitor.visit(AllSameExpressionVisitor.java:53)
      at org.apache.pig.newplan.logical.expression.ProjectExpression.accept(ProjectExpression.java:75)
      at org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
      at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
      at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:83)
      at org.apache.pig.newplan.logical.relational.LOJoin.accept(LOJoin.java:149)
      at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
      at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
      at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:262)

      This is not a problem on 0.9, trunk, since LogicalExpPlanMigrationVistor is dropped in 0.9.

        Attachments

        1. PIG-2077-1.patch
          2 kB
          Jianyong Dai

          Activity

            People

            • Assignee:
              daijy Jianyong Dai
              Reporter:
              daijy Jianyong Dai
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: