Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2077

Project UDF output inside a non-foreach statement fail on 0.8

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.8.1
    • 0.8.1
    • impl
    • None
    • Reviewed

    Description

      The following script fail on 0.8:

      A = load '1.txt' as (tracking_id, day:chararray);
      B = load '2.txt' as (tracking_id, timestamp:chararray);
      C = JOIN A by (tracking_id, day) LEFT OUTER, B by (tracking_id,  STRSPLIT(timestamp, ' ').$0);
      explain C;
      

      Error stack:
      Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
      at java.util.ArrayList.get(ArrayList.java:324)
      at org.apache.pig.newplan.logical.expression.ProjectExpression.findReferent(ProjectExpression.java:207)
      at org.apache.pig.newplan.logical.expression.ProjectExpression.getFieldSchema(ProjectExpression.java:121)
      at org.apache.pig.newplan.logical.optimizer.FieldSchemaResetter.execute(SchemaResetter.java:193)
      at org.apache.pig.newplan.logical.expression.AllSameExpressionVisitor.visit(AllSameExpressionVisitor.java:53)
      at org.apache.pig.newplan.logical.expression.ProjectExpression.accept(ProjectExpression.java:75)
      at org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
      at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
      at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:83)
      at org.apache.pig.newplan.logical.relational.LOJoin.accept(LOJoin.java:149)
      at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
      at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
      at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:262)

      This is not a problem on 0.9, trunk, since LogicalExpPlanMigrationVistor is dropped in 0.9.

      Attachments

        1. PIG-2077-1.patch
          2 kB
          Daniel Dai

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            daijy Daniel Dai
            daijy Daniel Dai
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment