Pig
  1. Pig
  2. PIG-1073

LogicalPlanCloner can't clone plan containing LOJoin

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: impl
    • Labels:
      None
    • Tags:
      logical layer

      Description

      Add following testcase in LogicalPlanBuilder.java

      public void testLogicalPlanCloner() throws CloneNotSupportedException

      { LogicalPlan lp = buildPlan("C = join ( load 'A') by $0, (load 'B') by $0;"); LogicalPlanCloner cloner = new LogicalPlanCloner(lp); cloner.getClonedPlan(); }

      and this fails with the following stacktrace:

      java.lang.NullPointerException
      at org.apache.pig.impl.logicalLayer.LOVisitor.visit(LOVisitor.java:171)
      at org.apache.pig.impl.logicalLayer.PlanSetter.visit(PlanSetter.java:63)
      at org.apache.pig.impl.logicalLayer.LOJoin.visit(LOJoin.java:213)
      at org.apache.pig.impl.logicalLayer.LOJoin.visit(LOJoin.java:45)
      at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67)
      at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
      at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50)
      at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
      at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.getClonedPlan(LogicalPlanCloneHelper.java:73)
      at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:46)
      at org.apache.pig.test.TestLogicalPlanBuilder.testLogicalPlanCloneHelper(TestLogicalPlanBuilder.java:2110)

      1. pig-1073-1.patch
        29 kB
        Ashutosh Chauhan

        Activity

        Hide
        Ashutosh Chauhan added a comment -

        It seems that fix is to override the visit method in LogicalPlanCloneHelper.java

        @Override
        protected void visit(LOJoin loJoin) throws VisitorException

        { .. }
        Show
        Ashutosh Chauhan added a comment - It seems that fix is to override the visit method in LogicalPlanCloneHelper.java @Override protected void visit(LOJoin loJoin) throws VisitorException { .. }
        Hide
        Santhosh Srinivasan added a comment -

        If my memory serves me correctly, the logical plan cloning was implemented (by me) for cloning inner plans for foreach. As such, the top level plan cloning was never tested and some items are marked as TODO (see visit methods for LOLoad, LOStore and LOStream).

        If you want to use it as you mention in your test cases, then you need to add code for cloning the LOLoad, LOStore, LOStream and LOJoin operators.

        Show
        Santhosh Srinivasan added a comment - If my memory serves me correctly, the logical plan cloning was implemented (by me) for cloning inner plans for foreach. As such, the top level plan cloning was never tested and some items are marked as TODO (see visit methods for LOLoad, LOStore and LOStream). If you want to use it as you mention in your test cases, then you need to add code for cloning the LOLoad, LOStore, LOStream and LOJoin operators.
        Hide
        Ashutosh Chauhan added a comment -

        Draft patch with testcase.

        Show
        Ashutosh Chauhan added a comment - Draft patch with testcase.
        Hide
        Ashutosh Chauhan added a comment -

        Current patch only partially fixes the problem. It seems we have bigger problems in a way visiting is done on query plans currently. I am working on fixing those.

        Show
        Ashutosh Chauhan added a comment - Current patch only partially fixes the problem. It seems we have bigger problems in a way visiting is done on query plans currently. I am working on fixing those.
        Hide
        Ashutosh Chauhan added a comment -

        updated patch

        Show
        Ashutosh Chauhan added a comment - updated patch

          People

          • Assignee:
            Ashutosh Chauhan
            Reporter:
            Ashutosh Chauhan
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development