[PIG-4644] PORelationToExprProject.clone() is broken

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.16.0, 0.15.1
• Hadoop Flags: Reviewed
• Assignee: Anthony Hsu
• Reporter: Ratandeep Ratti
• Created: 27/Jul/15 16:48
• Updated: 08/Jun/16 20:48
• Resolved: 21/Sep/15 23:06

Description

We are receiving the following exception when using Pig

Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject cannot be cast to org.apache.pig.backend.hadoop.executionen\
gine.physicalLayer.expressionOperators.PORelationToExprProject
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.PORelationToExprProject.clone(PORelationToExprProject.java:144)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.PORelationToExprProject.clone(PORelationToExprProject.java:50)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.clone(PhysicalPlan.java:227)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.clone(POForEach.java:639)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.clone(POForEach.java:53)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.clone(PhysicalPlan.java:227)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.mergeDiamondMROper(MultiQueryOptimizer.java:298)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.visitMROp(MultiQueryOptimizer.java:219)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:273)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:46)
        at org.apache.pig.impl.plan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:71)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.visit(MultiQueryOptimizer.java:94)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:629)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:148)
        at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)

On further investigation it seems that POProject's clone method is implemented as

    @Override
    public POProject clone() throws CloneNotSupportedException {
        ArrayList<Integer> cols = new ArrayList<Integer>(columns.size());
        // Can resuse the same Integer objects, as they are immutable
        for (Integer i : columns) {
            cols.add(i);
        }
        POProject clone = new POProject(new OperatorKey(mKey.scope,
            NodeIdGenerator.getGenerator().getNextNodeId(mKey.scope)),
            requestedParallelism, cols);
        clone.cloneHelper(this);
        clone.overloaded = overloaded;
        clone.startCol = startCol;
        clone.isProjectToEnd = isProjectToEnd;
        clone.resultType = resultType;
        return clone;
    }

It uses a constructor to clone POProject (which break the weak rule of object cloning)
In the subclass , PORelationToExprProject implements cloneable as

@Override
    public PORelationToExprProject clone() throws CloneNotSupportedException {
        return (PORelationToExprProject) super.clone();
    }

As seen from the POProject's implementation of cloneable, super.clone will never be of type PORelationToExprProject,