[TEZ-800] One-one edge with parallelism -1 fails if source vertex parallelism is not -1 as well

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.5.0
• Hadoop Flags: Reviewed
• Assignee: Bikas Saha
• Reporter: Rohini Palaniswamy
• Created: 06/Feb/14 07:08
• Updated: 28/Jan/18 08:15
• Resolved: 28/May/14 21:37

Description

// no input initializers. At this moment, only other case is 1-1 edge
// with uninitialized sources
boolean hasOneToOneUninitedSource = false;
for (Map.Entry<Vertex, Edge> entry : vertex.sourceVertices.entrySet()) {
if (entry.getValue().getEdgeProperty().getDataMovementType() ==
DataMovementType.ONE_TO_ONE) {
if (entry.getKey().getTotalTasks() == -1)

{ hasOneToOneUninitedSource = true; break; }

}
}

This checks for the source vertex which has the 1-1 edge to also have -1 parallelism.

Issue Links

• blocks: PIG-3846 Implement automatic reducer parallelism
• is required by: TEZ-388 Pig on Tez