[PIG-4456] Sort the leaves by SparkOperator.operatorKey in SparkLauncher#sparkOperToRDD

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: spark-branch
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: liyunzhang
• Reporter: liyunzhang
• Created: 10/Mar/15 03:40
• Updated: 21/Jun/17 09:18
• Resolved: 10/Mar/15 22:19

Description

Sort the leaves by SparkOperator.operatorKey to guarantee that sparkOperator with smaller operatorKey is executed before those with larger operatorKey if both are the leaves of spark plan, for example, sparkOperator(scope-65) and sparkOperator(scope-67) are all leaves of spark plan. SparkOperator(scope-65) should be executed before sparkOperator(scope-67) because the output of scope-65 maybe is used by scope-67.

Issue Links

• links to: review board