Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4059 Pig on Spark
  3. PIG-4456

Sort the leaves by SparkOperator.operatorKey in SparkLauncher#sparkOperToRDD

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: spark-branch
    • Fix Version/s: spark-branch
    • Component/s: spark
    • Labels:
      None

      Description

      Sort the leaves by SparkOperator.operatorKey to guarantee that sparkOperator with smaller operatorKey is executed before those with larger operatorKey if both are the leaves of spark plan, for example, sparkOperator(scope-65) and sparkOperator(scope-67) are all leaves of spark plan. SparkOperator(scope-65) should be executed before sparkOperator(scope-67) because the output of scope-65 maybe is used by scope-67.

        Attachments

        1. PIG-4456_1.patch
          1.0 kB
          liyunzhang
        2. PIG-4456.patch
          1 kB
          liyunzhang

          Issue Links

            Activity

              People

              • Assignee:
                kellyzly liyunzhang
                Reporter:
                kellyzly liyunzhang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: