XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • tez
    • None

    Description

      Currently, TezJob extends ControlledJob, and TezLauncher submits TezJob to the cluster via ControlJob. However, there is no reason for this since ControlJob always contains a single job in Tez unlike MR.

      There are some operators such as MergeJoin that consists of more than 1 disjoint DAGs. But this case is handled by TezPlanContainer. TezPlanContainer holds multiple DAGs, and TezLauncher compiles and submits them one by one in order.

      I propose we should rewrite TezJob and TezLauncher using ExecutorService.

      Attachments

        1. PIG-4060-4.patch
          48 kB
          Cheolsoo Park
        2. PIG-4060-3.patch
          47 kB
          Cheolsoo Park
        3. PIG-4060-2.patch
          46 kB
          Cheolsoo Park
        4. PIG-4060-1.patch
          46 kB
          Cheolsoo Park

        Activity

          People

            cheolsoo Cheolsoo Park
            cheolsoo Cheolsoo Park
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: