Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3446 Umbrella jira for Pig on Tez
  3. PIG-3539

Pig should be able to submit multiple DAG

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: tez-branch
    • Component/s: tez
    • Labels:
      None

      Description

      There could be several scenario Pig want to submit multiple DAG:
      1. a blocking operator in grunt
      2. a blocking operator in script (exec, fs)
      3. break a DAG into two for possible inner DAG optimization

      Pig should use TezSession so that we can reuse AM. TezSession should be a thread local variable. For those who use Pig Java API, they will use the same AM within a thread, but different AM in another thread.

        Attachments

        1. PIG-3539-1.patch
          69 kB
          Daniel Dai

          Activity

            People

            • Assignee:
              daijy Daniel Dai
              Reporter:
              daijy Daniel Dai
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: