1. Pig
  2. PIG-3446 Umbrella jira for Pig on Tez
  3. PIG-3780

Tez mini cluster tests run for a very long time with TezSession reuse on


    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: tez-branch
    • Fix Version/s: tez-branch
    • Component/s: tez
    • Labels:


      In the current tez branch, mini cluster unit tests are very slow. The reason is as follows:

      • TezSession reuse is by default on.
      • Each test case runs, and it waits for Tez AM to terminate.
      • After Tez AM times out (usually after several minutes), another test case runs.

      Two questions that I have are:

      1. Why doesn't TezSession reuse work in mini cluster?
      2. Why is TezSession reuse not disabled in some tests (e.g. TestAccumulator) where we explicitly set "tez.session.reuse" to false?

      As for #2, I realized that "tez.session.reuse" was never set in the properties object that is passed to PigServer. I am going to upload a patch that fixes this problem in this jira.

      As for #1, I don't have an answer yet. But I think we can fix this in a separate jira once we get Tez unit tests working again.

      1. PIG-3780-2.patch
        34 kB
        Cheolsoo Park
      2. PIG-3780-1.patch
        61 kB
        Cheolsoo Park


        Cheolsoo Park created issue -
        Cheolsoo Park made changes -
        Field Original Value New Value
        Attachment PIG-3780-1.patch [ 12631132 ]
        Cheolsoo Park made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Cheolsoo Park made changes -
        Attachment PIG-3780-2.patch [ 12631354 ]
        Cheolsoo Park made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]


          • Assignee:
            Cheolsoo Park
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            2 Start watching this issue


            • Created: