[PIG-3780] Tez mini cluster tests run for a very long time with TezSession reuse on

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: tez-branch
• Fix Version/s: tez-branch
• Component/s: tez
• Assignee: Cheolsoo Park
• Reporter: Cheolsoo Park
• Created: 26/Feb/14 00:36
• Updated: 21/Nov/14 05:59
• Resolved: 26/Feb/14 23:58

Description

In the current tez branch, mini cluster unit tests are very slow. The reason is as follows:

• TezSession reuse is by default on.
• Each test case runs, and it waits for Tez AM to terminate.
• After Tez AM times out (usually after several minutes), another test case runs.

Two questions that I have are:

1. Why doesn't TezSession reuse work in mini cluster?
2. Why is TezSession reuse not disabled in some tests (e.g. TestAccumulator) where we explicitly set "tez.session.reuse" to false?

As for #2, I realized that "tez.session.reuse" was never set in the properties object that is passed to PigServer. I am going to upload a patch that fixes this problem in this jira.

As for #1, I don't have an answer yet. But I think we can fix this in a separate jira once we get Tez unit tests working again.