[PIG-5245] TestGrunt.testStopOnFailure is flaky

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.17.0
• Hadoop Flags: Reviewed
• Assignee: Rohini Palaniswamy
• Reporter: Rohini Palaniswamy
• Created: 31/May/17 07:21
• Updated: 21/Jun/17 09:15
• Resolved: 31/May/17 20:30

Description

The test is supposed to run two tests in parallel, and one when fails other should be killed when stop on failure is configured. But the test is actually running only job at a time and based on the order in which jobs are run it is passing. This is because of the capacity scheduler configuration of the MiniCluster. It runs only one AM at a time due to resource restrictions. In a 16G node, when the first job runs it takes up 1536 (AM) + 1024 (task) memory.mb. Since only 10% of cluster resource is the default for running AMs and a single AM already takes up memory close to 1.6G, second job AM is not launched.