[MAPREDUCE-7311] Fix non-idempotent test in TestTaskProgressReporter

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.2, 3.2.4
• Fix Version/s: 3.4.0, 3.3.2, 3.2.4
• Component/s: test
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.2, 3.2.4
• Hadoop Flags: Reviewed
• Assignee: Zhengxi Li
• Reporter: Zhengxi Li
• Created: 29/Nov/20 23:36
• Updated: 12/Feb/24 00:00
• Resolved: 01/Sep/21 08:31

Description

The test `org.apache.hadoop.mapred.TestTaskProgressReporter.testBytesWrittenRespectingLimit` is not idempotent and fails if run twice in the same JVM, because it pollutes state shared among tests. It may be good to clean this state pollution so that some other tests do not fail in the future due to the shared state polluted by this test.

Details

Running `TestTaskProgressReporter.testBytesWrittenRespectingLimit` twice would result in the second run failing with the following assertion:

Assert.assertEquals(failFast, threadExited)

The root cause for this is that when`testBytesWrittenRespectingLimit` writes some bytes on the local file system, some counters are being incremented. The problem is that, after the test is done, the counter is not reset. With this polluted shared state, assumptions are broken, resulting in test failure in the second run.

PR link: https://github.com/apache/hadoop/pull/2500

Issue Links

• links to: GitHub Pull Request #2500