[TEZ-3924] TestDefaultSorter fails intermittently due random keys and interaction with RLE and partition collisions

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.2, 0.10.0
• Target Version/s: 0.9.2
• Assignee: Jonathan Turner Eagles
• Reporter: Jonathan Turner Eagles
• Created: 30/Apr/18 18:27
• Updated: 19/Apr/19 19:21
• Resolved: 04/May/18 20:58

Description

The random keys chosen have a probability that they will fall into the same partition and therefore will invalidate the test assumption of how many empty partitions there are.

Example of failure stack trace

Failed
org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengths

Failing for the past 1 build (Since Failed#671 )
Took 67 ms.
(no description)
Error Message
Number of empty partitions did not match!
Stacktrace
java.lang.AssertionError: Number of empty partitions did not match!
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengthsHelper(TestDefaultSorter.java:321)
	at org.apache.tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter.testEmptyCaseFileLengths(TestDefaultSorter.java:288)