[HDFS-16818] RBF TestRouterRPCMultipleDestinationMountTableResolver non-deterministic unit tests failures

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: rbf
• Target Version/s: 3.5.0
• Assignee: Unassigned
• Reporter: Xing Lin
• Created: 25/Oct/22 04:38
• Updated: 04/Jan/24 08:04

Description

TestRouterRPCMultipleDestinationMountTableResolver fails a couple of times nondeterministically when run multiple times. 

I repeated the following commands for 10+ times against 454157a3844cdd6c92ef650af6c3b323cbec88af in trunk and observed two types of failed runs.

mvn test -Dtest="TestRouterRPCMultipleDestinationMountTableResolver"

 

Failed run 1 output:

[ERROR] Failures:
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationHashAllOrder:177->testInvocation:221->testDirec
toryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:395 expected:<[COLD]> but was:<[HOT]>
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationHashOrder:193->testInvocation:221->testDirector
yAndFileLevelInvocation:298->verifyDirectoryLevelInvocations:395 expected:<[COLD]> but was:<[HOT]>
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationLocalOrder:201->testInvocation:221->testDirecto
ryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:395 expected:<[COLD]> but was:<[HOT]>
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationRandomOrder:185->testInvocation:221->testDirect
oryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:395 expected:<[COLD]> but was:<[HOT]>
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationSpaceOrder:169->testInvocation:221->testDirecto
ryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:395 expected:<[COLD]> but was:<[HOT]>
[INFO]
[ERROR] Tests run: 18, Failures: 5, Errors: 0, Skipped: 0

 

Failed run 2 output:

[ERROR] Failures:
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testECMultipleDestinations:430
[ERROR] Errors:
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationHashAllOrder:177->testInvocation:221->testDirec
toryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:397 NullPointer
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationHashOrder:193->testInvocation:221->testDirector
yAndFileLevelInvocation:298->verifyDirectoryLevelInvocations:397 NullPointer
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationLocalOrder:201->testInvocation:221->testDirecto
ryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:397 NullPointer
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationRandomOrder:185->testInvocation:221->testDirect
oryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:397 NullPointer
[ERROR]   TestRouterRPCMultipleDestinationMountTableResolver.testInvocationSpaceOrder:169->testInvocation:221->testDirecto
ryAndFileLevelInvocation:296->verifyDirectoryLevelInvocations:397 NullPointer
[INFO]
[ERROR] Tests run: 18, Failures: 1, Errors: 5, Skipped: 0