[HDFS-12822] HDFS unit test failure in AArch64. TestDirectoryScanner.testThrottling: Throttle is too permissive

• Type: Test
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: test
• Labels: dtest easyfix maven test

Environment

ARMv8 AArch64, Ubuntu16.04

• Flags: Important
• Assignee: Unassigned
• Reporter: Guangming Zhang
• Created: 16/Nov/17 03:23
• Updated: 17/Nov/17 05:26

Description

Description: Hi, When I ran the HDFS unit test and got a failure in TestDirectoryScanner.java test case :
TestDirectoryScanner.testThrottling:624 Throttle is too permissive
detail:
Running org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 227.046 sec <<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner) Time elapsed: 198.014 sec <<< FAILURE!
java.lang.AssertionError: Throttle is too permissive
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)

And below is the failure part of source code TestDirectoryScanner.java:

      ...........
      while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
        scanner = new DirectoryScanner(dataNode, fds, conf);
        ratio = runThrottleTest(blocks);
        retries -= 1;
      }

      // Waiting should be about 9x running.
      LOG.info("RATIO: " + ratio);
      assertTrue("Throttle is too restrictive", ratio <= 10f);
      assertTrue("Throttle is too permissive", ratio >= 7f);
    ............
    private float runThrottleTest(int blocks) throws IOException {
      scanner.setRetainDiffs(true);
      scan(blocks, 0, 0, 0, 0, 0);
      scanner.shutdown();
      assertFalse(scanner.getRunStatus());
      return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
    }
   .............

The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the code thrown out an assertTrue failure.
My questions are:
1. Why the ratio was set between 7f and 10f, is it a empirical value?
2. The ratio is smaller than 7f in AArch64 platform, is this value within normal range?

Could anyone help? Thanks a lot.