[HDFS-5003] TestNNThroughputBenchmark failed caused by existing directories

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1-win, 2.1.0-beta, 1.3.0, 3.0.0-alpha1
• Fix Version/s: 1-win, 2.1.0-beta, 1.3.0
• Component/s: test
• Target Version/s: 1-win, 2.1.0-beta, 1.3.0
• Hadoop Flags: Reviewed
• Assignee: Douma Fang Assign to me
• Reporter: Douma Fang
• Created: 16/Jul/13 22:57
• Updated: 12/May/16 18:16
• Resolved: 18/Jul/13 04:01

Description

This test failed on both Windows and Linux.
Here is the error information.

Testcase: testNNThroughput took 36.221 sec
Caused an ERROR
NNThroughputBenchmark: cannot mkdir D:\condor\condor\build\test\dfs\hosts\exclude
java.io.IOException: NNThroughputBenchmark: cannot mkdir D:\condor\condor\build\test\dfs\hosts\exclude
at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.<init>(NNThroughputBenchmark.java:111)
at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.runBenchmark(NNThroughputBenchmark.java:1168)
at org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark.testNNThroughput(TestNNThroughputBenchmark.java:38)

This test may not fail for the first run, but will fail for the second one.

The root cause is in the constructor of NNThroughputBenchmark

NNThroughputBenchmark(Configuration conf) throws IOException, LoginException  {
...
     config.set("dfs.hosts.exclude", "${hadoop.tmp.dir}/dfs/hosts/exclude");
     File excludeFile = new File(config.get("dfs.hosts.exclude", "exclude"));
     if(! excludeFile.exists()) {
      if(!excludeFile.getParentFile().mkdirs())
         throw new IOException("NNThroughputBenchmark: cannot mkdir " + excludeFile);
     }
     new FileOutputStream(excludeFile).close();

excludeFile.getParentFile() may already exist, then excludeFile.getParentFile().mkdirs() will return false, which however is not an expected behavior.