Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5003

TestNNThroughputBenchmark failed caused by existing directories

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1-win, 2.1.0-beta, 1.3.0, 3.0.0-alpha1
    • Fix Version/s: 1-win, 2.1.0-beta, 1.3.0
    • Component/s: test
    • Labels:
      None

      Description

      This test failed on both Windows and Linux.
      Here is the error information.

      Testcase: testNNThroughput took 36.221 sec
      Caused an ERROR
      NNThroughputBenchmark: cannot mkdir D:\condor\condor\build\test\dfs\hosts\exclude
      java.io.IOException: NNThroughputBenchmark: cannot mkdir D:\condor\condor\build\test\dfs\hosts\exclude
      at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.<init>(NNThroughputBenchmark.java:111)
      at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.runBenchmark(NNThroughputBenchmark.java:1168)
      at org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark.testNNThroughput(TestNNThroughputBenchmark.java:38)

      This test may not fail for the first run, but will fail for the second one.

      The root cause is in the constructor of NNThroughputBenchmark

      NNThroughputBenchmark(Configuration conf) throws IOException, LoginException  {
      ...
           config.set("dfs.hosts.exclude", "${hadoop.tmp.dir}/dfs/hosts/exclude");
           File excludeFile = new File(config.get("dfs.hosts.exclude", "exclude"));
           if(! excludeFile.exists()) {
            if(!excludeFile.getParentFile().mkdirs())
               throw new IOException("NNThroughputBenchmark: cannot mkdir " + excludeFile);
           }
           new FileOutputStream(excludeFile).close();
      
      

      excludeFile.getParentFile() may already exist, then excludeFile.getParentFile().mkdirs() will return false, which however is not an expected behavior.

        Attachments

        1. HADOOP-9739.1.trunk.patch
          0.9 kB
          Douma Fang
        2. HADOOP-9739.1.patch
          0.8 kB
          Douma Fang

          Activity

            People

            • Assignee:
              xifang Douma Fang
              Reporter:
              xifang Douma Fang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: