[HDFS-14501] BenchmarkThroughput.writeFile hangs with misconfigured BUFFER_SIZE

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.5.0
• Assignee: Hemanth Boyina
• Reporter: John Doe
• Created: 17/May/19 19:44
• Updated: 26/Aug/19 17:16

Description

When the configuration file is corrupted, reading BUFFER_SIZE from corrupted conf can return 0.
The "for" loop in BenchmarkThroughput.writeLocalFile function hangs endlessly.
Here is the code snippet.

  BUFFER_SIZE = conf.getInt("dfsthroughput.buffer.size", 4 * 1024);
  private Path writeFile(FileSystem fs,
                                String name,
                                Configuration conf,
                                long total
                                ) throws IOException {
    Path f = dir.getLocalPathForWrite(name, total, conf);
    System.out.print("Writing " + name);
    resetMeasurements();
    OutputStream out = fs.create(f);
    byte[] data = new byte[BUFFER_SIZE];
    for(long size = 0; size < total; size += BUFFER_SIZE) { //Bug!
      out.write(data);
    }
    out.close();
    printMeasurements();
    return f;
  }

This configuration error also affects HDFS-13513, HDFS-13514,