[HDFS-14481] BenchmarkThroughput.writeLocalFile hangs with misconfigured BUFFER_SIZE

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.5.0
• Component/s: test
• Assignee: Unassigned
• Reporter: John Doe
• Created: 08/May/19 18:00
• Updated: 13/May/19 06:47

Description

When the configuration file is corrupted, reading BUFFER_SIZE from corrupted conf can return 0.
The "for" loop in BenchmarkThroughput.writeLocalFile function hangs endlessly.
Here is the code snippet.

  BUFFER_SIZE = conf.getInt("dfsthroughput.buffer.size", 4 * 1024);

  private Path writeLocalFile(String name, Configuration conf,
                                     long total) throws IOException {
    Path path = dir.getLocalPathForWrite(name, total, conf);
    System.out.print("Writing " + name);
    resetMeasurements();
    OutputStream out = new FileOutputStream(new File(path.toString()));
    byte[] data = new byte[BUFFER_SIZE];
    for(long size=0; size < total; size += BUFFER_SIZE) {//Bug!!!
      System.out.println("inside for loop...size = " + size);
      out.write(data);
    }
    out.close();
    printMeasurements();
    return path;
  }

This configuration error also affects HDFS-13513, HDFS-13514