Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14481

BenchmarkThroughput.writeLocalFile hangs with misconfigured BUFFER_SIZE

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.5.0
    • Fix Version/s: None
    • Component/s: test
    • Labels:
      None

      Description

      When the configuration file is corrupted, reading BUFFER_SIZE from corrupted conf can return 0.
      TheĀ "for" loop in BenchmarkThroughput.writeLocalFile function hangs endlessly.
      Here is the code snippet.

        BUFFER_SIZE = conf.getInt("dfsthroughput.buffer.size", 4 * 1024);
      
        private Path writeLocalFile(String name, Configuration conf,
                                           long total) throws IOException {
          Path path = dir.getLocalPathForWrite(name, total, conf);
          System.out.print("Writing " + name);
          resetMeasurements();
          OutputStream out = new FileOutputStream(new File(path.toString()));
          byte[] data = new byte[BUFFER_SIZE];
          for(long size=0; size < total; size += BUFFER_SIZE) {//Bug!!!
            System.out.println("inside for loop...size = " + size);
            out.write(data);
          }
          out.close();
          printMeasurements();
          return path;
        }
      

      This configuration error also affects HDFS-13513, HDFS-13514

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dustinday John Doe
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: