Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13514

BenchmarkThroughput.readLocalFile hangs with misconfigured BUFFER_SIZE

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.5.0
    • Fix Version/s: None
    • Component/s: test
    • Labels:
      None

      Description

      When the BUFFER_SIZE is configured to be 0, the while loop in BenchmarkThroughput.readLocalFile function hangs endlessly.
      This is because when the data.size (i.e., BUFFER_SIZE) is 0, the val will always be 0 by invoking val=in.read(data).
      Here is the code snippet.

        BUFFER_SIZE = conf.getInt("dfsthroughput.buffer.size", 4 * 1024);//when dfsthroughput.buffer.size is configued to be 0
      
        private void readLocalFile(Path path, String name, Configuration conf) throws IOException {
          System.out.print("Reading " + name);
          resetMeasurements();
          InputStream in = new FileInputStream(new File(path.toString()));
          byte[] data = new byte[BUFFER_SIZE];
          long size = 0;
          while (size >= 0) {
            size = in.read(data);
          }
          in.close();
          printMeasurements();
        }
      

      The similar case is HDFS-13513

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dustinday John Doe
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: