Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7089

ReadMapper.doIO hangs with user misconfigured inputs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.5.0
    • Fix Version/s: None
    • Component/s: client, test
    • Labels:
      None

      Description

      When a user configures the bufferSize to be 0, the while loop in TestDFSIO$ReadMapper.doIO function hangs endlessly.

      This is because the loop stride, curSize is always 0, making the loop index actualSize always less than the upper bound totalSize.

      Here is the code snippet.

      int bufferSize = DEFAULT_BUFFER_SIZE;
      
      for(int i = 0; i < args.length; i++) { // parse command line
        ...
        else if (args[i].equals("-bufferSize")) {
        bufferSize = Integer.parseInt(args[++i]);
        }
        ...
      }
      
      public Long doIO(Reporter reporter, String name, long totalSize) throws IOException {
        InputStream in = (InputStream)this.stream;
        long actualSize = 0;
        while (actualSize < totalSize) {
          int curSize = in.read(buffer, 0, bufferSize);
          if(curSize < 0) break;
          actualSize += curSize;
          reporter.setStatus("reading " + name + "@" + actualSize + "/" + totalSize + " ::host = " + hostName);
        }
        return Long.valueOf(actualSize);
      }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dustinday John Doe
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: