Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7088

DistributedFSCheckMapper.doIO hangs with user misconfigured inputs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.5.0
    • Fix Version/s: None
    • Component/s: client, test
    • Labels:
      None

      Description

      When a user configures the bufferSize to be 0, the for loop in DistributedFSCheck$DistributedFSCheckMapper.doIO function hangs endlessly. Here is the code snippet.

          int bufferSize = DEFAULT_BUFFER_SIZE;
      
          for(int i = 0; i < args.length; i++) { // parse command line
           ...
           else if (args[i].equals("-bufferSize")) {
              bufferSize = Integer.parseInt(args[++i]);
            }
           ...
          }
          public Object doIO(Reporter reporter, String name,  long offset) throws IOException {
            // open file
            FSDataInputStream in = null;
            Path p = new Path(name);
            try {
              in = fs.open(p);
            } catch(IOException e) {
              return name + "@(missing)";
            }
            in.seek(offset);
            long actualSize = 0;
            try {
              long blockSize = fs.getDefaultBlockSize(p);
              reporter.setStatus("reading " + name + "@" + offset + "/" + blockSize);
              for( int curSize = bufferSize; 
                   curSize == bufferSize && actualSize < blockSize;
                   actualSize += curSize) {
                curSize = in.read(buffer, 0, bufferSize);
              }
            } catch(IOException e) {
              ...
            } finally {
              in.close();
            }
            return new Long(actualSize);
          }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dustinday John Doe
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: