[HADOOP-15425] CopyFilesMapper.doCopyFile hangs with misconfigured sizeBuf

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.5.0
• Component/s: tools
• Assignee: Unassigned
• Reporter: John Doe
• Created: 27/Apr/18 19:05
• Updated: 27/Apr/18 19:05

Description

When the sizeBuf is configured to be 0, the for loop in DistCpV1$CopyFilesMapper.doCopyFile function hangs endlessly.
This is because when the buf.size (i.e., sizeBuf) is 0, the bytesRead will always be 0 by invoking bytesRead=in.read(buf).
Here is the code snippet.

    sizeBuf = job.getInt("copy.buf.size", 128 * 1024); //when copy.buf.size = 0
    buffer = new byte[sizeBuf];

    private long doCopyFile(FileStatus srcstat, Path tmpfile, Path absdst, Reporter reporter) throws IOException {
      FSDataInputStream in = null;
      FSDataOutputStream out = null;
      long bytesCopied = 0L;
      try {
        Path srcPath = srcstat.getPath();
        // open src file
        in = srcPath.getFileSystem(job).open(srcPath);
        reporter.incrCounter(Counter.BYTESEXPECTED, srcstat.getLen());
        // open tmp file
        out = create(tmpfile, reporter, srcstat);
        LOG.info("Copying file " + srcPath + " of size " + srcstat.getLen() + " bytes...");
        
        // copy file
        for(int bytesRead; (bytesRead = in.read(buffer)) >= 0; ) {
          out.write(buffer, 0, bytesRead);
          bytesCopied += bytesRead;
          reporter.setStatus(... );
        }
      } finally {
        checkAndClose(in);
        checkAndClose(out);
      }
      return bytesCopied;
    }