[HADOOP-15415] copyBytes hangs when the configuration file is corrupted

• Type: Bug
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.23.0, 2.5.0, 3.1.0
• Component/s: common
• Assignee: Yiqun Lin
• Reporter: John Doe
• Created: 26/Apr/18 18:21
• Updated: 22/May/18 09:36

Description

The third parameter,  buffSize, is read from the configuration files or user-specified.

When the configuration file is corrupted or the user configures with a wrong value, i.e., 0, the bytesRead will always be 0, making the while loop's condition always true, hanging IOUtils.copyBytes() endlessly.

Here is the snippet of the code. There are four copyBytes in the following code. The 3rd and 4th copyBytes calls the 1st one. The 1st one calls the 2nd one. Hang happens in the while loop of the second copyBytes function.

 

//1st copyBytes
  public static void copyBytes(InputStream in, OutputStream out, int buffSize, boolean close) 
    throws IOException {
    try {
      copyBytes(in, out, buffSize);
      if(close) {
        out.close();
        out = null;
        in.close();
        in = null;
      }
    } finally {
      if(close) {
        closeStream(out);
        closeStream(in);
      }
    }
  }
  
//2nd copyBytes
  public static void copyBytes(InputStream in, OutputStream out, int buffSize) 
    throws IOException {
    PrintStream ps = out instanceof PrintStream ? (PrintStream)out : null;
    byte buf[] = new byte[buffSize];
    int bytesRead = in.read(buf);
    while (bytesRead >= 0) {
      out.write(buf, 0, bytesRead);
      if ((ps != null) && ps.checkError()) {
        throw new IOException("Unable to write to output stream.");
      }
      bytesRead = in.read(buf);
    }
  }

//3rd copyBytes
  public static void copyBytes(InputStream in, OutputStream out, Configuration conf)
    throws IOException {
    copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096), true);
  }
  
//4th copyBytes
  public static void copyBytes(InputStream in, OutputStream out, Configuration conf, boolean close)
    throws IOException {
    copyBytes(in, out, conf.getInt("io.file.buffer.size", 4096),  close);
  }