[HADOOP-15417] s3: retrieveBlock hangs when the configuration file is corrupted

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.23.0, 2.5.0
• Component/s: common, fs/s3
• Assignee: Unassigned
• Reporter: John Doe
• Created: 26/Apr/18 19:40
• Updated: 27/Apr/18 17:34
• Resolved: 27/Apr/18 10:06

Description

The bufferSize is read from the configuration files.

When the configuration file is corrupted, i.e.,bufferSize=0, the numRead will always be 0, making the while loop's condition always true, hanging Jets3tFileSystemStore.retrieveBlock() endlessly.

Here is the snippet of the code.

  private int bufferSize;

  this.bufferSize = conf.getInt( S3FileSystemConfigKeys.S3_STREAM_BUFFER_SIZE_KEY, S3FileSystemConfigKeys.S3_STREAM_BUFFER_SIZE_DEFAULT);

  public File retrieveBlock(Block block, long byteRangeStart)
    throws IOException {
    File fileBlock = null;
    InputStream in = null;
    OutputStream out = null;
    try {
      fileBlock = newBackupFile();
      in = get(blockToKey(block), byteRangeStart);
      out = new BufferedOutputStream(new FileOutputStream(fileBlock));
      byte[] buf = new byte[bufferSize];
      int numRead;
      while ((numRead = in.read(buf)) >= 0) {
        out.write(buf, 0, numRead);
      }
      return fileBlock;
    } catch (IOException e) {
      ...
    } finally {
      ...
    }
  }

Similar case: Hadoop-15415.