[HADOOP-4195] SequenceFile.Writer close() uses compressor after returning it to CodecPool.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.18.0
• Fix Version/s: 0.19.0
• Component/s: io
• Hadoop Flags: Reviewed
• Assignee: Arun Murthy
• Reporter: Hong Tang
• Created: 17/Sep/08 18:54
• Updated: 13/Jan/11 06:39
• Resolved: 18/Sep/08 02:01

Description

In function SequenceFile.Writer.close() (line 946): The first marked line returns the compressor while the second marked line will use the compressor again. This will lead to a race condition if another thread checks out the compressor between these two marked statements.

SequenceFile.java

    public synchronized void close() throws IOException {
      CodecPool.returnCompressor(compressor); // <==== compressor returned
      
      keySerializer.close();
      uncompressedValSerializer.close();
      if (compressedValSerializer != null) {
        compressedValSerializer.close(); // <===== compressor used
      }

      if (out != null) {
        
        // Close the underlying stream iff we own it...
        if (ownOutputStream) {
          out.close();
        } else {
          out.flush();
        }
        out = null;
      }
    }

Issue Links

• relates to: MAPREDUCE-2258 IFile reader closes stream and compressor in wrong order