Hadoop Common
  1. Hadoop Common
  2. HADOOP-7194

Potential Resource leak in IOUtils.java

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.1, 0.22.0, 0.23.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      IOUtils.java
      
      try {
            copyBytes(in, out, buffSize);
          } finally {
            if(close) {
              out.close();
              in.close();
            }
          }
      

      In the above code if any exception throws from the out.close() statement, in.close() statement will not execute and the input stream will not be closed.

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          Good catch! It seems that closing in the finally-block using closeStream(..) is sufficient. The following is not required.

                 if(close) {
                   out.close();
          +        out = null;
                   in.close();
          +        in = null;
                 }
          
          Show
          Tsz Wo Nicholas Sze added a comment - Good catch! It seems that closing in the finally-block using closeStream(..) is sufficient. The following is not required. if (close) { out.close(); + out = null ; in.close(); + in = null ; }
          Hide
          Devaraj K added a comment -

          Thanks Nicholas for reviewing.

          Closing the streams in the finally-block using closeStream(..) will suppress the exception if out.close()/in.close() throws and will not be thrown back to the caller.

           
          
          if(close) {
                   out.close();
          +        out = null;
                   in.close();
          +        in = null;
                 }
          

          With the above lines if out.close() throws exception that will be thrown back to the caller and also other streams will be closed in finally-block using closeStream(..).

          As per the discussions in the issue MAPREDUCE-2243, this way is followed.

          Show
          Devaraj K added a comment - Thanks Nicholas for reviewing. Closing the streams in the finally-block using closeStream(..) will suppress the exception if out.close()/in.close() throws and will not be thrown back to the caller. if(close) { out.close(); + out = null; in.close(); + in = null; } With the above lines if out.close() throws exception that will be thrown back to the caller and also other streams will be closed in finally-block using closeStream(..). As per the discussions in the issue MAPREDUCE-2243 , this way is followed.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Closing the streams in the finally-block using closeStream(..) will suppress the exception ...

          Good point.

          +1 then.

          Show
          Tsz Wo Nicholas Sze added a comment - > Closing the streams in the finally-block using closeStream(..) will suppress the exception ... Good point. +1 then.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12474511/HADOOP-7194.patch
          against trunk revision 1085122.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12474511/HADOOP-7194.patch against trunk revision 1085122. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Devaraj!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Devaraj!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #536 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/536/)
          HADOOP-7194. Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #536 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/536/ ) HADOOP-7194 . Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-22-branch #37 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/37/)
          HADOOP-7194. Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K

          Show
          Hudson added a comment - Integrated in Hadoop-Common-22-branch #37 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/37/ ) HADOOP-7194 . Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #645 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/645/)
          HADOOP-7194. Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #645 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/645/ ) HADOOP-7194 . Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K

            People

            • Assignee:
              Devaraj K
              Reporter:
              Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development