Hadoop Common
  1. Hadoop Common
  2. HADOOP-7194

Potential Resource leak in IOUtils.java

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.1, 0.22.0, 0.23.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      IOUtils.java
      
      try {
            copyBytes(in, out, buffSize);
          } finally {
            if(close) {
              out.close();
              in.close();
            }
          }
      

      In the above code if any exception throws from the out.close() statement, in.close() statement will not execute and the input stream will not be closed.

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #645 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/645/)
          HADOOP-7194. Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #645 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/645/ ) HADOOP-7194 . Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-22-branch #37 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/37/)
          HADOOP-7194. Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K

          Show
          Hudson added a comment - Integrated in Hadoop-Common-22-branch #37 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/37/ ) HADOOP-7194 . Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #536 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/536/)
          HADOOP-7194. Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #536 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/536/ ) HADOOP-7194 . Fix resource leak in IOUtils.copyBytes(..). Contributed by Devaraj K
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Devaraj!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Devaraj!
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12474511/HADOOP-7194.patch
          against trunk revision 1085122.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12474511/HADOOP-7194.patch against trunk revision 1085122. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/321//console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Closing the streams in the finally-block using closeStream(..) will suppress the exception ...

          Good point.

          +1 then.

          Show
          Tsz Wo Nicholas Sze added a comment - > Closing the streams in the finally-block using closeStream(..) will suppress the exception ... Good point. +1 then.
          Hide
          Devaraj K added a comment -

          Thanks Nicholas for reviewing.

          Closing the streams in the finally-block using closeStream(..) will suppress the exception if out.close()/in.close() throws and will not be thrown back to the caller.

           
          
          if(close) {
                   out.close();
          +        out = null;
                   in.close();
          +        in = null;
                 }
          

          With the above lines if out.close() throws exception that will be thrown back to the caller and also other streams will be closed in finally-block using closeStream(..).

          As per the discussions in the issue MAPREDUCE-2243, this way is followed.

          Show
          Devaraj K added a comment - Thanks Nicholas for reviewing. Closing the streams in the finally-block using closeStream(..) will suppress the exception if out.close()/in.close() throws and will not be thrown back to the caller. if(close) { out.close(); + out = null; in.close(); + in = null; } With the above lines if out.close() throws exception that will be thrown back to the caller and also other streams will be closed in finally-block using closeStream(..). As per the discussions in the issue MAPREDUCE-2243 , this way is followed.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Good catch! It seems that closing in the finally-block using closeStream(..) is sufficient. The following is not required.

                 if(close) {
                   out.close();
          +        out = null;
                   in.close();
          +        in = null;
                 }
          
          Show
          Tsz Wo Nicholas Sze added a comment - Good catch! It seems that closing in the finally-block using closeStream(..) is sufficient. The following is not required. if (close) { out.close(); + out = null ; in.close(); + in = null ; }

            People

            • Assignee:
              Devaraj K
              Reporter:
              Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development