Hadoop Common
  1. Hadoop Common
  2. HADOOP-10690

Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close()

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.0
    • Component/s: tools
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

          public void close() throws IOException {
            in.close();
          }
      

      The close() method should be protected by synchronized keyword.

        Issue Links

          Activity

          Vinod Kumar Vavilapalli made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Chris Nauroth made changes -
          Affects Version/s 3.0.0 [ 12320357 ]
          Chris Nauroth made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 2.7.0 [ 12327583 ]
          Fix Version/s 3.0.0 [ 12320357 ]
          Resolution Fixed [ 1 ]
          Chris Nauroth made changes -
          Target Version/s 2.6.0 [ 12327179 ] 2.7.0 [ 12327583 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12650666/HADOOP-10690.patch
          against trunk revision 828429d.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-azure.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4941//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4941//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12650666/HADOOP-10690.patch against trunk revision 828429d. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-tools/hadoop-azure. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4941//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4941//console This message is automatically generated.
          Hide
          Chen He added a comment -

          Hi Chris Nauroth, do we need to mark this ticket as closed since it is checked in?

          Show
          Chen He added a comment - Hi Chris Nauroth , do we need to mark this ticket as closed since it is checked in?
          Karthik Kambatla (Inactive) made changes -
          Target Version/s 3.0.0, 2.5.0 [ 12320357, 12326263 ] 2.6.0 [ 12327179 ]
          Chris Nauroth made changes -
          Link This issue relates to HADOOP-9629 [ HADOOP-9629 ]
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1808 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1808/)
          HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1808 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1808/ ) HADOOP-10690 . Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1781 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1781/)
          HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1781 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1781/ ) HADOOP-10690 . Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #590 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/590/)
          HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #590 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/590/ ) HADOOP-10690 . Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #5744 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5744/)
          HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5744 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5744/ ) HADOOP-10690 . Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1604236 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
          Chris Nauroth made changes -
          Fix Version/s 3.0.0 [ 12320357 ]
          Hide
          Chris Nauroth added a comment -

          I committed this to trunk. I'm keeping the issue open for now until we merge to branch-2.

          Show
          Chris Nauroth added a comment - I committed this to trunk. I'm keeping the issue open for now until we merge to branch-2.
          Chris Nauroth made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Affects Version/s 3.0.0 [ 12320357 ]
          Target Version/s 0.23.11, 2.5.0 [ 12324665, 12326263 ] 3.0.0, 2.5.0 [ 12320357, 12326263 ]
          Component/s tools [ 12319643 ]
          Hide
          Chris Nauroth added a comment -

          +1 for the patch. I plan to commit this later today.

          Show
          Chris Nauroth added a comment - +1 for the patch. I plan to commit this later today.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12650666/HADOOP-10690.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-azure.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4074//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4074//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12650666/HADOOP-10690.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-tools/hadoop-azure. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4074//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4074//console This message is automatically generated.
          Chen He made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 0.23.11, 2.5.0 [ 12324665, 12326263 ]
          Chen He made changes -
          Assignee Chen He [ airbots ]
          Chen He made changes -
          Field Original Value New Value
          Attachment HADOOP-10690.patch [ 12650666 ]
          Hide
          Chen He added a comment -

          Patch submitted.

          Show
          Chen He added a comment - Patch submitted.
          Hide
          Chris Nauroth added a comment -

          The equivalent class in the HDFS client is DFSInputStream, and it has a close method that is synchronized. I don't know that lack of synchronized here is causing any specific problems, but I suppose we ought to add it for maximum compatibility with HDFS semantics.

          The underlying class that ends up getting used from the Azure SDK is BlobInputStream. Reading its source, it looks like that class has a synchronized close. Even though it's synchronized at that layer, we might as well go ahead and add it here too.

          Show
          Chris Nauroth added a comment - The equivalent class in the HDFS client is DFSInputStream , and it has a close method that is synchronized. I don't know that lack of synchronized here is causing any specific problems, but I suppose we ought to add it for maximum compatibility with HDFS semantics. The underlying class that ends up getting used from the Azure SDK is BlobInputStream . Reading its source, it looks like that class has a synchronized close . Even though it's synchronized at that layer, we might as well go ahead and add it here too.
          Hide
          Chen He added a comment -

          Do we really need synchronized keyword on the close?

          Show
          Chen He added a comment - Do we really need synchronized keyword on the close?
          Ted Yu created issue -

            People

            • Assignee:
              Chen He
              Reporter:
              Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development