Hadoop Common
  1. Hadoop Common
  2. HADOOP-8587

HarFileSystem access of harMetaCache isn't threadsafe

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 2.0.0-alpha
    • Fix Version/s: 1.2.0, 0.23.3, 2.0.2-alpha
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HarFileSystem's use of the static harMetaCache map is not threadsafe. Credit to Todd for pointing this out.

      1. hadoop-8587.txt
        1 kB
        Eli Collins
      2. hadoop-8587.txt
        2 kB
        Eli Collins
      3. hadoop-8587-b1.txt
        1 kB
        Eli Collins

        Activity

        Eli Collins created issue -
        Eli Collins made changes -
        Field Original Value New Value
        Attachment hadoop-8587.txt [ 12535980 ]
        Eli Collins made changes -
        Assignee Eli Collins [ eli ]
        Affects Version/s 1.0.0 [ 12318244 ]
        Target Version/s 2.0.1-alpha [ 12321441 ] 1.2.0, 2.0.1-alpha [ 12321659, 12321441 ]
        Priority Major [ 3 ] Minor [ 4 ]
        Component/s fs [ 12310689 ]
        Eli Collins made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Eli Collins made changes -
        Attachment hadoop-8587.txt [ 12535996 ]
        Eli Collins made changes -
        Comment [ Integrated in Hadoop-Mapreduce-trunk #1133 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/])
            Revert previous commit, accidentally included HADOOP-8587. (Revision 1360055)

             Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
        Files :
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
        * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
        ]
        Eli Collins made changes -
        Comment [ Integrated in Hadoop-Common-trunk-Commit #2447 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2447/])
            Revert previous commit, accidentally included HADOOP-8587. (Revision 1360055)

             Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
        Files :
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
        * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
        ]
        Eli Collins made changes -
        Comment [ Integrated in Hadoop-Hdfs-trunk-Commit #2514 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2514/])
            Revert previous commit, accidentally included HADOOP-8587. (Revision 1360055)

             Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
        Files :
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
        * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
        ]
        Eli Collins made changes -
        Comment [ Integrated in Hadoop-Mapreduce-trunk-Commit #2466 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2466/])
            Revert previous commit, accidentally included HADOOP-8587. (Revision 1360055)

             Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
        Files :
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
        * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
        ]
        Eli Collins made changes -
        Comment [ Integrated in Hadoop-Hdfs-trunk #1100 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/])
            Revert previous commit, accidentally included HADOOP-8587. (Revision 1360055)

             Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
        Files :
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
        * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
        * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
        * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
        ]
        Eli Collins made changes -
        Comment [ -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12535980/hadoop-8587.txt
          against trunk revision .

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
                                Please justify why no new tests are needed for this patch.
                                Also please list what manual steps were performed to verify this patch.

            -1 javac. The patch appears to cause the build to fail.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1189//console

        This message is automatically generated. ]
        Eli Collins made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Eli Collins made changes -
        Attachment hadoop-8587-b1.txt [ 12536088 ]
        Eli Collins made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Target Version/s 1.2.0, 2.0.1-alpha [ 12321659, 12321441 ]
        Fix Version/s 1.2.0 [ 12321659 ]
        Fix Version/s 2.0.1-alpha [ 12321441 ]
        Resolution Fixed [ 1 ]
        Robert Joseph Evans made changes -
        Fix Version/s 0.23.3 [ 12320059 ]
        Arun C Murthy made changes -
        Fix Version/s 2.0.2-alpha [ 12322473 ]
        Fix Version/s 2.1.0-alpha [ 12321441 ]
        Fix Version/s 1.2.0 [ 12321659 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Matt Foley made changes -
        Fix Version/s 1.2.0 [ 12321659 ]

          People

          • Assignee:
            Eli Collins
            Reporter:
            Eli Collins
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development