Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8335

FSNamesystem should construct FSPermissionChecker only if permission is enabled

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0, 2.6.0, 2.7.0, 2.8.0, 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Only check permissions when permissions enabled in FSDirStatAndListingOp.getFileInfo() and getListingInt()

      Description

      FSNamesystem (2.5.x)/FSDirStatAndListingOp(current trunk) getFileInfo and getListingInt methods call getPermissionChecker() to construct a FSPermissionChecker regardless of isPermissionEnabled(). When permission checking is disabled, this leads to an unnecessary performance hit constructing a UserGroupInformation object that is never used.

      For example, from a stack dump when driving concurrent requests, they all end up blocking.

      Here's the thread holding the lock:

      "IPC Server handler 9 on 9000" daemon prio=10 tid=0x00007f78d8b9e800 nid=0x142f3 runnable [0x00007f78c2ddc000]
      java.lang.Thread.State: RUNNABLE
      at java.io.FileInputStream.readBytes(Native Method)
      at java.io.FileInputStream.read(FileInputStream.java:272)
      at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:334)

      • locked <0x00000007d9b105c0> (a java.lang.UNIXProcess$ProcessPipeInputStream)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
      • locked <0x00000007d9b1a888> (a java.io.InputStreamReader)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:154)
        at java.io.BufferedReader.read1(BufferedReader.java:205)
        at java.io.BufferedReader.read(BufferedReader.java:279)
      • locked <0x00000007d9b1a888> (a java.io.InputStreamReader)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.parseExecResult(Shell.java:715)
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:524)
        at org.apache.hadoop.util.Shell.run(Shell.java:455)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:774)
        at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:84)
        at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
        at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
        at org.apache.hadoop.security.Groups.getGroups(Groups.java:139)
        at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1474)
      • locked <0x00000007a6df75f8> (a org.apache.hadoop.security.UserGroupInformation)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.<init>(FSPermissionChecker.java:82)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getPermissionChecker(FSNamesystem.java:3534)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListingInt(FSNamesystem.java:4489)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListing(FSNamesystem.java:4478)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getListing(NameNodeRpcServer.java:898)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getListing(ClientNamenodeProtocolServerSideTranslatorPB.java:602)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)

      Here is (one of the many) threads waiting on the lock:

      "IPC Server handler 2 on 9000" daemon prio=10 tid=0x00007f78d8c48800 nid=0x142ec waiting for monitor entry [0x00007f78c34e3000]
      java.lang.Thread.State: BLOCKED (on object monitor)
      at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1472)

      • waiting to lock <0x00000007a6df75f8> (a org.apache.hadoop.security.UserGroupInformation)
        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.<init>(FSPermissionChecker.java:82)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getPermissionChecker(FSNamesystem.java:3534)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListingInt(FSNamesystem.java:4489)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListing(FSNamesystem.java:4478)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getListing(NameNodeRpcServer.java:898)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getListing(ClientNamenodeProtocolServerSideTranslatorPB.java:602)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
      1. HDFS-8335.2.patch
        3 kB
        Gabor Liptak
      2. HDFS-8335.3.patch
        3 kB
        Gabor Liptak
      3. HDFS-8335.patch
        3 kB
        Gabor Liptak

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 35s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        -1 javac 0m 26s The patch appears to cause the build to fail.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12731670/HDFS-8335.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 6471d18
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10893/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac 0m 26s The patch appears to cause the build to fail. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731670/HDFS-8335.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6471d18 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10893/console This message was automatically generated.
        Hide
        gliptak Gabor Liptak added a comment -

        https://builds.apache.org/job/PreCommit-HDFS-Build/10893/consoleText

        says the compile failed, but doesn't seem to have the actual compilation error listed ...

        Show
        gliptak Gabor Liptak added a comment - https://builds.apache.org/job/PreCommit-HDFS-Build/10893/consoleText says the compile failed, but doesn't seem to have the actual compilation error listed ...
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 17m 0s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 32s There were no new javac warning messages.
        +1 javadoc 9m 37s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 22s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 21s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 2s Pre-build of native portion
        -1 hdfs tests 163m 18s Tests failed in hadoop-hdfs.
            206m 39s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.TestAppendSnapshotTruncate
          hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
          hadoop.hdfs.server.namenode.ha.TestHAAppend
          hadoop.hdfs.TestClientReportBadBlock



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12746243/HDFS-8335.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 773c670
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11760/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11760/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11760/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11760/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 0s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 22s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 2s Pre-build of native portion -1 hdfs tests 163m 18s Tests failed in hadoop-hdfs.     206m 39s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.TestClientReportBadBlock Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746243/HDFS-8335.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 773c670 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11760/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11760/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11760/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11760/console This message was automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -

        Hi Gabor Liptak, can you please rebase the patch? Thanks.

        Show
        wheat9 Haohui Mai added a comment - Hi Gabor Liptak , can you please rebase the patch? Thanks.
        Hide
        gliptak Gabor Liptak added a comment -

        Haohui Mai Rebased

        Show
        gliptak Gabor Liptak added a comment - Haohui Mai Rebased
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 57s trunk passed
        +1 compile 0m 48s trunk passed with JDK v1.8.0_66
        +1 compile 0m 44s trunk passed with JDK v1.7.0_85
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 2m 4s trunk passed
        +1 javadoc 1m 13s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 58s trunk passed with JDK v1.7.0_85
        +1 mvninstall 1m 1s the patch passed
        +1 compile 0m 49s the patch passed with JDK v1.8.0_66
        +1 javac 0m 49s the patch passed
        +1 compile 0m 45s the patch passed with JDK v1.7.0_85
        +1 javac 0m 45s the patch passed
        +1 checkstyle 0m 17s the patch passed
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 1s Patch has no whitespace issues.
        +1 findbugs 2m 34s the patch passed
        +1 javadoc 1m 9s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 55s the patch passed with JDK v1.7.0_85
        -1 unit 58m 5s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 54m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_85.
        -1 asflicense 0m 20s Patch generated 58 ASF License warnings.
        142m 55s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
          hadoop.security.TestPermission
          hadoop.hdfs.TestCrcCorruption
          hadoop.hdfs.server.datanode.TestBlockScanner
        JDK v1.7.0_85 Failed junit tests hadoop.hdfs.TestReplaceDatanodeOnFailure
          hadoop.hdfs.TestErasureCodingPolicies
          hadoop.hdfs.server.namenode.TestAddStripedBlocks
          hadoop.security.TestPermission
          hadoop.hdfs.server.namenode.TestFSImageWithSnapshot



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773948/HDFS-8335.3.patch
        JIRA Issue HDFS-8335
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ccb10a25fa92 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4bff073
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
        JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13626/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 78MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13626/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 57s trunk passed +1 compile 0m 48s trunk passed with JDK v1.8.0_66 +1 compile 0m 44s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 2m 4s trunk passed +1 javadoc 1m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 58s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 1s the patch passed +1 compile 0m 49s the patch passed with JDK v1.8.0_66 +1 javac 0m 49s the patch passed +1 compile 0m 45s the patch passed with JDK v1.7.0_85 +1 javac 0m 45s the patch passed +1 checkstyle 0m 17s the patch passed +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 1s Patch has no whitespace issues. +1 findbugs 2m 34s the patch passed +1 javadoc 1m 9s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 55s the patch passed with JDK v1.7.0_85 -1 unit 58m 5s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 54m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 20s Patch generated 58 ASF License warnings. 142m 55s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.balancer.TestBalancer   hadoop.security.TestPermission   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.datanode.TestBlockScanner JDK v1.7.0_85 Failed junit tests hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.TestErasureCodingPolicies   hadoop.hdfs.server.namenode.TestAddStripedBlocks   hadoop.security.TestPermission   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773948/HDFS-8335.3.patch JIRA Issue HDFS-8335 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ccb10a25fa92 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4bff073 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13626/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13626/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 78MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13626/console This message was automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -

        +1. Committing it shortly.

        Show
        wheat9 Haohui Mai added a comment - +1. Committing it shortly.
        Hide
        wheat9 Haohui Mai added a comment -

        I've committed the patch to trunk and branch-2. Thanks Gabor Liptak for the contribution.

        Show
        wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks Gabor Liptak for the contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8880 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8880/)
        HDFS-8335. FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8880 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8880/ ) HDFS-8335 . FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #727 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/727/)
        HDFS-8335. FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #727 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/727/ ) HDFS-8335 . FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #717 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/717/)
        HDFS-8335. FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #717 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/717/ ) HDFS-8335 . FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2659 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2659/)
        HDFS-8335. FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2659 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2659/ ) HDFS-8335 . FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1450 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1450/)
        HDFS-8335. FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1450 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1450/ ) HDFS-8335 . FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/)
        HDFS-8335. FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/ ) HDFS-8335 . FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #640 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/640/)
        HDFS-8335. FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #640 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/640/ ) HDFS-8335 . FSNamesystem should construct FSPermissionChecker only if (wheat9: rev 977e0b3c4ce76746a3d8590d2d790fdc96c86ca5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        zhz Zhe Zhang added a comment -

        Quick note that I'm working on backporting HDFS-10744 to branch-2.7; since this change is not in branch-2.7, the overlapping part won't be backported.

        If someone plans to backport this to branch-2.7, please remember to add the optimization from HDFS-10744.

        Show
        zhz Zhe Zhang added a comment - Quick note that I'm working on backporting HDFS-10744 to branch-2.7; since this change is not in branch-2.7, the overlapping part won't be backported. If someone plans to backport this to branch-2.7, please remember to add the optimization from HDFS-10744 .

          People

          • Assignee:
            gliptak Gabor Liptak
            Reporter:
            dmbryson David Bryson
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development