Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6329

WebHdfs does not work if HA is enabled on NN but logical URI is not configured.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      After HDFS-6100, namenode unconditionally puts the logical name (name service id) as the token service when redirecting webhdfs requests to datanodes, if it detects HA.

      For HA configurations with no client-side failover proxy provider (e.g. IP failover), webhdfs does not work since the clients do not use logical name.

      1. HDFS-6329.patch
        7 kB
        Kihwal Lee
      2. HDFS-6329.patch
        5 kB
        Kihwal Lee
      3. HDFS-6329.v2.patch
        9 kB
        Kihwal Lee
      4. HDFS-6329.v3.patch
        9 kB
        Kihwal Lee
      5. HDFS-6329.v4.patch
        10 kB
        Kihwal Lee
      6. HDFS-6329.v5.patch
        9 kB
        Kihwal Lee

        Activity

        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #1752 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1752/)
        HDFS-6329. WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1752 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1752/ ) HDFS-6329 . WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1778 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1778/)
        HDFS-6329. WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1778 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1778/ ) HDFS-6329 . WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #560 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/560/)
        HDFS-6329. WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #560 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/560/ ) HDFS-6329 . WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #5599 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5599/)
        HDFS-6329. WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5599 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5599/ ) HDFS-6329 . WebHdfs does not work if HA is enabled on NN but logical URI is not configured. Contributed by Kihwal Lee. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1593470 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFSForHA.java
        Hide
        Kihwal Lee added a comment -

        Committed to trunk, branch-2 and branch-2.4. Thanks for the reviews, Daryn.

        Show
        Kihwal Lee added a comment - Committed to trunk, branch-2 and branch-2.4. Thanks for the reviews, Daryn.
        Hide
        Daryn Sharp added a comment -

        +1 LOG.debug("DSDS looks good")

        Show
        Daryn Sharp added a comment - +1 LOG.debug("DSDS looks good")
        Hide
        Kihwal Lee added a comment -

        Removed the debug line

        Show
        Kihwal Lee added a comment - Removed the debug line
        Hide
        Kihwal Lee added a comment -

        Ouch. Now you know what I use as a debug marker in logs. I will take it out.

        Maybe I missed it, but a test cast to simulate our HA IP-failover would be invaluable to preventing another regression.

        That's very tricky with the current state of MiniDFSCluster and other broken checks in HA. HDFS-6360 will fix MiniDFSCluster and HDFS-6334 will fix HA checks. Perhap we can add more meaningful tests after that.

        Show
        Kihwal Lee added a comment - Ouch. Now you know what I use as a debug marker in logs. I will take it out. Maybe I missed it, but a test cast to simulate our HA IP-failover would be invaluable to preventing another regression. That's very tricky with the current state of MiniDFSCluster and other broken checks in HA. HDFS-6360 will fix MiniDFSCluster and HDFS-6334 will fix HA checks. Perhap we can add more meaningful tests after that.
        Hide
        Daryn Sharp added a comment -

        You may want to remove this line LOG.info("KLKL uri = " + uri);.

        Maybe I missed it, but a test cast to simulate our HA IP-failover would be invaluable to preventing another regression.

        Show
        Daryn Sharp added a comment - You may want to remove this line LOG.info("KLKL uri = " + uri); . Maybe I missed it, but a test cast to simulate our HA IP-failover would be invaluable to preventing another regression.
        Hide
        Kihwal Lee added a comment -

        We should really sort things out and make test cases do the right thing instead of relying on side-effects of shared conf [ in MiniDFSCluster]. But that's for another day and I am simply making it behave like the old way for now.

        I saw unit test failures in other JIRAs due to this issue, so I filed HDFS-6360.

        Show
        Kihwal Lee added a comment - We should really sort things out and make test cases do the right thing instead of relying on side-effects of shared conf [ in MiniDFSCluster]. But that's for another day and I am simply making it behave like the old way for now. I saw unit test failures in other JIRAs due to this issue, so I filed HDFS-6360 .
        Hide
        Kihwal Lee added a comment -

        The test failures are not caused by this jira. The TestBPOfferService failure had the same error reported in HDFS-3930 and could not reproduce it on a desktop. The TestBalancerWithNodeGroup issue was reported in HDFS-6250.

        Show
        Kihwal Lee added a comment - The test failures are not caused by this jira. The TestBPOfferService failure had the same error reported in HDFS-3930 and could not reproduce it on a desktop. The TestBalancerWithNodeGroup issue was reported in HDFS-6250 .
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12643839/HDFS-6329.v4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.datanode.TestBPOfferService
        org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6851//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6851//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643839/HDFS-6329.v4.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.datanode.TestBPOfferService org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6851//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6851//console This message is automatically generated.
        Hide
        Kihwal Lee added a comment -

        Fixed MiniDFSCluster. We should really sort things out and make test cases do the right thing instead of relying on side-effects of shared conf. But that's for another day and I am simply making it behave like the old way for now.

        Removed the statement that is no longer used to fix the findbugs warning.

        Show
        Kihwal Lee added a comment - Fixed MiniDFSCluster. We should really sort things out and make test cases do the right thing instead of relying on side-effects of shared conf. But that's for another day and I am simply making it behave like the old way for now. Removed the statement that is no longer used to fix the findbugs warning.
        Hide
        Kihwal Lee added a comment -

        Does setClientNamenodeAddress need to set clientNamenodeAddress when it's not logical? Will the later extraction of the service from the RPC server not work?

        No, that won't work. If a non-logical service address is used, clients need to use that address to contact the namenode. For example, in a HA configured cluster with IP failover, you want the service address to be used by WebHdfs in order to enable failover. The rpc server address in HA config contains individual host names, so that DNs can talk to two NNs separately. If we do what you mentioned, the DFSClient created on behalf of a WebHdfs request will only be able to talk to one of the NNs.

        The test failures are due to yet another problem in MiniDFSCluster. The expected state of config after starting one or more namenode is all different in single NN, federation and HA. Certain test cases happens to work because NN and client share a conf and NN happens to smash some variables. I will have a new patch up shortly to fix the test problem.

        Show
        Kihwal Lee added a comment - Does setClientNamenodeAddress need to set clientNamenodeAddress when it's not logical? Will the later extraction of the service from the RPC server not work? No, that won't work. If a non-logical service address is used, clients need to use that address to contact the namenode. For example, in a HA configured cluster with IP failover, you want the service address to be used by WebHdfs in order to enable failover. The rpc server address in HA config contains individual host names, so that DNs can talk to two NNs separately. If we do what you mentioned, the DFSClient created on behalf of a WebHdfs request will only be able to talk to one of the NNs. The test failures are due to yet another problem in MiniDFSCluster. The expected state of config after starting one or more namenode is all different in single NN, federation and HA. Certain test cases happens to work because NN and client share a conf and NN happens to smash some variables. I will have a new patch up shortly to fix the test problem.
        Hide
        Daryn Sharp added a comment -

        Does setClientNamenodeAddress need to set clientNamenodeAddress when it's not logical? Will the later extraction of the service from the RPC server not work?

        Show
        Daryn Sharp added a comment - Does setClientNamenodeAddress need to set clientNamenodeAddress when it's not logical? Will the later extraction of the service from the RPC server not work?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12643640/HDFS-6329.v3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
        org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery
        org.apache.hadoop.hdfs.server.namenode.TestEditLogAutoroll
        org.apache.hadoop.hdfs.TestDistributedFileSystem

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6842//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6842//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6842//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643640/HDFS-6329.v3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery org.apache.hadoop.hdfs.server.namenode.TestEditLogAutoroll org.apache.hadoop.hdfs.TestDistributedFileSystem +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6842//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6842//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6842//console This message is automatically generated.
        Hide
        Kihwal Lee added a comment -

        Refined & simplified the initial check in setClientNamenodeAddress().

        Show
        Kihwal Lee added a comment - Refined & simplified the initial check in setClientNamenodeAddress() .
        Hide
        Kihwal Lee added a comment -

        The new patch fixes another issue in MiniDFSCluster. One conf was kept resued for creating NN, which internally sets the default file system.

        Show
        Kihwal Lee added a comment - The new patch fixes another issue in MiniDFSCluster. One conf was kept resued for creating NN, which internally sets the default file system.
        Hide
        Kihwal Lee added a comment -

        The idea was sound, but didn't work with the way MiniDFSCluster is setting up namenode. I modified the patch to

        • have MiniDFSCluster set fs.defaultFs to "hdfs://127.0.0.1:0" instead of "127.0.0.1:0".
        • make NameNode delay the setting of clientNamenodeAddress, if it is binding to a random port.
        Show
        Kihwal Lee added a comment - The idea was sound, but didn't work with the way MiniDFSCluster is setting up namenode. I modified the patch to have MiniDFSCluster set fs.defaultFs to "hdfs://127.0.0.1:0" instead of "127.0.0.1:0". make NameNode delay the setting of clientNamenodeAddress, if it is binding to a random port.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12643177/HDFS-6329.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.fs.shell.TestHdfsTextCommand
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
        org.apache.hadoop.hdfs.server.namenode.TestAclConfigFlag
        org.apache.hadoop.hdfs.TestClose
        org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
        org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
        org.apache.hadoop.hdfs.TestFSInputChecker
        org.apache.hadoop.hdfs.server.namenode.TestBackupNode
        org.apache.hadoop.hdfs.TestDataTransferProtocol
        org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
        org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
        org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
        org.apache.hadoop.hdfs.server.namenode.TestFSNamesystemMBean
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
        org.apache.hadoop.tools.TestJMXGet
        org.apache.hadoop.hdfs.TestDFSShell
        org.apache.hadoop.hdfs.server.namenode.TestFSImage
        org.apache.hadoop.hdfs.security.TestDelegationToken
        org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
        org.apache.hadoop.fs.permission.TestStickyBit
        org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
        org.apache.hadoop.hdfs.TestFileConcurrentReader
        org.apache.hadoop.hdfs.server.datanode.TestCachingStrategy
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
        org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
        org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
        org.apache.hadoop.fs.TestFcHdfsCreateMkdir
        org.apache.hadoop.hdfs.TestCrcCorruption
        org.apache.hadoop.hdfs.TestAppendDifferentChecksum
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
        org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
        org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser
        org.apache.hadoop.hdfs.server.namenode.TestSequentialBlockId
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
        org.apache.hadoop.hdfs.tools.TestGetGroups
        org.apache.hadoop.hdfs.TestDFSPermission
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
        org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
        org.apache.hadoop.hdfs.TestListFilesInFileContext
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
        org.apache.hadoop.fs.viewfs.TestViewFsDefaultValue
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
        org.apache.hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
        org.apache.hadoop.hdfs.TestDatanodeReport
        org.apache.hadoop.hdfs.TestDFSRemove
        org.apache.hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
        org.apache.hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN
        org.apache.hadoop.hdfs.TestRestartDFS
        org.apache.hadoop.hdfs.TestHdfsAdmin
        org.apache.hadoop.hdfs.TestFSOutputSummer
        org.apache.hadoop.hdfs.TestDatanodeRegistration
        org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
        org.apache.hadoop.hdfs.TestHDFSTrash
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
        org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl
        org.apache.hadoop.hdfs.TestDFSRollback
        org.apache.hadoop.hdfs.server.namenode.TestFSDirectory
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
        org.apache.hadoop.hdfs.TestClientReportBadBlock
        org.apache.hadoop.hdfs.server.namenode.TestAuditLogger
        org.apache.hadoop.hdfs.TestDecommission
        org.apache.hadoop.hdfs.TestBalancerBandwidth
        org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi
        org.apache.hadoop.hdfs.TestSmallBlock
        org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
        org.apache.hadoop.hdfs.server.datanode.TestHSync
        org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives
        org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication
        org.apache.hadoop.fs.TestEnhancedByteBufferAccess
        org.apache.hadoop.hdfs.TestSnapshotCommands
        org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
        org.apache.hadoop.hdfs.server.datanode.TestReadOnlySharedStorage
        org.apache.hadoop.hdfs.TestFileAppend2
        org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
        org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
        org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy
        org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
        org.apache.hadoop.hdfs.TestFileCreationDelete
        org.apache.hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
        org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
        org.apache.hadoop.hdfs.server.namenode.TestAuditLogs
        org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
        org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
        org.apache.hadoop.hdfs.TestFileAppend
        org.apache.hadoop.hdfs.TestMissingBlocksAlert
        org.apache.hadoop.hdfs.TestSeekBug
        org.apache.hadoop.hdfs.TestBlockReaderFactory
        org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
        org.apache.hadoop.hdfs.TestAbandonBlock
        org.apache.hadoop.hdfs.server.datanode.TestDiskError
        org.apache.hadoop.hdfs.TestRenameWhileOpen
        org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
        org.apache.hadoop.hdfs.server.datanode.TestIncrementalBrVariations
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
        org.apache.hadoop.hdfs.TestBlockReaderLocal
        org.apache.hadoop.cli.TestHDFSCLI
        org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
        org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl
        org.apache.hadoop.hdfs.TestGetBlocks
        org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade
        org.apache.hadoop.hdfs.server.datanode.TestStorageReport
        org.apache.hadoop.hdfs.TestDFSOutputStream
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeInitStorage
        org.apache.hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
        org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf
        org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
        org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
        org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
        org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean
        org.apache.hadoop.fs.viewfs.TestViewFsHdfs
        org.apache.hadoop.hdfs.TestParallelShortCircuitReadUnCached
        org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
        org.apache.hadoop.hdfs.TestBlockMissingException
        org.apache.hadoop.hdfs.server.balancer.TestBalancer
        org.apache.hadoop.hdfs.TestQuota
        org.apache.hadoop.fs.TestSymlinkHdfsDisable
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
        org.apache.hadoop.fs.TestUrlStreamHandler
        org.apache.hadoop.hdfs.TestFileCreation
        org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
        org.apache.hadoop.fs.TestHDFSFileContextMainOperations
        org.apache.hadoop.hdfs.TestSetrepDecreasing
        org.apache.hadoop.hdfs.TestFileCreationEmpty
        org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
        org.apache.hadoop.hdfs.TestDatanodeDeath
        org.apache.hadoop.hdfs.TestParallelRead
        org.apache.hadoop.cli.TestAclCLI
        org.apache.hadoop.hdfs.TestFileStatus
        org.apache.hadoop.hdfs.TestRollingUpgrade
        org.apache.hadoop.hdfs.TestFileAppend3
        org.apache.hadoop.hdfs.TestHFlush
        org.apache.hadoop.hdfs.TestIsMethodSupported
        org.apache.hadoop.hdfs.TestListFilesInDFS
        org.apache.hadoop.hdfs.TestPipelines
        org.apache.hadoop.hdfs.TestRollingUpgradeRollback
        org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
        org.apache.hadoop.security.TestRefreshUserMappings
        org.apache.hadoop.security.TestPermissionSymlinks
        org.apache.hadoop.hdfs.server.datanode.TestTransferRbw
        org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
        org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
        org.apache.hadoop.fs.TestGlobPaths
        org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
        org.apache.hadoop.hdfs.server.namenode.TestMetaSave
        org.apache.hadoop.hdfs.TestDFSStartupVersions
        org.apache.hadoop.TestRefreshCallQueue
        org.apache.hadoop.hdfs.TestFileCreationClient
        org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
        org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
        org.apache.hadoop.hdfs.TestLeaseRecovery
        org.apache.hadoop.hdfs.TestLocalDFS
        org.apache.hadoop.security.TestPermission
        org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
        org.apache.hadoop.hdfs.web.TestHttpsFileSystem
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
        org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
        org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
        org.apache.hadoop.hdfs.TestDFSRename
        org.apache.hadoop.hdfs.TestDFSAddressConfig
        org.apache.hadoop.hdfs.TestFetchImage
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestFullPathNameWithSnapshot
        org.apache.hadoop.hdfs.server.namenode.TestAddBlock
        org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
        org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
        org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
        org.apache.hadoop.hdfs.TestSetrepIncreasing
        org.apache.hadoop.hdfs.TestLeaseRecovery2
        org.apache.hadoop.net.TestNetworkTopology
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeRpcServer
        org.apache.hadoop.hdfs.TestWriteRead
        org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
        org.apache.hadoop.hdfs.TestMiniDFSCluster
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing
        org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
        org.apache.hadoop.hdfs.TestFileCorruption
        org.apache.hadoop.hdfs.TestHDFSFileSystemContract
        org.apache.hadoop.hdfs.TestParallelShortCircuitRead
        org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
        org.apache.hadoop.hdfs.TestReadWhileWriting
        org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
        org.apache.hadoop.hdfs.TestDFSFinalize
        org.apache.hadoop.hdfs.TestMultiThreadedHflush
        org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
        org.apache.hadoop.hdfs.server.namenode.TestCreateEditsLog
        org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
        org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache
        org.apache.hadoop.hdfs.TestDFSUpgrade
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery
        org.apache.hadoop.hdfs.TestDFSMkdirs
        org.apache.hadoop.hdfs.TestDatanodeConfig
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
        org.apache.hadoop.hdfs.server.namenode.TestStartup
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean
        org.apache.hadoop.hdfs.TestEncryptedTransfer
        org.apache.hadoop.hdfs.TestReplication
        org.apache.hadoop.hdfs.TestConnCache
        org.apache.hadoop.hdfs.server.namenode.TestHostsFiles
        org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
        org.apache.hadoop.hdfs.server.datanode.TestIncrementalBlockReports
        org.apache.hadoop.hdfs.TestRead
        org.apache.hadoop.fs.TestResolveHdfsSymlink
        org.apache.hadoop.hdfs.TestDFSShellGenericOptions
        org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
        org.apache.hadoop.hdfs.web.TestWebHdfsWithAuthenticationFilter
        org.apache.hadoop.hdfs.TestSafeMode
        org.apache.hadoop.hdfs.web.TestWebHDFS
        org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead
        org.apache.hadoop.hdfs.TestDFSClientRetries
        org.apache.hadoop.hdfs.TestDisableConnCache
        org.apache.hadoop.fs.TestFcHdfsSetUMask
        org.apache.hadoop.hdfs.server.namenode.TestAddBlockRetry
        org.apache.hadoop.fs.TestSymlinkHdfsFileSystem
        org.apache.hadoop.fs.TestSymlinkHdfsFileContext
        org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl
        org.apache.hadoop.hdfs.TestFileAppend4
        org.apache.hadoop.fs.viewfs.TestViewFsAtHdfsRoot
        org.apache.hadoop.hdfs.TestModTime
        org.apache.hadoop.hdfs.TestLease
        org.apache.hadoop.hdfs.TestDataTransferKeepalive
        org.apache.hadoop.hdfs.TestLargeBlock
        org.apache.hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
        org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache
        org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
        org.apache.hadoop.hdfs.TestClientBlockVerification
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot
        org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
        org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes
        org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
        org.apache.hadoop.hdfs.TestSetTimes
        org.apache.hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
        org.apache.hadoop.fs.viewfs.TestViewFsFileStatusHdfs
        org.apache.hadoop.hdfs.TestDatanodeBlockScanner
        org.apache.hadoop.hdfs.web.TestWebHDFSAcl
        org.apache.hadoop.hdfs.TestPersistBlocks
        org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
        org.apache.hadoop.hdfs.server.namenode.TestINodeFile
        org.apache.hadoop.hdfs.server.namenode.TestFileLimit
        org.apache.hadoop.fs.TestFcHdfsPermission
        org.apache.hadoop.hdfs.TestParallelUnixDomainRead
        org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
        org.apache.hadoop.hdfs.server.namenode.TestFsck
        org.apache.hadoop.cli.TestCacheAdminCLI
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
        org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
        org.apache.hadoop.hdfs.security.token.block.TestBlockToken
        org.apache.hadoop.hdfs.TestDistributedFileSystem
        org.apache.hadoop.hdfs.TestPread
        org.apache.hadoop.hdfs.TestBlocksScheduledCounter
        org.apache.hadoop.hdfs.TestFileAppendRestart
        org.apache.hadoop.hdfs.server.namenode.TestLeaseManager
        org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
        org.apache.hadoop.hdfs.server.namenode.TestEditLog

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6807//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6807//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6807//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643177/HDFS-6329.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.fs.shell.TestHdfsTextCommand org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength org.apache.hadoop.hdfs.server.namenode.TestAclConfigFlag org.apache.hadoop.hdfs.TestClose org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.TestFSInputChecker org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.TestDataTransferProtocol org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting org.apache.hadoop.hdfs.server.namenode.TestFSNamesystemMBean org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters org.apache.hadoop.tools.TestJMXGet org.apache.hadoop.hdfs.TestDFSShell org.apache.hadoop.hdfs.server.namenode.TestFSImage org.apache.hadoop.hdfs.security.TestDelegationToken org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade org.apache.hadoop.fs.permission.TestStickyBit org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.hdfs.server.datanode.TestCachingStrategy org.apache.hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks org.apache.hadoop.fs.TestFcHdfsCreateMkdir org.apache.hadoop.hdfs.TestCrcCorruption org.apache.hadoop.hdfs.TestAppendDifferentChecksum org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser org.apache.hadoop.hdfs.server.namenode.TestSequentialBlockId org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap org.apache.hadoop.hdfs.tools.TestGetGroups org.apache.hadoop.hdfs.TestDFSPermission org.apache.hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot org.apache.hadoop.hdfs.TestDFSUpgradeFromImage org.apache.hadoop.hdfs.TestListFilesInFileContext org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker org.apache.hadoop.fs.viewfs.TestViewFsDefaultValue org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics org.apache.hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot org.apache.hadoop.hdfs.TestParallelShortCircuitReadNoChecksum org.apache.hadoop.hdfs.TestDatanodeReport org.apache.hadoop.hdfs.TestDFSRemove org.apache.hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage org.apache.hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN org.apache.hadoop.hdfs.TestRestartDFS org.apache.hadoop.hdfs.TestHdfsAdmin org.apache.hadoop.hdfs.TestFSOutputSummer org.apache.hadoop.hdfs.TestDatanodeRegistration org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks org.apache.hadoop.hdfs.TestHDFSTrash org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl org.apache.hadoop.hdfs.TestDFSRollback org.apache.hadoop.hdfs.server.namenode.TestFSDirectory org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks org.apache.hadoop.hdfs.TestClientReportBadBlock org.apache.hadoop.hdfs.server.namenode.TestAuditLogger org.apache.hadoop.hdfs.TestDecommission org.apache.hadoop.hdfs.TestBalancerBandwidth org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi org.apache.hadoop.hdfs.TestSmallBlock org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage org.apache.hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys org.apache.hadoop.hdfs.server.datanode.TestHSync org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication org.apache.hadoop.fs.TestEnhancedByteBufferAccess org.apache.hadoop.hdfs.TestSnapshotCommands org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement org.apache.hadoop.hdfs.server.datanode.TestReadOnlySharedStorage org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean org.apache.hadoop.hdfs.TestFileCreationDelete org.apache.hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader org.apache.hadoop.hdfs.server.namenode.TestAuditLogs org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling org.apache.hadoop.hdfs.TestFileAppend org.apache.hadoop.hdfs.TestMissingBlocksAlert org.apache.hadoop.hdfs.TestSeekBug org.apache.hadoop.hdfs.TestBlockReaderFactory org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner org.apache.hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots org.apache.hadoop.hdfs.TestAbandonBlock org.apache.hadoop.hdfs.server.datanode.TestDiskError org.apache.hadoop.hdfs.TestRenameWhileOpen org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery org.apache.hadoop.hdfs.server.datanode.TestIncrementalBrVariations org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics org.apache.hadoop.hdfs.TestBlockReaderLocal org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl org.apache.hadoop.hdfs.TestGetBlocks org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade org.apache.hadoop.hdfs.server.datanode.TestStorageReport org.apache.hadoop.hdfs.TestDFSOutputStream org.apache.hadoop.hdfs.server.datanode.TestDataNodeInitStorage org.apache.hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks org.apache.hadoop.fs.loadGenerator.TestLoadGenerator org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean org.apache.hadoop.fs.viewfs.TestViewFsHdfs org.apache.hadoop.hdfs.TestParallelShortCircuitReadUnCached org.apache.hadoop.hdfs.server.namenode.TestCheckpoint org.apache.hadoop.hdfs.TestBlockMissingException org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.TestQuota org.apache.hadoop.fs.TestSymlinkHdfsDisable org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication org.apache.hadoop.fs.TestUrlStreamHandler org.apache.hadoop.hdfs.TestFileCreation org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode org.apache.hadoop.fs.TestHDFSFileContextMainOperations org.apache.hadoop.hdfs.TestSetrepDecreasing org.apache.hadoop.hdfs.TestFileCreationEmpty org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing org.apache.hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot org.apache.hadoop.hdfs.TestDatanodeDeath org.apache.hadoop.hdfs.TestParallelRead org.apache.hadoop.cli.TestAclCLI org.apache.hadoop.hdfs.TestFileStatus org.apache.hadoop.hdfs.TestRollingUpgrade org.apache.hadoop.hdfs.TestFileAppend3 org.apache.hadoop.hdfs.TestHFlush org.apache.hadoop.hdfs.TestIsMethodSupported org.apache.hadoop.hdfs.TestListFilesInDFS org.apache.hadoop.hdfs.TestPipelines org.apache.hadoop.hdfs.TestRollingUpgradeRollback org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache org.apache.hadoop.security.TestRefreshUserMappings org.apache.hadoop.security.TestPermissionSymlinks org.apache.hadoop.hdfs.server.datanode.TestTransferRbw org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename org.apache.hadoop.fs.TestGlobPaths org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction org.apache.hadoop.hdfs.server.namenode.TestMetaSave org.apache.hadoop.hdfs.TestDFSStartupVersions org.apache.hadoop.TestRefreshCallQueue org.apache.hadoop.hdfs.TestFileCreationClient org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks org.apache.hadoop.hdfs.TestLeaseRecovery org.apache.hadoop.hdfs.TestLocalDFS org.apache.hadoop.security.TestPermission org.apache.hadoop.hdfs.qjournal.TestNNWithQJM org.apache.hadoop.hdfs.web.TestHttpsFileSystem org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens org.apache.hadoop.hdfs.TestDFSRename org.apache.hadoop.hdfs.TestDFSAddressConfig org.apache.hadoop.hdfs.TestFetchImage org.apache.hadoop.hdfs.server.namenode.snapshot.TestFullPathNameWithSnapshot org.apache.hadoop.hdfs.server.namenode.TestAddBlock org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.server.namenode.TestEditLogRace org.apache.hadoop.hdfs.TestDFSClientExcludedNodes org.apache.hadoop.hdfs.TestSetrepIncreasing org.apache.hadoop.hdfs.TestLeaseRecovery2 org.apache.hadoop.net.TestNetworkTopology org.apache.hadoop.hdfs.server.namenode.TestNameNodeRpcServer org.apache.hadoop.hdfs.TestWriteRead org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer org.apache.hadoop.hdfs.TestMiniDFSCluster org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup org.apache.hadoop.hdfs.TestFileCorruption org.apache.hadoop.hdfs.TestHDFSFileSystemContract org.apache.hadoop.hdfs.TestParallelShortCircuitRead org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot org.apache.hadoop.hdfs.TestReadWhileWriting org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount org.apache.hadoop.hdfs.TestDFSFinalize org.apache.hadoop.hdfs.TestMultiThreadedHflush org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.server.namenode.TestCreateEditsLog org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache org.apache.hadoop.hdfs.TestDFSUpgrade org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery org.apache.hadoop.hdfs.TestDFSMkdirs org.apache.hadoop.hdfs.TestDatanodeConfig org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion org.apache.hadoop.hdfs.server.namenode.TestStartup org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean org.apache.hadoop.hdfs.TestEncryptedTransfer org.apache.hadoop.hdfs.TestReplication org.apache.hadoop.hdfs.TestConnCache org.apache.hadoop.hdfs.server.namenode.TestHostsFiles org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract org.apache.hadoop.hdfs.server.datanode.TestIncrementalBlockReports org.apache.hadoop.hdfs.TestRead org.apache.hadoop.fs.TestResolveHdfsSymlink org.apache.hadoop.hdfs.TestDFSShellGenericOptions org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy org.apache.hadoop.hdfs.web.TestWebHdfsWithAuthenticationFilter org.apache.hadoop.hdfs.TestSafeMode org.apache.hadoop.hdfs.web.TestWebHDFS org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead org.apache.hadoop.hdfs.TestDFSClientRetries org.apache.hadoop.hdfs.TestDisableConnCache org.apache.hadoop.fs.TestFcHdfsSetUMask org.apache.hadoop.hdfs.server.namenode.TestAddBlockRetry org.apache.hadoop.fs.TestSymlinkHdfsFileSystem org.apache.hadoop.fs.TestSymlinkHdfsFileContext org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl org.apache.hadoop.hdfs.TestFileAppend4 org.apache.hadoop.fs.viewfs.TestViewFsAtHdfsRoot org.apache.hadoop.hdfs.TestModTime org.apache.hadoop.hdfs.TestLease org.apache.hadoop.hdfs.TestDataTransferKeepalive org.apache.hadoop.hdfs.TestLargeBlock org.apache.hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps org.apache.hadoop.hdfs.TestClientBlockVerification org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead org.apache.hadoop.hdfs.TestSetTimes org.apache.hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot org.apache.hadoop.fs.viewfs.TestViewFsFileStatusHdfs org.apache.hadoop.hdfs.TestDatanodeBlockScanner org.apache.hadoop.hdfs.web.TestWebHDFSAcl org.apache.hadoop.hdfs.TestPersistBlocks org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport org.apache.hadoop.hdfs.server.namenode.TestINodeFile org.apache.hadoop.hdfs.server.namenode.TestFileLimit org.apache.hadoop.fs.TestFcHdfsPermission org.apache.hadoop.hdfs.TestParallelUnixDomainRead org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart org.apache.hadoop.hdfs.server.namenode.TestFsck org.apache.hadoop.cli.TestCacheAdminCLI org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure org.apache.hadoop.hdfs.server.namenode.TestAllowFormat org.apache.hadoop.hdfs.security.token.block.TestBlockToken org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.TestPread org.apache.hadoop.hdfs.TestBlocksScheduledCounter org.apache.hadoop.hdfs.TestFileAppendRestart org.apache.hadoop.hdfs.server.namenode.TestLeaseManager org.apache.hadoop.hdfs.TestWriteConfigurationToDFS org.apache.hadoop.hdfs.server.namenode.TestEditLog +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6807//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6807//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6807//console This message is automatically generated.
        Hide
        Kihwal Lee added a comment -

        The proposed patch sets clientNamenodeAddress in NameNode from the defaultFS setting before the config is altered by initializeGenericKeys. This address is used for pupulating the NN address filed in webhdfs redirects.

        Show
        Kihwal Lee added a comment - The proposed patch sets clientNamenodeAddress in NameNode from the defaultFS setting before the config is altered by initializeGenericKeys . This address is used for pupulating the NN address filed in webhdfs redirects.

          People

          • Assignee:
            Kihwal Lee
            Reporter:
            Kihwal Lee
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development