Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9142

Separating Configuration object for namenode(s) in MiniDFSCluster

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      When setting up simpleHAFederatedTopology in MiniDFSCluster, each Namenode should have its own configuration object, and the configuration should have "dfs.namenode.http-address-<nameservice>-<namenodeid>" set up correctly for all <nameserivce, namenodeid> pair

      1. HDFS-9142-branch-2.v1.patch
        9 kB
        Siqi Li
      2. HDFS-9142.v7.patch
        10 kB
        Siqi Li
      3. HDFS-9142.v6.patch
        10 kB
        Siqi Li
      4. HDFS-9142.v5.patch
        8 kB
        Siqi Li
      5. HDFS-9142.v4.patch
        10 kB
        Siqi Li
      6. HDFS-9142.v3.patch
        11 kB
        Siqi Li
      7. HDFS-9142.v2.patch
        10 kB
        Siqi Li
      8. HDFS-9142.v1.patch
        10 kB
        Siqi Li

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2416 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2416/)
        HDFS-9142. Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2416 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2416/ ) HDFS-9142 . Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2450 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2450/)
        HDFS-9142. Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2450 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2450/ ) HDFS-9142 . Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/)
        HDFS-9142. Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/ ) HDFS-9142 . Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/)
        HDFS-9142. Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/ ) HDFS-9142 . Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #516 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/516/)
        HDFS-9142. Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #516 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/516/ ) HDFS-9142 . Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1242 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1242/)
        HDFS-9142. Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1242 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1242/ ) HDFS-9142 . Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8602 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8602/)
        HDFS-9142. Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8602 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8602/ ) HDFS-9142 . Separating Configuration object for namenode(s) in (mingma: rev de8efc65a455c10ae7280b5982c48f9aca84c9d4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        mingma Ming Ma added a comment -

        Thanks Mingliang Liu for the code review.

        Show
        mingma Ming Ma added a comment - Thanks Mingliang Liu for the code review.
        Hide
        mingma Ming Ma added a comment -

        Thanks Siqi Li for the contribution. I have committed it to branch-2.

        Show
        mingma Ming Ma added a comment - Thanks Siqi Li for the contribution. I have committed it to branch-2.
        Hide
        mingma Ming Ma added a comment -

        +1 on the latest patch. I have committed to trunk. However it doesn't apply to branch-2. Siqi Li can you provide a patch for branch-2?

        Show
        mingma Ming Ma added a comment - +1 on the latest patch. I have committed to trunk. However it doesn't apply to branch-2. Siqi Li can you provide a patch for branch-2?
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 7m 57s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 8m 0s There were no new javac warning messages.
        -1 release audit 0m 15s The applied patch generated 1 release audit warnings.
        +1 checkstyle 1m 28s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 4s Pre-build of native portion
        -1 hdfs tests 189m 49s Tests failed in hadoop-hdfs.
            213m 10s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.datanode.TestBpServiceActorScheduler



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12765652/HDFS-9142.v7.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / 0841940
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12870/artifact/patchprocess/patchReleaseAuditProblems.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12870/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12870/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12870/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 7m 57s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 8m 0s There were no new javac warning messages. -1 release audit 0m 15s The applied patch generated 1 release audit warnings. +1 checkstyle 1m 28s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 4s Pre-build of native portion -1 hdfs tests 189m 49s Tests failed in hadoop-hdfs.     213m 10s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.TestBpServiceActorScheduler Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765652/HDFS-9142.v7.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 0841940 Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12870/artifact/patchprocess/patchReleaseAuditProblems.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12870/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12870/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12870/console This message was automatically generated.
        Hide
        l201514 Siqi Li added a comment -

        ok. I have removed "System.out.println" in patch v7

        Show
        l201514 Siqi Li added a comment - ok. I have removed "System.out.println" in patch v7
        Hide
        mingma Ming Ma added a comment -

        Siqi Li, there are some debug statements like "System.out.println" in the latest patch. Can you update the patch? After that, we can track if it is submitted to https://builds.apache.org/job/PreCommit-HDFS-Build/.

        Show
        mingma Ming Ma added a comment - Siqi Li , there are some debug statements like "System.out.println" in the latest patch. Can you update the patch? After that, we can track if it is submitted to https://builds.apache.org/job/PreCommit-HDFS-Build/ .
        Hide
        l201514 Siqi Li added a comment -

        Ming Ma, it seems that patch v6 doesn't have a test report. Is there a way to start a test?

        Show
        l201514 Siqi Li added a comment - Ming Ma , it seems that patch v6 doesn't have a test report. Is there a way to start a test?
        Hide
        mingma Ming Ma added a comment -

        Thanks Siqi Li for identifying the issue. Having each namenode use its own Configuration should work.

        • Why is the removal of if (!federation && lastDefaultFileSystem != null)? That is still needed for non federated case? It might explain the test failure.
        • Some code can be shared in the two steps a) set up the nameservice related configs for each newly created NN. b) copy all NNs's nameservice related configs to the all NNs. In both case, it needs something like private static void copyKeys(Configuration srcConf, Configuration destConf, String nameserviceId, String nnId), where it copies RPC/HTTP/HTTPS addresses for the specific nameservice/nnid from some conf to another conf.
        Show
        mingma Ming Ma added a comment - Thanks Siqi Li for identifying the issue. Having each namenode use its own Configuration should work. Why is the removal of if (!federation && lastDefaultFileSystem != null) ? That is still needed for non federated case? It might explain the test failure. Some code can be shared in the two steps a) set up the nameservice related configs for each newly created NN. b) copy all NNs's nameservice related configs to the all NNs. In both case, it needs something like private static void copyKeys(Configuration srcConf, Configuration destConf, String nameserviceId, String nnId) , where it copies RPC/HTTP/HTTPS addresses for the specific nameservice/nnid from some conf to another conf.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 7m 43s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 48s There were no new javac warning messages.
        -1 release audit 0m 13s The applied patch generated 1 release audit warnings.
        +1 checkstyle 1m 23s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
        +1 findbugs 2m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 13s Pre-build of native portion
        -1 hdfs tests 186m 29s Tests failed in hadoop-hdfs.
            209m 16s  



        Reason Tests
        Failed unit tests hadoop.hdfs.tools.TestGetGroups
          hadoop.hdfs.TestHdfsAdmin
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          hadoop.hdfs.TestClientReportBadBlock
          hadoop.hdfs.TestSafeMode
          hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
          hadoop.hdfs.tools.TestDebugAdmin
          hadoop.hdfs.TestSetrepIncreasing
          hadoop.cli.TestErasureCodingCLI
          hadoop.hdfs.TestMultiThreadedHflush
          hadoop.hdfs.TestEncryptionZonesWithKMS
          hadoop.hdfs.tools.TestStoragePolicyCommands
          hadoop.hdfs.TestEncryptedTransfer
          hadoop.security.TestPermissionSymlinks
          hadoop.hdfs.TestDFSRollback
          hadoop.fs.TestUnbuffer
          hadoop.hdfs.TestQuota
          hadoop.hdfs.TestFileAppend2
          hadoop.hdfs.TestDFSClientRetries
          hadoop.security.TestRefreshUserMappings
          hadoop.hdfs.server.namenode.TestCheckpoint
          hadoop.hdfs.TestReadWhileWriting
          hadoop.hdfs.server.namenode.TestAuditLogs
          hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.TestDFSUpgrade
          hadoop.hdfs.TestGetBlocks
          hadoop.fs.permission.TestStickyBit
          hadoop.hdfs.TestLeaseRecovery2
          hadoop.cli.TestXAttrCLI
          hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.fs.TestGlobPaths
          hadoop.hdfs.TestDFSShell
          hadoop.hdfs.server.namenode.TestCacheDirectives
          hadoop.security.TestPermission
          hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
          hadoop.hdfs.server.namenode.TestINodeFile
          hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot
          hadoop.hdfs.TestSetrepDecreasing
          hadoop.hdfs.TestDFSFinalize
          hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
          hadoop.hdfs.TestDFSStorageStateRecovery
          hadoop.hdfs.TestDisableConnCache
          hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
          hadoop.hdfs.TestRestartDFS
          hadoop.cli.TestHDFSCLI
          hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.TestFileCreation
          hadoop.cli.TestAclCLI
          hadoop.hdfs.TestDFSPermission
          hadoop.cli.TestDeleteCLI
          hadoop.cli.TestCryptoAdminCLI
          hadoop.hdfs.TestRollingUpgradeRollback
          hadoop.hdfs.server.namenode.TestFileContextAcl
          hadoop.hdfs.server.namenode.TestNameNodeXAttr
          hadoop.hdfs.TestRollingUpgrade
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.TestDFSStartupVersions
          hadoop.hdfs.TestFetchImage
          hadoop.cli.TestCacheAdminCLI
          hadoop.hdfs.web.TestWebHDFSXAttr
          hadoop.hdfs.server.namenode.TestFsck
          hadoop.hdfs.TestSnapshotCommands
          hadoop.hdfs.TestClose
          hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
          hadoop.hdfs.TestFileStatus
          hadoop.hdfs.TestFsShellPermission
          hadoop.fs.loadGenerator.TestLoadGenerator
          hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
          hadoop.hdfs.server.namenode.TestFileContextXAttr
          hadoop.hdfs.server.namenode.TestStorageRestore



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764818/HDFS-9142.v4.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / a68b6eb
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12774/artifact/patchprocess/patchReleaseAuditProblems.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12774/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12774/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12774/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 7m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 48s There were no new javac warning messages. -1 release audit 0m 13s The applied patch generated 1 release audit warnings. +1 checkstyle 1m 23s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. +1 findbugs 2m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 13s Pre-build of native portion -1 hdfs tests 186m 29s Tests failed in hadoop-hdfs.     209m 16s   Reason Tests Failed unit tests hadoop.hdfs.tools.TestGetGroups   hadoop.hdfs.TestHdfsAdmin   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.TestClientReportBadBlock   hadoop.hdfs.TestSafeMode   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.tools.TestDebugAdmin   hadoop.hdfs.TestSetrepIncreasing   hadoop.cli.TestErasureCodingCLI   hadoop.hdfs.TestMultiThreadedHflush   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.tools.TestStoragePolicyCommands   hadoop.hdfs.TestEncryptedTransfer   hadoop.security.TestPermissionSymlinks   hadoop.hdfs.TestDFSRollback   hadoop.fs.TestUnbuffer   hadoop.hdfs.TestQuota   hadoop.hdfs.TestFileAppend2   hadoop.hdfs.TestDFSClientRetries   hadoop.security.TestRefreshUserMappings   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.TestReadWhileWriting   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestDFSUpgrade   hadoop.hdfs.TestGetBlocks   hadoop.fs.permission.TestStickyBit   hadoop.hdfs.TestLeaseRecovery2   hadoop.cli.TestXAttrCLI   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.fs.TestGlobPaths   hadoop.hdfs.TestDFSShell   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.security.TestPermission   hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot   hadoop.hdfs.TestSetrepDecreasing   hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot   hadoop.hdfs.TestDFSStorageStateRecovery   hadoop.hdfs.TestDisableConnCache   hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens   hadoop.hdfs.TestRestartDFS   hadoop.cli.TestHDFSCLI   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.TestFileCreation   hadoop.cli.TestAclCLI   hadoop.hdfs.TestDFSPermission   hadoop.cli.TestDeleteCLI   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.TestRollingUpgradeRollback   hadoop.hdfs.server.namenode.TestFileContextAcl   hadoop.hdfs.server.namenode.TestNameNodeXAttr   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.TestDFSStartupVersions   hadoop.hdfs.TestFetchImage   hadoop.cli.TestCacheAdminCLI   hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.TestSnapshotCommands   hadoop.hdfs.TestClose   hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing   hadoop.hdfs.TestFileStatus   hadoop.hdfs.TestFsShellPermission   hadoop.fs.loadGenerator.TestLoadGenerator   hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade   hadoop.hdfs.server.namenode.TestFileContextXAttr   hadoop.hdfs.server.namenode.TestStorageRestore Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764818/HDFS-9142.v4.patch Optional Tests javac unit findbugs checkstyle git revision trunk / a68b6eb Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12774/artifact/patchprocess/patchReleaseAuditProblems.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12774/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12774/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12774/console This message was automatically generated.
        Hide
        l201514 Siqi Li added a comment -

        Test failures seem to be unrelated

        Show
        l201514 Siqi Li added a comment - Test failures seem to be unrelated
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 8m 7s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
        +1 javac 10m 14s There were no new javac warning messages.
        +1 release audit 0m 28s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 32s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 56s mvn install still works.
        +1 eclipse:eclipse 0m 42s The patch built with eclipse:eclipse.
        +1 findbugs 3m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 46s Pre-build of native portion
        -1 hdfs tests 126m 22s Tests failed in hadoop-hdfs.
            154m 31s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
        Timed out tests org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12762437/HDFS-9142.v3.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / 83e65c5
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12684/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12684/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12684/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 8m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 10m 14s There were no new javac warning messages. +1 release audit 0m 28s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 32s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 56s mvn install still works. +1 eclipse:eclipse 0m 42s The patch built with eclipse:eclipse. +1 findbugs 3m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 46s Pre-build of native portion -1 hdfs tests 126m 22s Tests failed in hadoop-hdfs.     154m 31s   Reason Tests Failed unit tests hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer Timed out tests org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12762437/HDFS-9142.v3.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 83e65c5 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12684/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12684/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12684/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 8m 2s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 56s There were no new javac warning messages.
        +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 25s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 18s Pre-build of native portion
        -1 hdfs tests 196m 55s Tests failed in hadoop-hdfs.
            220m 41s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
          hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          hadoop.hdfs.server.namenode.TestEditLogAutoroll
          hadoop.hdfs.TestCrcCorruption
          hadoop.hdfs.server.namenode.ha.TestHAFsck



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12762253/HDFS-9142.v2.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / d1b9b85
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12670/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12670/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12670/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 8m 2s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 56s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 25s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 18s Pre-build of native portion -1 hdfs tests 196m 55s Tests failed in hadoop-hdfs.     220m 41s   Reason Tests Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.namenode.TestEditLogAutoroll   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.namenode.ha.TestHAFsck Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12762253/HDFS-9142.v2.patch Optional Tests javac unit findbugs checkstyle git revision trunk / d1b9b85 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12670/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12670/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12670/console This message was automatically generated.
        Hide
        l201514 Siqi Li added a comment -

        Thanks for the review, I have updated the javadoc in patch v2

        Show
        l201514 Siqi Li added a comment - Thanks for the review, I have updated the javadoc in patch v2
        Hide
        liuml07 Mingliang Liu added a comment -

        I think the patch makes sense. One minor comment: can you add @return javadoc to the configureNameNodes method as its name is not straightforward to the newly added returning behavior.

        Show
        liuml07 Mingliang Liu added a comment - I think the patch makes sense. One minor comment: can you add @return javadoc to the configureNameNodes method as its name is not straightforward to the newly added returning behavior.

          People

          • Assignee:
            l201514 Siqi Li
            Reporter:
            l201514 Siqi Li
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development