Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9715

Check storage ID uniqueness on datanode startup

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We should fix this to check storage ID uniqueness on datanode startup. If someone has manually edited the storage ID files, or if they have duplicated a directory (or re-added an old disk) they could end up with a duplicate storage ID and not realize it.

      The HDFS-7575 fix does generate storage UUID for each storage, but not checks the uniqueness of these UUIDs.

      1. HDFS-9715.00.patch
        5 kB
        Lei (Eddy) Xu
      2. HDFS-9715.01.patch
        6 kB
        Lei (Eddy) Xu
      3. HDFS-9715.02.patch
        6 kB
        Lei (Eddy) Xu
      4. HDFS-9715.03.patch
        10 kB
        Lei (Eddy) Xu

        Activity

        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Add a patch to check the existence of StorageUUID in FsDatasetImpl. It raises IOE if a storage with the same storage UUID already exists.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Add a patch to check the existence of StorageUUID in FsDatasetImpl . It raises IOE if a storage with the same storage UUID already exists.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 3s trunk passed
        +1 compile 0m 42s trunk passed with JDK v1.8.0_66
        +1 compile 0m 43s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 0m 55s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 2m 1s trunk passed
        +1 javadoc 1m 11s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 53s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 48s the patch passed
        +1 compile 0m 40s the patch passed with JDK v1.8.0_66
        +1 javac 0m 40s the patch passed
        +1 compile 0m 41s the patch passed with JDK v1.7.0_91
        +1 javac 0m 41s the patch passed
        -1 checkstyle 0m 19s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 119 unchanged - 0 fixed = 120 total (was 119)
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 11s the patch passed
        +1 javadoc 1m 4s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 45s the patch passed with JDK v1.7.0_91
        -1 unit 54m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 51m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        132m 31s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSFinalize
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.datanode.TestDataNodeMetrics
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSFinalize
          hadoop.hdfs.server.datanode.TestBlockScanner



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784771/HDFS-9715.00.patch
        JIRA Issue HDFS-9715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0e7e1da87839 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8a91109
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14273/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 77MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14273/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 3s trunk passed +1 compile 0m 42s trunk passed with JDK v1.8.0_66 +1 compile 0m 43s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 2m 1s trunk passed +1 javadoc 1m 11s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 53s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 48s the patch passed +1 compile 0m 40s the patch passed with JDK v1.8.0_66 +1 javac 0m 40s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_91 +1 javac 0m 41s the patch passed -1 checkstyle 0m 19s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 119 unchanged - 0 fixed = 120 total (was 119) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 11s the patch passed +1 javadoc 1m 4s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 45s the patch passed with JDK v1.7.0_91 -1 unit 54m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 51m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 132m 31s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.datanode.TestDataNodeMetrics JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.datanode.TestBlockScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784771/HDFS-9715.00.patch JIRA Issue HDFS-9715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0e7e1da87839 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8a91109 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14273/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14273/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14273/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Patch looks good.
        Nits:
        1. IMO, activeVolume(..) can be renamed to activateVolume(..) ?
        2. LOG.info(error_msg); could be LOG.error(..) ?

        +1, once these are done.

        Show
        vinayrpet Vinayakumar B added a comment - Patch looks good. Nits: 1. IMO, activeVolume(..) can be renamed to activateVolume(..) ? 2. LOG.info(error_msg); could be LOG.error(..) ? +1, once these are done.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hey, Vinayakumar B Thanks a lot for the review.

        I updated to patch to address your reviews and to fix checkstyle warnings. It also fixes the test failure from TestDFSFinalize.

        The rest of failures seems not relevant and passed on my laptop.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hey, Vinayakumar B Thanks a lot for the review. I updated to patch to address your reviews and to fix checkstyle warnings. It also fixes the test failure from TestDFSFinalize . The rest of failures seems not relevant and passed on my laptop.
        Hide
        andrew.wang Andrew Wang added a comment -

        Smallest nit ever, the javadoc for the activateVolume method still says "Active" rather than "Activate". Otherwise I'm +1 too, thanks Eddy.

        Show
        andrew.wang Andrew Wang added a comment - Smallest nit ever, the javadoc for the activateVolume method still says "Active" rather than "Activate". Otherwise I'm +1 too, thanks Eddy.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Thanks for the reviews, Andrew Wang

        Updated the patch to address your comment

        Show
        eddyxu Lei (Eddy) Xu added a comment - Thanks for the reviews, Andrew Wang Updated the patch to address your comment
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 6m 34s trunk passed
        +1 compile 0m 39s trunk passed with JDK v1.8.0_66
        +1 compile 0m 40s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 49s trunk passed
        +1 javadoc 1m 2s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 45s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 0m 44s the patch passed
        +1 compile 0m 36s the patch passed with JDK v1.8.0_66
        +1 javac 0m 36s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.7.0_91
        +1 javac 0m 38s the patch passed
        +1 checkstyle 0m 20s the patch passed
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 2s the patch passed
        +1 javadoc 1m 2s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 45s the patch passed with JDK v1.7.0_91
        -1 unit 54m 39s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 55m 54s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        135m 43s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSRollback
          hadoop.hdfs.TestDFSUpgradeFromImage
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSRollback



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785643/HDFS-9715.02.patch
        JIRA Issue HDFS-9715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c580289dd555 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ed55950
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14330/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 77MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14330/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 34s trunk passed +1 compile 0m 39s trunk passed with JDK v1.8.0_66 +1 compile 0m 40s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 49s trunk passed +1 javadoc 1m 2s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 45s trunk passed with JDK v1.7.0_91 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 44s the patch passed +1 compile 0m 36s the patch passed with JDK v1.8.0_66 +1 javac 0m 36s the patch passed +1 compile 0m 38s the patch passed with JDK v1.7.0_91 +1 javac 0m 38s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 2s the patch passed +1 javadoc 1m 2s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 45s the patch passed with JDK v1.7.0_91 -1 unit 54m 39s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 55m 54s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 135m 43s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSRollback   hadoop.hdfs.TestDFSUpgradeFromImage JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSRollback Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785643/HDFS-9715.02.patch JIRA Issue HDFS-9715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c580289dd555 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ed55950 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14330/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14330/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14330/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        TestDFSRollback is failing because it expects, all data dirs to have same VERSION file, but it will not after changing UpgradeUtilities.java.

        So its better to exclude only storageId while comparing the props between VERSION files across data dirs.

        Show
        vinayrpet Vinayakumar B added a comment - TestDFSRollback is failing because it expects, all data dirs to have same VERSION file, but it will not after changing UpgradeUtilities.java . So its better to exclude only storageId while comparing the props between VERSION files across data dirs.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hey, Vinayakumar B

        Yes, you are right. I updated the patch to address the TestDFSRollback. Will much appreciate to have another review.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hey, Vinayakumar B Yes, you are right. I updated the patch to address the TestDFSRollback . Will much appreciate to have another review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 6m 59s trunk passed
        +1 compile 0m 43s trunk passed with JDK v1.8.0_66
        +1 compile 0m 39s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 55s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 2m 14s trunk passed
        +1 javadoc 1m 8s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 54s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 45s the patch passed with JDK v1.8.0_66
        +1 javac 0m 45s the patch passed
        +1 compile 0m 44s the patch passed with JDK v1.7.0_91
        +1 javac 0m 44s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 15s the patch passed
        +1 javadoc 1m 9s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 53s the patch passed with JDK v1.7.0_91
        -1 unit 73m 57s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 61m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 29s Patch does not generate ASF License warnings.
        163m 0s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.qjournal.client.TestQuorumJournalManager
        JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785866/HDFS-9715.03.patch
        JIRA Issue HDFS-9715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0689cb665a7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4ae543f
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14350/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14350/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 59s trunk passed +1 compile 0m 43s trunk passed with JDK v1.8.0_66 +1 compile 0m 39s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 2m 14s trunk passed +1 javadoc 1m 8s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 54s trunk passed with JDK v1.7.0_91 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 46s the patch passed +1 compile 0m 45s the patch passed with JDK v1.8.0_66 +1 javac 0m 45s the patch passed +1 compile 0m 44s the patch passed with JDK v1.7.0_91 +1 javac 0m 44s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 15s the patch passed +1 javadoc 1m 9s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 53s the patch passed with JDK v1.7.0_91 -1 unit 73m 57s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 61m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 29s Patch does not generate ASF License warnings. 163m 0s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestFileAppend   hadoop.hdfs.qjournal.client.TestQuorumJournalManager JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785866/HDFS-9715.03.patch JIRA Issue HDFS-9715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0689cb665a7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4ae543f Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14350/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14350/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14350/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk, branch-2 and branch-2.8.
        Thanks for the contribution Lei (Eddy) Xu,
        Thanks Andrew Wang for review.

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk, branch-2 and branch-2.8. Thanks for the contribution Lei (Eddy) Xu , Thanks Andrew Wang for review.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9234 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9234/)
        HDFS-9715. Check storage ID uniqueness on datanode startup (Contributed (vinayakumarb: rev 04375756a5ed6e907ee7548469c2c508aebbafb7)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9234 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9234/ ) HDFS-9715 . Check storage ID uniqueness on datanode startup (Contributed (vinayakumarb: rev 04375756a5ed6e907ee7548469c2c508aebbafb7) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/UpgradeUtilities.java
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Thanks a lot for the reviews, Vinayakumar B and Andrew Wang

        Show
        eddyxu Lei (Eddy) Xu added a comment - Thanks a lot for the reviews, Vinayakumar B and Andrew Wang

          People

          • Assignee:
            eddyxu Lei (Eddy) Xu
            Reporter:
            eddyxu Lei (Eddy) Xu
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development