Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10225

DataNode hot swap drives should disallow storage type changes.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The current hot swap code only differentiate data dirs by their paths. People might want to change the types of certain data dirs from the default value in an existing cluster.

      1. HDFS-10225.000.patch
        17 kB
        Lei (Eddy) Xu
      2. HDFS-10225.001.patch
        17 kB
        Lei (Eddy) Xu
      3. HDFS-10225.002.patch
        4 kB
        Lei (Eddy) Xu
      4. HDFS-10225.003.patch
        4 kB
        Lei (Eddy) Xu

        Activity

        Hide
        xiaochen Xiao Chen added a comment -

        I forgot to mention, failed tests looked unrelated.

        Show
        xiaochen Xiao Chen added a comment - I forgot to mention, failed tests looked unrelated.
        Hide
        xiaochen Xiao Chen added a comment -

        Committed to branch-2 and branch-2.8, compiled and passed the test before pushing. Thanks!

        Show
        xiaochen Xiao Chen added a comment - Committed to branch-2 and branch-2.8, compiled and passed the test before pushing. Thanks!
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Thanks Xiao Chen for the reviews and commit. It should be OK to backport to branch-2.
        Thanks for the views Colin P. McCabe Arpit Agarwal.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Thanks Xiao Chen for the reviews and commit. It should be OK to backport to branch-2. Thanks for the views Colin P. McCabe Arpit Agarwal .
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10134 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10134/)
        HDFS-10225. DataNode hot swap drives should disallow storage type (xiao: rev 132deb4cacc413a85a6af2f390ec79a76c91961c)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10134 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10134/ ) HDFS-10225 . DataNode hot swap drives should disallow storage type (xiao: rev 132deb4cacc413a85a6af2f390ec79a76c91961c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        xiaochen Xiao Chen added a comment - - edited

        Committed to trunk, thanks Lei (Eddy) Xu for the contribution and Colin P. McCabe , Arpit Agarwal for reviews!

        Lei (Eddy) Xu, do you want to target this for earlier branches? This is incompatible in behavior, but changing storage type doesn't work currently. So I think it's okay to backport if there's a need. (Not sure if I should check the 'Imcompatible' flag here or not.

        Show
        xiaochen Xiao Chen added a comment - - edited Committed to trunk, thanks Lei (Eddy) Xu for the contribution and Colin P. McCabe , Arpit Agarwal for reviews! Lei (Eddy) Xu , do you want to target this for earlier branches? This is incompatible in behavior, but changing storage type doesn't work currently. So I think it's okay to backport if there's a need. (Not sure if I should check the 'Imcompatible' flag here or not.
        Hide
        xiaochen Xiao Chen added a comment -

        +1 on patch 3, will commit shortly.

        Show
        xiaochen Xiao Chen added a comment - +1 on patch 3, will commit shortly.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 17s trunk passed
        +1 compile 0m 51s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 0m 58s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 57s trunk passed
        +1 javadoc 0m 59s trunk passed
        +1 mvninstall 0m 58s the patch passed
        +1 compile 0m 47s the patch passed
        +1 javac 0m 47s the patch passed
        +1 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 190 unchanged - 1 fixed = 190 total (was 191)
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 0s the patch passed
        +1 javadoc 0m 56s the patch passed
        -1 unit 74m 33s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 25s The patch does not generate ASF License warnings.
        96m 37s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
          hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.server.blockmanagement.TestBlockManager
        Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818938/HDFS-10225.003.patch
        JIRA Issue HDFS-10225
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux cec7c92ed488 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e340064
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16126/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16126/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16126/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 17s trunk passed +1 compile 0m 51s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 57s trunk passed +1 javadoc 0m 59s trunk passed +1 mvninstall 0m 58s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed +1 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 190 unchanged - 1 fixed = 190 total (was 191) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 0s the patch passed +1 javadoc 0m 56s the patch passed -1 unit 74m 33s hadoop-hdfs in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 96m 37s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.blockmanagement.TestBlockManager Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818938/HDFS-10225.003.patch JIRA Issue HDFS-10225 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cec7c92ed488 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e340064 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16126/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16126/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16126/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 44s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 0m 53s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 41s trunk passed
        +1 javadoc 0m 58s trunk passed
        +1 mvninstall 0m 50s the patch passed
        +1 compile 0m 55s the patch passed
        +1 javac 0m 55s the patch passed
        +1 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 190 unchanged - 1 fixed = 190 total (was 191)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 57s the patch passed
        +1 javadoc 0m 58s the patch passed
        -1 unit 62m 49s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        83m 45s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818938/HDFS-10225.003.patch
        JIRA Issue HDFS-10225
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 80d6a4c449b8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 38128ba
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16119/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16119/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16119/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 44s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 58s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 55s the patch passed +1 javac 0m 55s the patch passed +1 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 190 unchanged - 1 fixed = 190 total (was 191) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 58s the patch passed -1 unit 62m 49s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 83m 45s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818938/HDFS-10225.003.patch JIRA Issue HDFS-10225 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 80d6a4c449b8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 38128ba Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16119/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16119/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16119/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Thank you for the work, Eddy. +1 pending jenkins.

        Show
        xiaochen Xiao Chen added a comment - Thank you for the work, Eddy. +1 pending jenkins.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Good catch, Xiao Chen

        Any reason not to put the newly added code after the locations.isEmpty() check?

        Done

        Javadoc @throws on parseChangedVolumes needs to be updated.

        Done.

        Thanks a lot for the review.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Good catch, Xiao Chen Any reason not to put the newly added code after the locations.isEmpty() check? Done Javadoc @throws on parseChangedVolumes needs to be updated. Done. Thanks a lot for the review.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Lei (Eddy) Xu for the new rev.

        Nits:

        • Any reason not to put the newly added code after the locations.isEmpty() check?
        • Javadoc @throws on parseChangedVolumes needs to be updated.
        Show
        xiaochen Xiao Chen added a comment - Thanks Lei (Eddy) Xu for the new rev. Nits: Any reason not to put the newly added code after the locations.isEmpty() check? Javadoc @throws on parseChangedVolumes needs to be updated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hi, Colin P. McCabe and Arpit Agarwal Thanks for the suggestions. I modified the patch to disallow changing storage type in the DataNode runtime.

        So that in order to achieve the purpose, user should either restart DN or, remove the disk and add it back with a new storage type.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hi, Colin P. McCabe and Arpit Agarwal Thanks for the suggestions. I modified the patch to disallow changing storage type in the DataNode runtime. So that in order to achieve the purpose, user should either restart DN or, remove the disk and add it back with a new storage type.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Hi Lei (Eddy) Xu, I prefer we disallow changing the storage type of a directory without a restart. Is there a strong use case for doing this without restarts? Thanks.

        Show
        arpitagarwal Arpit Agarwal added a comment - Hi Lei (Eddy) Xu , I prefer we disallow changing the storage type of a directory without a restart. Is there a strong use case for doing this without restarts? Thanks.
        Hide
        cmccabe Colin P. McCabe added a comment -

        It seems cleaner to remove the volume and re-add it with the new storage type, rather than mutating the existing volume. Otherwise, we need to think about synchronization here around every use of storage type.

        There are also some changes that look unrelated, like moving the call to parseChangedVolunes up several lines.

        Show
        cmccabe Colin P. McCabe added a comment - It seems cleaner to remove the volume and re-add it with the new storage type, rather than mutating the existing volume. Otherwise, we need to think about synchronization here around every use of storage type. There are also some changes that look unrelated, like moving the call to parseChangedVolunes up several lines.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Updated the patch to address checkstyle warnings.

        The warning in DataNode.java has not been changed. If it is a concern, can we file a new JIRA for fix ChangedVolumes together.

        The ASF license warning is not relevant.

        Ran all tests, and for all the tests passed on trunk passed locally.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Updated the patch to address checkstyle warnings. The warning in DataNode.java has not been changed. If it is a concern, can we file a new JIRA for fix ChangedVolumes together. The ASF license warning is not relevant. Ran all tests, and for all the tests passed on trunk passed locally.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        +1 mvninstall 8m 53s trunk passed
        +1 compile 1m 14s trunk passed with JDK v1.8.0_74
        +1 compile 0m 58s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 1m 6s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 2m 34s trunk passed
        +1 javadoc 1m 45s trunk passed with JDK v1.8.0_74
        +1 javadoc 2m 21s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 59s the patch passed
        +1 compile 1m 4s the patch passed with JDK v1.8.0_74
        +1 javac 1m 4s the patch passed
        +1 compile 0m 54s the patch passed with JDK v1.7.0_95
        +1 javac 0m 54s the patch passed
        -1 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: patch generated 2 new + 333 unchanged - 0 fixed = 335 total (was 333)
        +1 mvnsite 1m 9s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 38s the patch passed
        +1 javadoc 1m 36s the patch passed with JDK v1.8.0_74
        +1 javadoc 2m 30s the patch passed with JDK v1.7.0_95
        -1 unit 109m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 93m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        -1 asflicense 0m 30s Patch generated 1 ASF License warnings.
        238m 14s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.cli.TestHDFSCLI
          hadoop.hdfs.server.namenode.ha.TestEditLogTailer
          hadoop.hdfs.shortcircuit.TestShortCircuitCache
          hadoop.hdfs.server.datanode.TestDataNodeMetrics
          hadoop.hdfs.security.TestDelegationTokenForProxyUser
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.namenode.TestEditLog
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          hadoop.hdfs.TestReconstructStripedFile
          hadoop.hdfs.qjournal.TestSecureNNWithQJM
          hadoop.hdfs.TestEncryptionZonesWithKMS
          hadoop.hdfs.server.datanode.TestDirectoryScanner
        JDK v1.7.0_95 Failed junit tests hadoop.cli.TestHDFSCLI
          hadoop.hdfs.security.TestDelegationTokenForProxyUser
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.server.datanode.TestDirectoryScanner



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795742/HDFS-10225.000.patch
        JIRA Issue HDFS-10225
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 3f5650469329 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 948b758
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14968/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14968/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 8m 53s trunk passed +1 compile 1m 14s trunk passed with JDK v1.8.0_74 +1 compile 0m 58s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 2m 34s trunk passed +1 javadoc 1m 45s trunk passed with JDK v1.8.0_74 +1 javadoc 2m 21s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 59s the patch passed +1 compile 1m 4s the patch passed with JDK v1.8.0_74 +1 javac 1m 4s the patch passed +1 compile 0m 54s the patch passed with JDK v1.7.0_95 +1 javac 0m 54s the patch passed -1 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: patch generated 2 new + 333 unchanged - 0 fixed = 335 total (was 333) +1 mvnsite 1m 9s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 38s the patch passed +1 javadoc 1m 36s the patch passed with JDK v1.8.0_74 +1 javadoc 2m 30s the patch passed with JDK v1.7.0_95 -1 unit 109m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 93m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 30s Patch generated 1 ASF License warnings. 238m 14s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.TestReconstructStripedFile   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_95 Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795742/HDFS-10225.000.patch JIRA Issue HDFS-10225 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3f5650469329 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 948b758 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14968/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14968/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14968/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        +1 mvninstall 6m 43s trunk passed
        +1 compile 0m 42s trunk passed with JDK v1.8.0_74
        +1 compile 0m 42s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 2m 11s trunk passed
        +1 javadoc 1m 4s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 46s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 45s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.8.0_74
        +1 javac 0m 38s the patch passed
        +1 compile 0m 39s the patch passed with JDK v1.7.0_95
        +1 javac 0m 39s the patch passed
        -1 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: patch generated 2 new + 332 unchanged - 0 fixed = 334 total (was 332)
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 13s the patch passed
        +1 javadoc 1m 5s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 46s the patch passed with JDK v1.7.0_95
        -1 unit 55m 50s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 55m 12s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        -1 asflicense 0m 22s Patch generated 1 ASF License warnings.
        136m 48s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.hdfs.TestHFlush
          hadoop.cli.TestHDFSCLI
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
          hadoop.hdfs.TestHFlush
          hadoop.cli.TestHDFSCLI



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795742/HDFS-10225.000.patch
        JIRA Issue HDFS-10225
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9fd31ef7b903 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 948b758
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14971/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14971/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 6m 43s trunk passed +1 compile 0m 42s trunk passed with JDK v1.8.0_74 +1 compile 0m 42s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 2m 11s trunk passed +1 javadoc 1m 4s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 46s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 45s the patch passed +1 compile 0m 38s the patch passed with JDK v1.8.0_74 +1 javac 0m 38s the patch passed +1 compile 0m 39s the patch passed with JDK v1.7.0_95 +1 javac 0m 39s the patch passed -1 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: patch generated 2 new + 332 unchanged - 0 fixed = 334 total (was 332) +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 13s the patch passed +1 javadoc 1m 5s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_95 -1 unit 55m 50s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 55m 12s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 22s Patch generated 1 ASF License warnings. 136m 48s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.cli.TestHDFSCLI JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.TestHFlush   hadoop.cli.TestHDFSCLI Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795742/HDFS-10225.000.patch JIRA Issue HDFS-10225 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9fd31ef7b903 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 948b758 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14971/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14971/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14971/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Upload an initial patch to set FsVolumeSpi#storageType directly from reconfig code.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Upload an initial patch to set FsVolumeSpi#storageType directly from reconfig code.

          People

          • Assignee:
            eddyxu Lei (Eddy) Xu
            Reporter:
            eddyxu Lei (Eddy) Xu
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development