Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7833

DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      DataNode reconfiguration never recalculates FsDatasetImpl#validVolsRequired. This may cause incorrect behavior of the dfs.datanode.failed.volumes.tolerated property if reconfiguration causes the DataNode to run with a different total number of volumes.

      1. HDFS-7833.000.patch
        6 kB
        Lei (Eddy) Xu
      2. HDFS-7833.001.patch
        7 kB
        Lei (Eddy) Xu
      3. HDFS-7833.002.patch
        7 kB
        Lei (Eddy) Xu
      4. HDFS-7833.003.patch
        7 kB
        Lei (Eddy) Xu

        Activity

        Hide
        cnauroth Chris Nauroth added a comment -

        This is a repeat of the comment I mentioned on HDFS-7830. Thank you to Lei (Eddy) Xu for volunteering to take assignment of the issue.

        Another potential problem that I've noticed in the DataNode reconfiguration code is that it never recalculates FsDatasetImpl#validVolsRequired. This is a final variable calculated as (# volumes configured) - (# volume failures tolerated):

            this.validVolsRequired = volsConfigured - volFailuresTolerated;
        

        If this variable is not updated for DataNode reconfigurations, then it could lead to some unexpected situations. For example:

        1. DataNode starts running with 6 volumes (all healthy) and dfs.datanode.failed.volumes.tolerated set to 2.
        2. FsDatasetImpl#validVolsRequired is set to 6 - 2 = 4.
        3. DataNode is reconfigured to run with 8 volumes (all still healthy).
        4. Now 3 volumes fail. The admin would expect the DataNode to abort, but there are 8 - 3 = 5 good volumes left, and FsDatasetImpl#validVolsRequired is still 4, so FsDatasetImpl#hasEnoughResource returns true.
        Show
        cnauroth Chris Nauroth added a comment - This is a repeat of the comment I mentioned on HDFS-7830 . Thank you to Lei (Eddy) Xu for volunteering to take assignment of the issue. Another potential problem that I've noticed in the DataNode reconfiguration code is that it never recalculates FsDatasetImpl#validVolsRequired . This is a final variable calculated as (# volumes configured) - (# volume failures tolerated): this .validVolsRequired = volsConfigured - volFailuresTolerated; If this variable is not updated for DataNode reconfigurations, then it could lead to some unexpected situations. For example: DataNode starts running with 6 volumes (all healthy) and dfs.datanode.failed.volumes.tolerated set to 2. FsDatasetImpl#validVolsRequired is set to 6 - 2 = 4. DataNode is reconfigured to run with 8 volumes (all still healthy). Now 3 volumes fail. The admin would expect the DataNode to abort, but there are 8 - 3 = 5 good volumes left, and FsDatasetImpl#validVolsRequired is still 4, so FsDatasetImpl#hasEnoughResource returns true.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hi, Chris Nauroth, thanks a lot for reporting this issue. I am trying to understand this bug. It seems to me that if we follow the same old logic, we need also update FsDdatasetImpl#validVolsRequired() when it removes a directory intentionly , but do not update it when the FsVolumeImpl is removed due to checkDirs. As an alternative, since now failure volumes are kept in FsDatasetImp#volumeFailureInfos, can we just let FsDatasetImpl#hasEnoughResources() become:

        public boolean hasEnoughResource() {
           return volumeFailureInfos.size() < volFailuresTolerated;
        }
        

        Does it make sense to you, Chris Nauroth?

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hi, Chris Nauroth , thanks a lot for reporting this issue. I am trying to understand this bug. It seems to me that if we follow the same old logic, we need also update FsDdatasetImpl#validVolsRequired() when it removes a directory intentionly , but do not update it when the FsVolumeImpl is removed due to checkDirs . As an alternative, since now failure volumes are kept in FsDatasetImp#volumeFailureInfos , can we just let FsDatasetImpl#hasEnoughResources() become: public boolean hasEnoughResource() { return volumeFailureInfos.size() < volFailuresTolerated; } Does it make sense to you, Chris Nauroth ?
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Upload a patch based on the above discussion.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Upload a patch based on the above discussion.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12705686/HDFS-7833.000.patch
        against trunk revision 61a4c7f.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
        org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache

        The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestLeaseRecovery2

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9983//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9983//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705686/HDFS-7833.000.patch against trunk revision 61a4c7f. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestLeaseRecovery2 Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9983//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9983//console This message is automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        Hi Lei (Eddy) Xu. Thank you for the patch. Overall, this looks correct to me. When I I filed this issue, it was before some of the discussion we had in HDFS-7722, and I had 2 cases in mind that could trigger this bug:

        • Admin reconfigures DataNode to remove a path that has a failed volume. As per discussion in HDFS-7722, we've made the decision that this case should not clear volume failure information. Since this is logically still considered a volume failure, there is no harm done to the check for sufficient resources. IOW, after the discussion in HDFS-7722, we don't have to worry about this case anymore.
        • Admin reconfigures DataNode and adds a few new paths that weren't there before. This case is still a problem.

        To properly cover the second case, let's add a test that does something like this:

        1. Start a DataNode with 2 volumes and dfs.datanode.failed.volumes.tolerated set to 1.
        2. Run DataNode reconfiguration to add a new volume. Now we're up to 3 volumes total.
        3. Fail a volume. Assert that the DataNode continues running.
        4. Fail another volume. Assert that the DataNode stops running.

        Without your patch, I expect this test would fail on the last step, because validVolsRequired would have been calculated as 1, and we still have 1 volume remaining. After applying your patch, I expect the test would then pass.

        Show
        cnauroth Chris Nauroth added a comment - Hi Lei (Eddy) Xu . Thank you for the patch. Overall, this looks correct to me. When I I filed this issue, it was before some of the discussion we had in HDFS-7722 , and I had 2 cases in mind that could trigger this bug: Admin reconfigures DataNode to remove a path that has a failed volume. As per discussion in HDFS-7722 , we've made the decision that this case should not clear volume failure information. Since this is logically still considered a volume failure, there is no harm done to the check for sufficient resources. IOW, after the discussion in HDFS-7722 , we don't have to worry about this case anymore. Admin reconfigures DataNode and adds a few new paths that weren't there before. This case is still a problem. To properly cover the second case, let's add a test that does something like this: Start a DataNode with 2 volumes and dfs.datanode.failed.volumes.tolerated set to 1. Run DataNode reconfiguration to add a new volume. Now we're up to 3 volumes total. Fail a volume. Assert that the DataNode continues running. Fail another volume. Assert that the DataNode stops running. Without your patch, I expect this test would fail on the last step, because validVolsRequired would have been calculated as 1, and we still have 1 volume remaining. After applying your patch, I expect the test would then pass.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Chris Nauroth Thanks for reviewing this.

        I updated the patch to add the test you described.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Chris Nauroth Thanks for reviewing this. I updated the patch to add the test you described.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12705796/HDFS-7833.001.patch
        against trunk revision e37ca22.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
        org.apache.hadoop.tracing.TestTracing

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9995//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9995//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705796/HDFS-7833.001.patch against trunk revision e37ca22. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA org.apache.hadoop.tracing.TestTracing Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9995//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9995//console This message is automatically generated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        These failing tests are not related.

        Show
        eddyxu Lei (Eddy) Xu added a comment - These failing tests are not related.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hi, Chris Nauroth. I just rebase to the newest trunk. Would you take another look?

        Thanks a lot!

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hi, Chris Nauroth . I just rebase to the newest trunk . Would you take another look? Thanks a lot!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 36s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 29s There were no new javac warning messages.
        +1 javadoc 9m 37s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 7m 25s There were no new checkstyle issues.
        +1 install 1m 32s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 native 3m 13s Pre-build of native portion
        -1 hdfs tests 165m 10s Tests failed in hadoop-hdfs.
            213m 6s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12728032/HDFS-7833.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 5ce3a77
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10378/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10378/testReport/
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10378/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 29s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 7m 25s There were no new checkstyle issues. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 13s Pre-build of native portion -1 hdfs tests 165m 10s Tests failed in hadoop-hdfs.     213m 6s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728032/HDFS-7833.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 5ce3a77 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10378/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10378/testReport/ Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10378/console This message was automatically generated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hey, Chris Nauroth. Would you mind to take a look at this JIRA? The above test failure is not relevant.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hey, Chris Nauroth . Would you mind to take a look at this JIRA? The above test failure is not relevant.
        Hide
        cnauroth Chris Nauroth added a comment -

        Hello Lei (Eddy) Xu. I apologize for the delay. Unfortunately, the patch needs a rebase now. Would you please upload a v003? I'll prioritize reviewing it.

        The new test looks like what I had in mind. I'll review in detail after the rebase. Thanks for incorporating the feedback.

        Show
        cnauroth Chris Nauroth added a comment - Hello Lei (Eddy) Xu . I apologize for the delay. Unfortunately, the patch needs a rebase now. Would you please upload a v003? I'll prioritize reviewing it. The new test looks like what I had in mind. I'll review in detail after the rebase. Thanks for incorporating the feedback.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hi, Chris Nauroth Thanks for looking into this. I rebased the patch in v003.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hi, Chris Nauroth Thanks for looking into this. I rebased the patch in v003.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 36s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 27s There were no new javac warning messages.
        +1 javadoc 9m 40s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 38s The applied patch generated 2 new checkstyle issues (total was 275, now 275).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 3m 6s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 native 3m 14s Pre-build of native portion
        -1 hdfs tests 166m 56s Tests failed in hadoop-hdfs.
            208m 13s  



        Reason Tests
        Failed unit tests hadoop.tracing.TestTraceAdmin



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12730951/HDFS-7833.003.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 185e63a
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10836/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10836/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10836/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10836/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 27s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 38s The applied patch generated 2 new checkstyle issues (total was 275, now 275). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 6s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 166m 56s Tests failed in hadoop-hdfs.     208m 13s   Reason Tests Failed unit tests hadoop.tracing.TestTraceAdmin Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12730951/HDFS-7833.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 185e63a checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10836/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10836/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10836/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10836/console This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for the patch. The test failure and checkstyle warnings in the last run were unrelated. I committed this to trunk and branch-2. Eddy, thank you for the contribution.

        Show
        cnauroth Chris Nauroth added a comment - +1 for the patch. The test failure and checkstyle warnings in the last run were unrelated. I committed this to trunk and branch-2. Eddy, thank you for the contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7754 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7754/)
        HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7754 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7754/ ) HDFS-7833 . DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Thanks a lot for reviewing and committing this, Chris Nauroth!

        Show
        eddyxu Lei (Eddy) Xu added a comment - Thanks a lot for reviewing and committing this, Chris Nauroth !
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/187/)
        HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/187/ ) HDFS-7833 . DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #920 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/920/)
        HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #920 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/920/ ) HDFS-7833 . DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2118 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2118/)
        HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2118 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2118/ ) HDFS-7833 . DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/177/)
        HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/177/ ) HDFS-7833 . DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/187/)
        HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #187 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/187/ ) HDFS-7833 . DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2136/)
        HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2136/ ) HDFS-7833 . DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. (cnauroth: rev 6633a8474d7e92fa028ede8fd6c6e41b6c5887f5) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

          People

          • Assignee:
            eddyxu Lei (Eddy) Xu
            Reporter:
            cnauroth Chris Nauroth
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development