Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2430

The number of failed or low-resource volumes the NN can tolerate should be configurable

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.0-alpha
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently the number of failed or low-resource volumes the NN can tolerate is effectively hard-coded at 1. It would be nice if this were configurable.

      1. HDFS-2430.patch
        51 kB
        Aaron T. Myers
      2. HDFS-2430.patch
        50 kB
        Aaron T. Myers
      3. HDFS-2430.patch
        49 kB
        Aaron T. Myers
      4. HDFS-2430.patch
        49 kB
        Aaron T. Myers
      5. HDFS-2430.patch
        48 kB
        Aaron T. Myers
      6. HDFS-2430.patch
        38 kB
        Aaron T. Myers

        Activity

        Hide
        Milind Bhandarkar added a comment -

        +1.

        Show
        Milind Bhandarkar added a comment - +1.
        Hide
        Suresh Srinivas added a comment -

        Aaron, with the patch from HDFS-2422, the javadoc for NameNodeResourceChecker is not valid. It says <code>hasAvailableDiskSpace</code> - which will return true if and only if the NameNode has disk space available on all volumes which are configured to be checked. Can you please address that in HDFS-2430.

        Show
        Suresh Srinivas added a comment - Aaron, with the patch from HDFS-2422 , the javadoc for NameNodeResourceChecker is not valid. It says <code>hasAvailableDiskSpace</code> - which will return true if and only if the NameNode has disk space available on all volumes which are configured to be checked . Can you please address that in HDFS-2430 .
        Hide
        Aaron T. Myers added a comment -

        Thanks for catching that, Suresh. I updated the method javadoc but didn't notice that the class javadoc mentions this behavior as well. I will be sure to fix this comment when I address this JIRA.

        Show
        Aaron T. Myers added a comment - Thanks for catching that, Suresh. I updated the method javadoc but didn't notice that the class javadoc mentions this behavior as well. I will be sure to fix this comment when I address this JIRA.
        Hide
        Aaron T. Myers added a comment -

        While just being able to configure the number of tolerated dir failures from the single pool of dirs specified by the dfs.namenode.name.dir config would be an improvement, I actually think the best solution would to be able to specify multiple distinct pools of name dirs, where the number of failed volumes tolerated could be configured per-pool. This will be useful since not all name dirs are created equal. For example, if an operator has 4 name dirs configured, 3 of which are local, with the other being on a remote machine mounted via NFS, the operator might want to configure the NN to tolerate up to 2 failures of the 3 local dirs, but stop immediately if the NFS dir goes away. I think such a scheme will be necessary for an NFS-based HA solution, as described as one of the options in HDFS-1623, but this configuration can be useful generally as well, so might as well be developed on trunk.

        I can imagine two high-level designs:

        1. The operator can configure exactly two distinct pools of name dirs, configured via something like dfs.namenode.name.dirs.required and dfs.namenode.name.dirs.redundant. If any single dir specified in the .required config goes offline, the NN will not continue to operate. The number of acceptable failed dirs in the .redundant pool would be configurable by a third option, dfs.name.node.name.dirs.failures.tolerated.
        2. The operator can specify N distinct pools of name dirs, configured via something like dfs.namenode.name.dirs.pool.0, dfs.namenode.name.dirs.pool.1, etc. For each of these configured pools, the number of failed volumes tolerated could be configured individually, e.g. dfs.namenode.name.dirs.failures.tolerated.pool.0, dfs.namenode.name.dirs.failures.tolerated.pool.1, etc.

        Under either of these schemes, it would be an error to specify the same dir in multiple pools.

        Any thoughts? Option 2 is obviously more flexible, but I don't want to thrust our operators deeper into configuration hell than they already are.

        Show
        Aaron T. Myers added a comment - While just being able to configure the number of tolerated dir failures from the single pool of dirs specified by the dfs.namenode.name.dir config would be an improvement, I actually think the best solution would to be able to specify multiple distinct pools of name dirs, where the number of failed volumes tolerated could be configured per-pool. This will be useful since not all name dirs are created equal. For example, if an operator has 4 name dirs configured, 3 of which are local, with the other being on a remote machine mounted via NFS, the operator might want to configure the NN to tolerate up to 2 failures of the 3 local dirs, but stop immediately if the NFS dir goes away. I think such a scheme will be necessary for an NFS-based HA solution, as described as one of the options in HDFS-1623 , but this configuration can be useful generally as well, so might as well be developed on trunk. I can imagine two high-level designs: The operator can configure exactly two distinct pools of name dirs, configured via something like dfs.namenode.name.dirs.required and dfs.namenode.name.dirs.redundant . If any single dir specified in the .required config goes offline, the NN will not continue to operate. The number of acceptable failed dirs in the .redundant pool would be configurable by a third option, dfs.name.node.name.dirs.failures.tolerated . The operator can specify N distinct pools of name dirs, configured via something like dfs.namenode.name.dirs.pool.0 , dfs.namenode.name.dirs.pool.1 , etc. For each of these configured pools, the number of failed volumes tolerated could be configured individually, e.g. dfs.namenode.name.dirs.failures.tolerated.pool.0 , dfs.namenode.name.dirs.failures.tolerated.pool.1 , etc. Under either of these schemes, it would be an error to specify the same dir in multiple pools. Any thoughts? Option 2 is obviously more flexible, but I don't want to thrust our operators deeper into configuration hell than they already are.
        Hide
        Todd Lipcon added a comment -

        I think option #1 is better - agree #2 is more flexible but I can't think of any real use cases.

        The other thing that needs to be considered in this design, though, is that we need to think separately about edits storage vs name directories. Edits may be stored in other implementations in the not-so-distant future (eg bookkeeper). So it's not just the name dirs, but also the edits managers which need to have such a configuration.

        Show
        Todd Lipcon added a comment - I think option #1 is better - agree #2 is more flexible but I can't think of any real use cases. The other thing that needs to be considered in this design, though, is that we need to think separately about edits storage vs name directories. Edits may be stored in other implementations in the not-so-distant future (eg bookkeeper). So it's not just the name dirs, but also the edits managers which need to have such a configuration.
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the comments, Todd. Here's a patch which addresses the issue. This implements option 1 as described above.

        I deviated slightly from what I initially described regarding the configuration parameters. Instead of two new configs, there's just a new "required" config. The previous edit dir config options are treated as they were before - as redundant directories. Some of these may be marked "required" using the new config introduced by this patch.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the comments, Todd. Here's a patch which addresses the issue. This implements option 1 as described above. I deviated slightly from what I initially described regarding the configuration parameters. Instead of two new configs, there's just a new "required" config. The previous edit dir config options are treated as they were before - as redundant directories. Some of these may be marked "required" using the new config introduced by this patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12501035/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.TestAbandonBlock

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1472//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1472//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501035/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestAbandonBlock +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1472//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1472//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        The test failure seems unrelated. I just tried running that test locally with the patch applied and it passed without issue.

        Show
        Aaron T. Myers added a comment - The test failure seems unrelated. I just tried running that test locally with the patch applied and it passed without issue.
        Hide
        Eli Collins added a comment -

        Agree that option #1 is better. The notion of a NN "required" dir here parallels the notion of a "critical" dir in the DN (proposed in HDFS-1848). I'll check out the patch.

        Show
        Eli Collins added a comment - Agree that option #1 is better. The notion of a NN "required" dir here parallels the notion of a "critical" dir in the DN (proposed in HDFS-1848 ). I'll check out the patch.
        Hide
        Aaron T. Myers added a comment -

        Hold off a minute, Eli. I just realized that I completely forgot to make the number of failures in the redundant pool configurable. Let me fix that before you spend time reviewing it.

        Show
        Aaron T. Myers added a comment - Hold off a minute, Eli. I just realized that I completely forgot to make the number of failures in the redundant pool configurable. Let me fix that before you spend time reviewing it.
        Hide
        Aaron T. Myers added a comment -

        Here's a patch which does everything the previous patch did, as well as implements the configurable number of redundant dirs, as well as adds a proper unit test for NameNodeResourcePolicy.

        Please review away.

        Show
        Aaron T. Myers added a comment - Here's a patch which does everything the previous patch did, as well as implements the configurable number of redundant dirs, as well as adds a proper unit test for NameNodeResourcePolicy . Please review away.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12501394/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 8 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.TestAbandonBlock
        org.apache.hadoop.hdfs.server.namenode.TestClusterId
        org.apache.hadoop.cli.TestHDFSCLI

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1491//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1491//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501394/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestAbandonBlock org.apache.hadoop.hdfs.server.namenode.TestClusterId org.apache.hadoop.cli.TestHDFSCLI +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1491//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1491//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Here's a patch which fixes the TestClusterId failure. That test was getting the configured cluster ID using a backdoor that wasn't bothering to set any edits dirs. The fix was:

        --- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
        +++ hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
        @@ -49,7 +49,7 @@ public class TestClusterId {
           private String getClusterId(Configuration config) throws IOException {
             // see if cluster id not empty.
             Collection<URI> dirsToFormat = FSNamesystem.getNamespaceDirs(config);
        -    Collection<URI> editsToFormat = new ArrayList<URI>(0);
        +    Collection<URI> editsToFormat = FSNamesystem.getNamespaceEditsDirs(config);
             FSImage fsImage = new FSImage(config, dirsToFormat, editsToFormat);
             
             Iterator<StorageDirectory> sdit =
        

        The TestAbandonBlock and TestHDFSCLI test failures are unrelated.

        Show
        Aaron T. Myers added a comment - Here's a patch which fixes the TestClusterId failure. That test was getting the configured cluster ID using a backdoor that wasn't bothering to set any edits dirs. The fix was: --- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java +++ hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java @@ -49,7 +49,7 @@ public class TestClusterId { private String getClusterId(Configuration config) throws IOException { // see if cluster id not empty. Collection<URI> dirsToFormat = FSNamesystem.getNamespaceDirs(config); - Collection<URI> editsToFormat = new ArrayList<URI>(0); + Collection<URI> editsToFormat = FSNamesystem.getNamespaceEditsDirs(config); FSImage fsImage = new FSImage(config, dirsToFormat, editsToFormat); Iterator<StorageDirectory> sdit = The TestAbandonBlock and TestHDFSCLI test failures are unrelated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12501431/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 11 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.TestAbandonBlock
        org.apache.hadoop.cli.TestHDFSCLI

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1497//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1497//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501431/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestAbandonBlock org.apache.hadoop.cli.TestHDFSCLI +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1497//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1497//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        The api for NNResourcePolicy#areResourcesAvailable is a little strange, if a required resource is not available we return false, but if there are not enough redundant resources we throw a RTE. Do callers need to distinguish between these two cases? If not, we should return false in both cases.
        If so, there should be two separate apis. I'd make the method name better reflect the purpose in the comment (whether we can log edits), the method is generic but the particular policy is edits-specific.

        Does the dir checking use canonical paths so we're sure the checks pass in case of symlinks?

        Nits:

        • NNResourcePolicy needs param javadocs
        • NNResourcePolicy, NNResourceChecker, and CheckableNNResource need @InterfaceAudience.Private annotations

        Otherwise looks great.

        Show
        Eli Collins added a comment - The api for NNResourcePolicy#areResourcesAvailable is a little strange, if a required resource is not available we return false, but if there are not enough redundant resources we throw a RTE. Do callers need to distinguish between these two cases? If not, we should return false in both cases. If so, there should be two separate apis. I'd make the method name better reflect the purpose in the comment (whether we can log edits), the method is generic but the particular policy is edits-specific. Does the dir checking use canonical paths so we're sure the checks pass in case of symlinks? Nits: NNResourcePolicy needs param javadocs NNResourcePolicy, NNResourceChecker, and CheckableNNResource need @InterfaceAudience.Private annotations Otherwise looks great.
        Hide
        Eli Collins added a comment -

        Per HDFS-208 might be worth considering other warnings to log eg if there's only a single redundant resource left. Could also standardize the location/format of the log check and output to make life easier for monitoring tools. Ditto for metrics (feel free to address in another jira).

        Show
        Eli Collins added a comment - Per HDFS-208 might be worth considering other warnings to log eg if there's only a single redundant resource left. Could also standardize the location/format of the log check and output to make life easier for monitoring tools. Ditto for metrics (feel free to address in another jira).
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the review, Eli. Comments below.

        The api for NNResourcePolicy#areResourcesAvailable is a little strange, if a required resource is not available we return false, but if there are not enough redundant resources we throw a RTE. Do callers need to distinguish between these two cases? If not, we should return false in both cases.
        If so, there should be two separate apis. I'd make the method name better reflect the purpose in the comment (whether we can log edits), the method is generic but the particular policy is edits-specific.

        It only throws an RTE if the configured number of redundant resources is less than the configured minimum number of redundant resources. i.e. it throws only in the case of misconfiguration. If indeed the number of configured redundant resources is greater than the minimum, but the number of available redundant resources is below the minimum, the method will return false. Does that seem reasonable?

        Does the dir checking use canonical paths so we're sure the checks pass in case of symlinks?

        Yes indeed. FSNS#getStorageDirs ends up calling o.a.h.hdfs.server.common.Util#fileAsURI which calls File#getCanonicalFile.

        I'll upload a patch in a moment which addresses the nits.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the review, Eli. Comments below. The api for NNResourcePolicy#areResourcesAvailable is a little strange, if a required resource is not available we return false, but if there are not enough redundant resources we throw a RTE. Do callers need to distinguish between these two cases? If not, we should return false in both cases. If so, there should be two separate apis. I'd make the method name better reflect the purpose in the comment (whether we can log edits), the method is generic but the particular policy is edits-specific. It only throws an RTE if the configured number of redundant resources is less than the configured minimum number of redundant resources. i.e. it throws only in the case of misconfiguration. If indeed the number of configured redundant resources is greater than the minimum, but the number of available redundant resources is below the minimum, the method will return false. Does that seem reasonable? Does the dir checking use canonical paths so we're sure the checks pass in case of symlinks? Yes indeed. FSNS#getStorageDirs ends up calling o.a.h.hdfs.server.common.Util#fileAsURI which calls File#getCanonicalFile . I'll upload a patch in a moment which addresses the nits.
        Hide
        Aaron T. Myers added a comment -

        Updated patch addressing Eli's comments.

        Show
        Aaron T. Myers added a comment - Updated patch addressing Eli's comments.
        Hide
        Aaron T. Myers added a comment -

        Per HDFS-208 might be worth considering other warnings to log eg if there's only a single redundant resource left. Could also standardize the location/format of the log check and output to make life easier for monitoring tools. Ditto for metrics (feel free to address in another jira).

        I've filed HDFS-2613 to address this.

        Show
        Aaron T. Myers added a comment - Per HDFS-208 might be worth considering other warnings to log eg if there's only a single redundant resource left. Could also standardize the location/format of the log check and output to make life easier for monitoring tools. Ditto for metrics (feel free to address in another jira). I've filed HDFS-2613 to address this.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12505685/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 11 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1617//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1617//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505685/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1617//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1617//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        It only throws an RTE if the configured number of redundant resources is less than the configured minimum number of redundant resources. i.e. it throws only in the case of misconfiguration. If indeed the number of configured redundant resources is greater than the minimum, but the number of available redundant resources is below the minimum, the method will return false. Does that seem reasonable?

        Ah, I think it would be more clear if the static (configuration) one-time check was separated from the dynamic check (eg so you don't have to check all callers to reason about the method) but I don't feel strongly. If you keep the current behavior please add a throws clause to the javadoc and add a comment per your above explanation. If you want to address it feel free to do so in a follow on jira. +1 to the latest patch.

        Show
        Eli Collins added a comment - It only throws an RTE if the configured number of redundant resources is less than the configured minimum number of redundant resources. i.e. it throws only in the case of misconfiguration. If indeed the number of configured redundant resources is greater than the minimum, but the number of available redundant resources is below the minimum, the method will return false. Does that seem reasonable? Ah, I think it would be more clear if the static (configuration) one-time check was separated from the dynamic check (eg so you don't have to check all callers to reason about the method) but I don't feel strongly. If you keep the current behavior please add a throws clause to the javadoc and add a comment per your above explanation. If you want to address it feel free to do so in a follow on jira. +1 to the latest patch.
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the review, Eli. I've added an @throws comment per your suggestion.

        The attached patch is also rebased on trunk, which had a few conflicts since HDFS-1580 was committed.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the review, Eli. I've added an @throws comment per your suggestion. The attached patch is also rebased on trunk, which had a few conflicts since HDFS-1580 was committed.
        Hide
        Eli Collins added a comment -

        +1 rebase on 1580 looks correct.

        Show
        Eli Collins added a comment - +1 rebase on 1580 looks correct.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12506178/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 11 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.TestDistributedFileSystem
        org.apache.hadoop.hdfs.TestConnCache
        org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
        org.apache.hadoop.hdfs.TestSafeMode
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit
        org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
        org.apache.hadoop.hdfs.TestFileAppend
        org.apache.hadoop.hdfs.TestDatanodeDeath
        org.apache.hadoop.hdfs.server.namenode.TestFileLimit
        org.apache.hadoop.hdfs.TestLease
        org.apache.hadoop.hdfs.TestShortCircuitLocalRead
        org.apache.hadoop.hdfs.TestDFSRename
        org.apache.hadoop.hdfs.TestPipelines
        org.apache.hadoop.hdfs.TestSetrepDecreasing
        org.apache.hadoop.hdfs.TestFileAppend2
        org.apache.hadoop.security.TestRefreshUserMappings
        org.apache.hadoop.tools.TestDelegationTokenFetcher
        org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
        org.apache.hadoop.hdfs.TestHFlush
        org.apache.hadoop.hdfs.TestLeaseRecovery
        org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
        org.apache.hadoop.hdfs.server.datanode.TestMulitipleNNDataBlockScanner
        org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
        org.apache.hadoop.fs.TestGlobPaths
        org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
        org.apache.hadoop.fs.TestResolveHdfsSymlink
        org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart
        org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
        org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
        org.apache.hadoop.hdfs.TestDfsOverAvroRpc
        org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
        org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
        org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
        org.apache.hadoop.hdfs.TestAbandonBlock
        org.apache.hadoop.fs.TestUrlStreamHandler
        org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage
        org.apache.hadoop.hdfs.TestBlockMissingException
        org.apache.hadoop.fs.viewfs.TestViewFsFileStatusHdfs
        org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
        org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
        org.apache.hadoop.hdfs.TestFileCreation
        org.apache.hadoop.fs.TestFcHdfsPermission
        org.apache.hadoop.hdfs.TestCrcCorruption
        org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
        org.apache.hadoop.hdfs.TestListFilesInFileContext
        org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
        org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
        org.apache.hadoop.hdfs.TestListPathServlet
        org.apache.hadoop.hdfs.TestMultiThreadedHflush
        org.apache.hadoop.hdfs.TestDFSFinalize
        org.apache.hadoop.hdfs.TestLocalDFS
        org.apache.hadoop.hdfs.server.namenode.TestBackupNode
        org.apache.hadoop.hdfs.TestDFSStartupVersions
        org.apache.hadoop.hdfs.server.balancer.TestBalancer
        org.apache.hadoop.hdfs.tools.TestGetGroups
        org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
        org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
        org.apache.hadoop.hdfs.TestDataTransferProtocol
        org.apache.hadoop.hdfs.TestDFSClientRetries
        org.apache.hadoop.hdfs.TestAppendDifferentChecksum
        org.apache.hadoop.hdfs.TestFileCreationDelete
        org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
        org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade
        org.apache.hadoop.hdfs.TestMiniDFSCluster
        org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
        org.apache.hadoop.hdfs.TestFileCreationClient
        org.apache.hadoop.hdfs.server.datanode.TestWriteToReplica
        org.apache.hadoop.hdfs.TestListFilesInDFS
        org.apache.hadoop.hdfs.TestDatanodeConfig
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
        org.apache.hadoop.hdfs.server.namenode.TestMetaSave
        org.apache.hadoop.hdfs.server.namenode.TestClusterId
        org.apache.hadoop.hdfs.server.namenode.TestFsck
        org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
        org.apache.hadoop.hdfs.TestDFSMkdirs
        org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
        org.apache.hadoop.fs.TestFcHdfsCreateMkdir
        org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
        org.apache.hadoop.hdfs.TestModTime
        org.apache.hadoop.hdfs.TestDFSRollback
        org.apache.hadoop.security.TestPermission
        org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream
        org.apache.hadoop.hdfs.server.namenode.TestEditLog
        org.apache.hadoop.hdfs.server.namenode.TestStreamFile
        org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs
        org.apache.hadoop.hdfs.TestDecommission
        org.apache.hadoop.fs.TestHDFSFileContextMainOperations
        org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
        org.apache.hadoop.hdfs.server.datanode.TestDiskError
        org.apache.hadoop.hdfs.server.datanode.TestTransferRbw
        org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean
        org.apache.hadoop.hdfs.TestDatanodeBlockScanner
        org.apache.hadoop.hdfs.TestFileCorruption
        org.apache.hadoop.hdfs.TestClientBlockVerification
        org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
        org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
        org.apache.hadoop.hdfs.TestFileAppend3
        org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
        org.apache.hadoop.hdfs.TestFileAppend4
        org.apache.hadoop.hdfs.TestBlocksScheduledCounter
        org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
        org.apache.hadoop.hdfs.server.namenode.TestStartup
        org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp
        org.apache.hadoop.hdfs.TestWriteRead
        org.apache.hadoop.hdfs.TestRenameWhileOpen
        org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
        org.apache.hadoop.hdfs.TestDFSRemove
        org.apache.hadoop.fs.viewfs.TestViewFsHdfs
        org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
        org.apache.hadoop.hdfs.TestDatanodeReport
        org.apache.hadoop.hdfs.TestHDFSFileSystemContract
        org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
        org.apache.hadoop.hdfs.TestFSInputChecker
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
        org.apache.hadoop.hdfs.server.datanode.TestBlockReport
        org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
        org.apache.hadoop.hdfs.TestSetTimes
        org.apache.hadoop.hdfs.TestPread
        org.apache.hadoop.hdfs.security.TestClientProtocolWithDelegationToken
        org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
        org.apache.hadoop.hdfs.security.TestDelegationToken
        org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
        org.apache.hadoop.hdfs.TestDFSShellGenericOptions
        org.apache.hadoop.hdfs.TestDFSPermission
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
        org.apache.hadoop.hdfs.TestHDFSTrash
        org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser
        org.apache.hadoop.fs.TestFcHdfsSymlink
        org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
        org.apache.hadoop.hdfs.TestFileStatus
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper
        org.apache.hadoop.hdfs.TestReadWhileWriting
        org.apache.hadoop.hdfs.TestHftpFileSystem
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
        org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
        org.apache.hadoop.hdfs.server.namenode.TestSafeMode
        org.apache.hadoop.hdfs.TestLeaseRenewer
        org.apache.hadoop.hdfs.TestDatanodeRegistration
        org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp
        org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures
        org.apache.hadoop.hdfs.TestLargeBlock
        org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
        org.apache.hadoop.hdfs.TestParallelRead
        org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
        org.apache.hadoop.hdfs.server.datanode.TestInterDatanodeProtocol
        org.apache.hadoop.hdfs.server.datanode.TestDatanodeRestart
        org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
        org.apache.hadoop.hdfs.TestFSOutputSummer
        org.apache.hadoop.hdfs.TestFileCreationEmpty
        org.apache.hadoop.hdfs.TestBalancerBandwidth
        org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
        org.apache.hadoop.fs.permission.TestStickyBit
        org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
        org.apache.hadoop.hdfs.security.token.block.TestBlockToken
        org.apache.hadoop.hdfs.TestHftpDelegationToken
        org.apache.hadoop.hdfs.TestRestartDFS
        org.apache.hadoop.hdfs.TestGetBlocks
        org.apache.hadoop.hdfs.TestSmallBlock
        org.apache.hadoop.hdfs.TestSetrepIncreasing
        org.apache.hadoop.hdfs.TestHDFSServerPorts
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
        org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
        org.apache.hadoop.hdfs.TestReplication
        org.apache.hadoop.hdfs.TestDFSUpgrade
        org.apache.hadoop.tools.TestJMXGet
        org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
        org.apache.hadoop.hdfs.TestQuota
        org.apache.hadoop.hdfs.TestFileConcurrentReader

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1641//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1641//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506178/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.TestConnCache org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite org.apache.hadoop.hdfs.TestSafeMode org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat org.apache.hadoop.hdfs.TestFileAppend org.apache.hadoop.hdfs.TestDatanodeDeath org.apache.hadoop.hdfs.server.namenode.TestFileLimit org.apache.hadoop.hdfs.TestLease org.apache.hadoop.hdfs.TestShortCircuitLocalRead org.apache.hadoop.hdfs.TestDFSRename org.apache.hadoop.hdfs.TestPipelines org.apache.hadoop.hdfs.TestSetrepDecreasing org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.security.TestRefreshUserMappings org.apache.hadoop.tools.TestDelegationTokenFetcher org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes org.apache.hadoop.hdfs.TestHFlush org.apache.hadoop.hdfs.TestLeaseRecovery org.apache.hadoop.fs.loadGenerator.TestLoadGenerator org.apache.hadoop.hdfs.server.datanode.TestMulitipleNNDataBlockScanner org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount org.apache.hadoop.fs.TestGlobPaths org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction org.apache.hadoop.fs.TestResolveHdfsSymlink org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens org.apache.hadoop.hdfs.TestDfsOverAvroRpc org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker org.apache.hadoop.hdfs.TestAbandonBlock org.apache.hadoop.fs.TestUrlStreamHandler org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage org.apache.hadoop.hdfs.TestBlockMissingException org.apache.hadoop.fs.viewfs.TestViewFsFileStatusHdfs org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks org.apache.hadoop.hdfs.TestFileCreation org.apache.hadoop.fs.TestFcHdfsPermission org.apache.hadoop.hdfs.TestCrcCorruption org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport org.apache.hadoop.hdfs.TestListFilesInFileContext org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.TestListPathServlet org.apache.hadoop.hdfs.TestMultiThreadedHflush org.apache.hadoop.hdfs.TestDFSFinalize org.apache.hadoop.hdfs.TestLocalDFS org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.TestDFSStartupVersions org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.tools.TestGetGroups org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps org.apache.hadoop.hdfs.TestDataTransferProtocol org.apache.hadoop.hdfs.TestDFSClientRetries org.apache.hadoop.hdfs.TestAppendDifferentChecksum org.apache.hadoop.hdfs.TestFileCreationDelete org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade org.apache.hadoop.hdfs.TestMiniDFSCluster org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete org.apache.hadoop.hdfs.TestFileCreationClient org.apache.hadoop.hdfs.server.datanode.TestWriteToReplica org.apache.hadoop.hdfs.TestListFilesInDFS org.apache.hadoop.hdfs.TestDatanodeConfig org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean org.apache.hadoop.hdfs.server.namenode.TestMetaSave org.apache.hadoop.hdfs.server.namenode.TestClusterId org.apache.hadoop.hdfs.server.namenode.TestFsck org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks org.apache.hadoop.hdfs.TestDFSMkdirs org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace org.apache.hadoop.fs.TestFcHdfsCreateMkdir org.apache.hadoop.hdfs.server.namenode.TestAllowFormat org.apache.hadoop.hdfs.TestModTime org.apache.hadoop.hdfs.TestDFSRollback org.apache.hadoop.security.TestPermission org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream org.apache.hadoop.hdfs.server.namenode.TestEditLog org.apache.hadoop.hdfs.server.namenode.TestStreamFile org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs org.apache.hadoop.hdfs.TestDecommission org.apache.hadoop.fs.TestHDFSFileContextMainOperations org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode org.apache.hadoop.hdfs.server.datanode.TestDiskError org.apache.hadoop.hdfs.server.datanode.TestTransferRbw org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean org.apache.hadoop.hdfs.TestDatanodeBlockScanner org.apache.hadoop.hdfs.TestFileCorruption org.apache.hadoop.hdfs.TestClientBlockVerification org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery org.apache.hadoop.hdfs.TestFileAppend3 org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks org.apache.hadoop.hdfs.TestFileAppend4 org.apache.hadoop.hdfs.TestBlocksScheduledCounter org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus org.apache.hadoop.hdfs.server.namenode.TestStartup org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp org.apache.hadoop.hdfs.TestWriteRead org.apache.hadoop.hdfs.TestRenameWhileOpen org.apache.hadoop.hdfs.TestDFSUpgradeFromImage org.apache.hadoop.hdfs.TestDFSRemove org.apache.hadoop.fs.viewfs.TestViewFsHdfs org.apache.hadoop.hdfs.server.namenode.TestCheckpoint org.apache.hadoop.hdfs.TestDatanodeReport org.apache.hadoop.hdfs.TestHDFSFileSystemContract org.apache.hadoop.hdfs.TestDFSClientExcludedNodes org.apache.hadoop.hdfs.TestFSInputChecker org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics org.apache.hadoop.hdfs.server.datanode.TestBlockReport org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner org.apache.hadoop.hdfs.TestSetTimes org.apache.hadoop.hdfs.TestPread org.apache.hadoop.hdfs.security.TestClientProtocolWithDelegationToken org.apache.hadoop.hdfs.server.namenode.TestEditLogRace org.apache.hadoop.hdfs.security.TestDelegationToken org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional org.apache.hadoop.hdfs.TestDFSShellGenericOptions org.apache.hadoop.hdfs.TestDFSPermission org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS org.apache.hadoop.hdfs.TestHDFSTrash org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser org.apache.hadoop.fs.TestFcHdfsSymlink org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark org.apache.hadoop.hdfs.TestFileStatus org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper org.apache.hadoop.hdfs.TestReadWhileWriting org.apache.hadoop.hdfs.TestHftpFileSystem org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.server.namenode.TestSafeMode org.apache.hadoop.hdfs.TestLeaseRenewer org.apache.hadoop.hdfs.TestDatanodeRegistration org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures org.apache.hadoop.hdfs.TestLargeBlock org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork org.apache.hadoop.hdfs.TestParallelRead org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage org.apache.hadoop.hdfs.server.datanode.TestInterDatanodeProtocol org.apache.hadoop.hdfs.server.datanode.TestDatanodeRestart org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes org.apache.hadoop.hdfs.TestFSOutputSummer org.apache.hadoop.hdfs.TestFileCreationEmpty org.apache.hadoop.hdfs.TestBalancerBandwidth org.apache.hadoop.hdfs.TestWriteConfigurationToDFS org.apache.hadoop.fs.permission.TestStickyBit org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader org.apache.hadoop.hdfs.security.token.block.TestBlockToken org.apache.hadoop.hdfs.TestHftpDelegationToken org.apache.hadoop.hdfs.TestRestartDFS org.apache.hadoop.hdfs.TestGetBlocks org.apache.hadoop.hdfs.TestSmallBlock org.apache.hadoop.hdfs.TestSetrepIncreasing org.apache.hadoop.hdfs.TestHDFSServerPorts org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings org.apache.hadoop.hdfs.TestReplication org.apache.hadoop.hdfs.TestDFSUpgrade org.apache.hadoop.tools.TestJMXGet org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration org.apache.hadoop.hdfs.TestQuota org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1641//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1641//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Something went majorly wrong with that test run - OOMs and "Unable to create native thread." I've also run a selection of the failed tests on my local box, and they all passed.

        I've just kicked Jenkins to re-run test-patch. We'll see how that goes.

        Show
        Aaron T. Myers added a comment - Something went majorly wrong with that test run - OOMs and "Unable to create native thread." I've also run a selection of the failed tests on my local box, and they all passed. I've just kicked Jenkins to re-run test-patch. We'll see how that goes.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12506178/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 11 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.TestDistributedFileSystem
        org.apache.hadoop.hdfs.TestConnCache
        org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
        org.apache.hadoop.hdfs.TestSafeMode
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit
        org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
        org.apache.hadoop.hdfs.TestFileAppend
        org.apache.hadoop.hdfs.TestDatanodeDeath
        org.apache.hadoop.hdfs.server.namenode.TestFileLimit
        org.apache.hadoop.hdfs.TestLease
        org.apache.hadoop.hdfs.TestShortCircuitLocalRead
        org.apache.hadoop.hdfs.TestPipelines
        org.apache.hadoop.hdfs.TestSetrepDecreasing
        org.apache.hadoop.hdfs.TestLeaseRecovery2
        org.apache.hadoop.hdfs.TestFileAppend2
        org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
        org.apache.hadoop.hdfs.TestHFlush
        org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
        org.apache.hadoop.hdfs.server.datanode.TestMulitipleNNDataBlockScanner
        org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
        org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
        org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart
        org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
        org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
        org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
        org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
        org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
        org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage
        org.apache.hadoop.hdfs.TestBlockMissingException
        org.apache.hadoop.hdfs.TestClientReportBadBlock
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
        org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
        org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
        org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
        org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
        org.apache.hadoop.hdfs.TestMultiThreadedHflush
        org.apache.hadoop.hdfs.TestDFSFinalize
        org.apache.hadoop.hdfs.server.namenode.TestBackupNode
        org.apache.hadoop.hdfs.TestDFSStartupVersions
        org.apache.hadoop.hdfs.server.balancer.TestBalancer
        org.apache.hadoop.hdfs.tools.TestGetGroups
        org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
        org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
        org.apache.hadoop.hdfs.TestDataTransferProtocol
        org.apache.hadoop.hdfs.TestDFSClientRetries
        org.apache.hadoop.hdfs.TestAppendDifferentChecksum
        org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
        org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade
        org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
        org.apache.hadoop.hdfs.server.datanode.TestWriteToReplica
        org.apache.hadoop.hdfs.TestListFilesInDFS
        org.apache.hadoop.hdfs.TestDatanodeConfig
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
        org.apache.hadoop.hdfs.server.namenode.TestMetaSave
        org.apache.hadoop.hdfs.server.namenode.TestFsck
        org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
        org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
        org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
        org.apache.hadoop.hdfs.TestModTime
        org.apache.hadoop.hdfs.TestDFSRollback
        org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream
        org.apache.hadoop.hdfs.server.namenode.TestEditLog
        org.apache.hadoop.hdfs.server.namenode.TestStreamFile
        org.apache.hadoop.hdfs.TestDecommission
        org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
        org.apache.hadoop.hdfs.server.datanode.TestDiskError
        org.apache.hadoop.hdfs.server.datanode.TestTransferRbw
        org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean
        org.apache.hadoop.hdfs.TestDatanodeBlockScanner
        org.apache.hadoop.hdfs.TestFileCorruption
        org.apache.hadoop.hdfs.TestClientBlockVerification
        org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
        org.apache.hadoop.hdfs.TestFileAppend3
        org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
        org.apache.hadoop.hdfs.TestBlocksScheduledCounter
        org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
        org.apache.hadoop.hdfs.server.namenode.TestStartup
        org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp
        org.apache.hadoop.hdfs.TestWriteRead
        org.apache.hadoop.hdfs.TestRenameWhileOpen
        org.apache.hadoop.hdfs.TestDFSRemove
        org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
        org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
        org.apache.hadoop.hdfs.TestFSInputChecker
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
        org.apache.hadoop.hdfs.server.datanode.TestBlockReport
        org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
        org.apache.hadoop.hdfs.TestSetTimes
        org.apache.hadoop.hdfs.TestPread
        org.apache.hadoop.hdfs.security.TestClientProtocolWithDelegationToken
        org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
        org.apache.hadoop.hdfs.security.TestDelegationToken
        org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
        org.apache.hadoop.hdfs.TestDFSShellGenericOptions
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
        org.apache.hadoop.hdfs.TestHDFSTrash
        org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser
        org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
        org.apache.hadoop.hdfs.TestFileStatus
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
        org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper
        org.apache.hadoop.hdfs.TestHftpFileSystem
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
        org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
        org.apache.hadoop.hdfs.server.namenode.TestSafeMode
        org.apache.hadoop.hdfs.TestDatanodeRegistration
        org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp
        org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures
        org.apache.hadoop.hdfs.TestLargeBlock
        org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
        org.apache.hadoop.hdfs.TestParallelRead
        org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
        org.apache.hadoop.hdfs.server.datanode.TestInterDatanodeProtocol
        org.apache.hadoop.hdfs.server.datanode.TestDatanodeRestart
        org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
        org.apache.hadoop.hdfs.TestFSOutputSummer
        org.apache.hadoop.hdfs.TestFileCreationEmpty
        org.apache.hadoop.hdfs.TestBalancerBandwidth
        org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
        org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
        org.apache.hadoop.hdfs.security.token.block.TestBlockToken
        org.apache.hadoop.hdfs.TestGetBlocks
        org.apache.hadoop.hdfs.TestSmallBlock
        org.apache.hadoop.hdfs.TestSetrepIncreasing
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
        org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
        org.apache.hadoop.hdfs.TestReplication
        org.apache.hadoop.hdfs.TestDFSUpgrade
        org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
        org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
        org.apache.hadoop.hdfs.TestQuota
        org.apache.hadoop.hdfs.TestFileConcurrentReader

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1642//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1642//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506178/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.TestConnCache org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite org.apache.hadoop.hdfs.TestSafeMode org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat org.apache.hadoop.hdfs.TestFileAppend org.apache.hadoop.hdfs.TestDatanodeDeath org.apache.hadoop.hdfs.server.namenode.TestFileLimit org.apache.hadoop.hdfs.TestLease org.apache.hadoop.hdfs.TestShortCircuitLocalRead org.apache.hadoop.hdfs.TestPipelines org.apache.hadoop.hdfs.TestSetrepDecreasing org.apache.hadoop.hdfs.TestLeaseRecovery2 org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes org.apache.hadoop.hdfs.TestHFlush org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract org.apache.hadoop.hdfs.server.datanode.TestMulitipleNNDataBlockScanner org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage org.apache.hadoop.hdfs.TestBlockMissingException org.apache.hadoop.hdfs.TestClientReportBadBlock org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.TestMultiThreadedHflush org.apache.hadoop.hdfs.TestDFSFinalize org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.TestDFSStartupVersions org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.tools.TestGetGroups org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps org.apache.hadoop.hdfs.TestDataTransferProtocol org.apache.hadoop.hdfs.TestDFSClientRetries org.apache.hadoop.hdfs.TestAppendDifferentChecksum org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete org.apache.hadoop.hdfs.server.datanode.TestWriteToReplica org.apache.hadoop.hdfs.TestListFilesInDFS org.apache.hadoop.hdfs.TestDatanodeConfig org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean org.apache.hadoop.hdfs.server.namenode.TestMetaSave org.apache.hadoop.hdfs.server.namenode.TestFsck org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace org.apache.hadoop.hdfs.server.namenode.TestAllowFormat org.apache.hadoop.hdfs.TestModTime org.apache.hadoop.hdfs.TestDFSRollback org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream org.apache.hadoop.hdfs.server.namenode.TestEditLog org.apache.hadoop.hdfs.server.namenode.TestStreamFile org.apache.hadoop.hdfs.TestDecommission org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode org.apache.hadoop.hdfs.server.datanode.TestDiskError org.apache.hadoop.hdfs.server.datanode.TestTransferRbw org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean org.apache.hadoop.hdfs.TestDatanodeBlockScanner org.apache.hadoop.hdfs.TestFileCorruption org.apache.hadoop.hdfs.TestClientBlockVerification org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog org.apache.hadoop.hdfs.TestFileAppend3 org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks org.apache.hadoop.hdfs.TestBlocksScheduledCounter org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus org.apache.hadoop.hdfs.server.namenode.TestStartup org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp org.apache.hadoop.hdfs.TestWriteRead org.apache.hadoop.hdfs.TestRenameWhileOpen org.apache.hadoop.hdfs.TestDFSRemove org.apache.hadoop.hdfs.server.namenode.TestCheckpoint org.apache.hadoop.hdfs.TestDFSClientExcludedNodes org.apache.hadoop.hdfs.TestFSInputChecker org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics org.apache.hadoop.hdfs.server.datanode.TestBlockReport org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner org.apache.hadoop.hdfs.TestSetTimes org.apache.hadoop.hdfs.TestPread org.apache.hadoop.hdfs.security.TestClientProtocolWithDelegationToken org.apache.hadoop.hdfs.server.namenode.TestEditLogRace org.apache.hadoop.hdfs.security.TestDelegationToken org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional org.apache.hadoop.hdfs.TestDFSShellGenericOptions org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS org.apache.hadoop.hdfs.TestHDFSTrash org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark org.apache.hadoop.hdfs.TestFileStatus org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper org.apache.hadoop.hdfs.TestHftpFileSystem org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.server.namenode.TestSafeMode org.apache.hadoop.hdfs.TestDatanodeRegistration org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures org.apache.hadoop.hdfs.TestLargeBlock org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork org.apache.hadoop.hdfs.TestParallelRead org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage org.apache.hadoop.hdfs.server.datanode.TestInterDatanodeProtocol org.apache.hadoop.hdfs.server.datanode.TestDatanodeRestart org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes org.apache.hadoop.hdfs.TestFSOutputSummer org.apache.hadoop.hdfs.TestFileCreationEmpty org.apache.hadoop.hdfs.TestBalancerBandwidth org.apache.hadoop.hdfs.TestWriteConfigurationToDFS org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader org.apache.hadoop.hdfs.security.token.block.TestBlockToken org.apache.hadoop.hdfs.TestGetBlocks org.apache.hadoop.hdfs.TestSmallBlock org.apache.hadoop.hdfs.TestSetrepIncreasing org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings org.apache.hadoop.hdfs.TestReplication org.apache.hadoop.hdfs.TestDFSUpgrade org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration org.apache.hadoop.hdfs.TestQuota org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1642//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1642//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Same sort of odd failure. I've just kicked Jenkins again. Third time's a charm?

        Show
        Aaron T. Myers added a comment - Same sort of odd failure. I've just kicked Jenkins again. Third time's a charm?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12506178/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 11 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
        org.apache.hadoop.hdfs.TestParallelRead
        org.apache.hadoop.hdfs.TestCrcCorruption
        org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf
        org.apache.hadoop.hdfs.TestQuota
        org.apache.hadoop.hdfs.security.token.block.TestBlockToken
        org.apache.hadoop.hdfs.TestFileAppend3
        org.apache.hadoop.hdfs.security.TestDelegationToken
        org.apache.hadoop.hdfs.tools.TestGetGroups

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1652//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1652//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506178/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 11 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer org.apache.hadoop.hdfs.TestParallelRead org.apache.hadoop.hdfs.TestCrcCorruption org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf org.apache.hadoop.hdfs.TestQuota org.apache.hadoop.hdfs.security.token.block.TestBlockToken org.apache.hadoop.hdfs.TestFileAppend3 org.apache.hadoop.hdfs.security.TestDelegationToken org.apache.hadoop.hdfs.tools.TestGetGroups +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1652//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1652//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Most of those test failures seem to be spurious, but the recently-introduced TestGenericJournalConf failure does seem legitimate. I'm looking into that.

        Show
        Aaron T. Myers added a comment - Most of those test failures seem to be spurious, but the recently-introduced TestGenericJournalConf failure does seem legitimate. I'm looking into that.
        Hide
        Aaron T. Myers added a comment -

        Updated patch to address the TestGenericJournalConf failure. The only difference between this patch and the last is:

        diff --git hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apa
        index 5679900..00fe43f 100644
        — hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
        +++ hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
        @@ -118,6 +118,7 @@ public class TestGenericJournalConf {
        DummyJournalManager.class.getName());
        conf.set(DFSConfigKeys.DFS_NAMENODE_EDITS_DIR_KEY,
        "dummy://test");
        + conf.setInt(DFSConfigKeys.DFS_NAMENODE_CHECKED_VOLUMES_MINIMUM_KEY, 0);
        try {
        cluster = new MiniDFSCluster.Builder(conf).numDataNodes(0).build();
        cluster.waitActive();

        Show
        Aaron T. Myers added a comment - Updated patch to address the TestGenericJournalConf failure. The only difference between this patch and the last is: diff --git hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apa index 5679900..00fe43f 100644 — hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java +++ hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java @@ -118,6 +118,7 @@ public class TestGenericJournalConf { DummyJournalManager.class.getName()); conf.set(DFSConfigKeys.DFS_NAMENODE_EDITS_DIR_KEY, "dummy://test"); + conf.setInt(DFSConfigKeys.DFS_NAMENODE_CHECKED_VOLUMES_MINIMUM_KEY, 0); try { cluster = new MiniDFSCluster.Builder(conf).numDataNodes(0).build(); cluster.waitActive();
        Hide
        Aaron T. Myers added a comment -

        I ran all the tests which failed in the last run on my local box, and they all passed:

        -------------------------------------------------------
        T E S T S
        -------------------------------------------------------
        Running org.apache.hadoop.hdfs.TestQuota
        Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.869 sec
        Running org.apache.hadoop.hdfs.tools.TestGetGroups
        Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.481 sec
        Running org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
        Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.276 sec
        Running org.apache.hadoop.hdfs.security.TestDelegationToken
        Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.952 sec
        Running org.apache.hadoop.hdfs.security.token.block.TestBlockToken
        Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.599 sec
        Running org.apache.hadoop.hdfs.TestFileAppend3
        Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.35 sec
        Running org.apache.hadoop.hdfs.TestParallelRead
        Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.063 sec
        Running org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf
        Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.19 sec

        Results :

        Tests run: 36, Failures: 0, Errors: 0, Skipped: 0

        Show
        Aaron T. Myers added a comment - I ran all the tests which failed in the last run on my local box, and they all passed: ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.hdfs.TestQuota Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.869 sec Running org.apache.hadoop.hdfs.tools.TestGetGroups Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.481 sec Running org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.276 sec Running org.apache.hadoop.hdfs.security.TestDelegationToken Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.952 sec Running org.apache.hadoop.hdfs.security.token.block.TestBlockToken Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.599 sec Running org.apache.hadoop.hdfs.TestFileAppend3 Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.35 sec Running org.apache.hadoop.hdfs.TestParallelRead Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.063 sec Running org.apache.hadoop.hdfs.server.namenode.TestGenericJournalConf Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.19 sec Results : Tests run: 36, Failures: 0, Errors: 0, Skipped: 0
        Hide
        Eli Collins added a comment -

        +1 to the delta, lgtm

        Show
        Eli Collins added a comment - +1 to the delta, lgtm
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12506503/HDFS-2430.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 15 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1657//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1657//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506503/HDFS-2430.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1657//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1657//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Thanks a lot for the reviews, Eli. I've just committed this to trunk.

        Show
        Aaron T. Myers added a comment - Thanks a lot for the reviews, Eli. I've just committed this to trunk.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1448 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1448/)
        HDFS-2430. The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1448 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1448/ ) HDFS-2430 . The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1376 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1376/)
        HDFS-2430. The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1376 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1376/ ) HDFS-2430 . The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1400 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1400/)
        HDFS-2430. The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1400 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1400/ ) HDFS-2430 . The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #921 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/921/)
        HDFS-2430. The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #921 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/921/ ) HDFS-2430 . The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #888 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/888/)
        HDFS-2430. The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers

        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #888 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/888/ ) HDFS-2430 . The number of failed or low-resource volumes the NN can tolerate should be configurable. Contributed by Aaron T. Myers atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211650 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckableNameNodeResource.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourcePolicy.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourcePolicy.java

          People

          • Assignee:
            Aaron T. Myers
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development