Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      As part of HADOOP-8427, large, extremely useful sections of the Rack Awareness documentation that was added in HADOOP-6616 was wiped out. We should restore it as a separate document.

      1. HADOOP-11380.patch
        7 kB
        Allen Wittenauer

        Activity

        Hide
        aw Allen Wittenauer added a comment -

        -00:

        Initial patch.

        Show
        aw Allen Wittenauer added a comment - -00: Initial patch.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12686090/HADOOP-11380.patch
        against trunk revision 5776a41.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 111 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.ha.TestZKFailoverControllerStress

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5214//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5214//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5214//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12686090/HADOOP-11380.patch against trunk revision 5776a41. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 111 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverControllerStress Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5214//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5214//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5214//console This message is automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        +1 as is, though it could be good to list common problems with rack awareness setup. You did mention missing topos, there is also

        1. wrong IP addresses in the script -may result in missing nodes from the topology, even if you think you have them
        2. machines having wrong IP addrs. Same
        3. wrong machine to rack mapping: risk of scheduling "adjacent node" work on different racks, leading to increased backbone traffic. Possibly increased backbone traffic on writes. Possibly reduced resilience to failures.
        4. broken script. Check it on the CLI first. Consider keeping under SCM, maybe even tests (e.g PyUnit)
        5. out of date script. Have a process for updating it. Checking out from SCM is usually sufficient

        BTW, you know you can edit documents in markdown now, under site/markdown/*.md ? Multiple editors, automatic rendering if pushed to github

        Show
        stevel@apache.org Steve Loughran added a comment - +1 as is, though it could be good to list common problems with rack awareness setup. You did mention missing topos, there is also wrong IP addresses in the script -may result in missing nodes from the topology, even if you think you have them machines having wrong IP addrs. Same wrong machine to rack mapping: risk of scheduling "adjacent node" work on different racks, leading to increased backbone traffic. Possibly increased backbone traffic on writes. Possibly reduced resilience to failures. broken script. Check it on the CLI first. Consider keeping under SCM, maybe even tests (e.g PyUnit) out of date script. Have a process for updating it. Checking out from SCM is usually sufficient BTW, you know you can edit documents in markdown now, under site/markdown/*.md ? Multiple editors, automatic rendering if pushed to github
        Hide
        aw Allen Wittenauer added a comment -

        BTW, you know you can edit documents in markdown now, under site/markdown/*.md ?

        OH! No, I didn't! Yes, this would be MUCH better to do under markdown. I'm tempted to move some other docs to that as well. apt is... interesting. Awesome! Thanks!

        Show
        aw Allen Wittenauer added a comment - BTW, you know you can edit documents in markdown now, under site/markdown/*.md ? OH! No, I didn't! Yes, this would be MUCH better to do under markdown. I'm tempted to move some other docs to that as well. apt is... interesting. Awesome! Thanks!
        Hide
        aw Allen Wittenauer added a comment -

        The lack of a standard around table support in markdown makes it a less than great choice for a lot of documents.

        Show
        aw Allen Wittenauer added a comment - The lack of a standard around table support in markdown makes it a less than great choice for a lot of documents.
        Hide
        aw Allen Wittenauer added a comment -

        I think I'm going to commit as-is. We need to do a more comprehensive doc uplift anyway.

        Thanks for the review Steve Loughran!

        Show
        aw Allen Wittenauer added a comment - I think I'm going to commit as-is. We need to do a more comprehensive doc uplift anyway. Thanks for the review Steve Loughran !
        Hide
        aw Allen Wittenauer added a comment -

        Committed to trunk.

        Show
        aw Allen Wittenauer added a comment - Committed to trunk.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #6722 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6722/)
        HADOOP-11380. Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5)

        • hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #6722 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6722/ ) HADOOP-11380 . Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5) hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #43 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/43/)
        HADOOP-11380. Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5)

        • hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #43 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/43/ ) HADOOP-11380 . Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5) hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #777 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/777/)
        HADOOP-11380. Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5)

        • hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #777 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/777/ ) HADOOP-11380 . Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5) hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1994 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1994/)
        HADOOP-11380. Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1994 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1994/ ) HADOOP-11380 . Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #1975 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1975/)
        HADOOP-11380. Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5)

        • hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1975 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1975/ ) HADOOP-11380 . Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5) hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #40 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/40/)
        HADOOP-11380. Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5)

        • hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #40 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/40/ ) HADOOP-11380 . Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5) hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #44 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/44/)
        HADOOP-11380. Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5)

        • hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #44 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/44/ ) HADOOP-11380 . Restore Rack Awareness documenation (aw) (aw: rev e8a67bed10d749864a3bb2589c6686c40bebccc5) hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm hadoop-common-project/hadoop-common/CHANGES.txt

          People

          • Assignee:
            aw Allen Wittenauer
            Reporter:
            aw Allen Wittenauer
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development