Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4341

add doc about timeline performance tool usage

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. YARN-4341.2.patch
      4 kB
      Chang Li
    2. YARN-4341.3.patch
      4 kB
      Chang Li
    3. YARN-4341.4.patch
      4 kB
      Chang Li
    4. YARN-4341.patch
      4 kB
      Chang Li

      Issue Links

        Activity

        Hide
        lichangleo Chang Li added a comment -

        Sangjin Lee, please help review this doc. Thanks!

        Show
        lichangleo Chang Li added a comment - Sangjin Lee , please help review this doc. Thanks!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 7s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvnsite 0m 14s trunk passed
        +1 mvnsite 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        1m 2s



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-12
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771981/YARN-4341.patch
        JIRA Issue YARN-4341
        Optional Tests asflicense mvnsite
        uname Linux 652e1ff1e703 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh
        git revision trunk / 06a7031
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9672/artifact/patchprocess/whitespace-eol.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        Max memory used 32MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9672/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvnsite 0m 14s trunk passed +1 mvnsite 0m 13s the patch passed -1 whitespace 0m 0s The patch has 5 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 1m 2s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-12 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771981/YARN-4341.patch JIRA Issue YARN-4341 Optional Tests asflicense mvnsite uname Linux 652e1ff1e703 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh git revision trunk / 06a7031 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9672/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site Max memory used 32MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9672/console This message was automatically generated.
        Hide
        lichangleo Chang Li added a comment -

        .2 fix whitespace

        Show
        lichangleo Chang Li added a comment - .2 fix whitespace
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvnsite 0m 23s trunk passed
        +1 mvnsite 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        1m 10s



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-12
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771983/YARN-4341.2.patch
        JIRA Issue YARN-4341
        Optional Tests asflicense mvnsite
        uname Linux ac984c878f28 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh
        git revision trunk / 06a7031
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        Max memory used 31MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9673/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvnsite 0m 23s trunk passed +1 mvnsite 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 1m 10s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-12 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771983/YARN-4341.2.patch JIRA Issue YARN-4341 Optional Tests asflicense mvnsite uname Linux ac984c878f28 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh git revision trunk / 06a7031 modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site Max memory used 31MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9673/console This message was automatically generated.
        Hide
        sjlee0 Sangjin Lee added a comment -

        Sorry Chang Li it took me a while to get to this. Some corrections and suggestions below.

        (Highlights)

        • "Timeline..." -> "The timeline..."
        • "help measure..." -> "helps measure..."
        • "Test will launch..." -> "The test launches..."
        • "JobHistoryFileReplay mapper" -> "JobHistoryFileReplay mappers"
        • ".. to timeline server." -> "... to the timeline server."
        • "In the end," -> "At the end,"
        • "transaction rate(ops/s)" -> "the transaction rate (ops/s)"
        • "and transaction rate in total" -> "and the total transaction rate"
        • "print out" -> "printed out"
        • "To run the test..." -> "Running the test..."
        • "IO rate(KB/s)" -> "the I/O rate (KB/s)"
        • "IO rate total." -> "the total I/O rate."

        (Usages)

        • "Usages" -> "Usage"
        • "Each mapper write user specified number of timeline entities to timelineserver and each timeline entity is created with user specified size." -> "Each mapper writes a user-specified number of timeline entities with a user-specified size to the timeline server."
        • "Each mappe replay..." -> "Each mapper replays..."
        • "... to be replayed. suggest to launch mappers no more than..." -> "...to be replayed; the number of mappers should be no more than..."
        Show
        sjlee0 Sangjin Lee added a comment - Sorry Chang Li it took me a while to get to this. Some corrections and suggestions below. (Highlights) "Timeline..." -> "The timeline..." "help measure..." -> "helps measure..." "Test will launch..." -> "The test launches..." "JobHistoryFileReplay mapper" -> "JobHistoryFileReplay mappers" ".. to timeline server." -> "... to the timeline server." "In the end," -> "At the end," "transaction rate(ops/s)" -> "the transaction rate (ops/s)" "and transaction rate in total" -> "and the total transaction rate" "print out" -> "printed out" "To run the test..." -> "Running the test..." "IO rate(KB/s)" -> "the I/O rate (KB/s)" "IO rate total." -> "the total I/O rate." (Usages) "Usages" -> "Usage" "Each mapper write user specified number of timeline entities to timelineserver and each timeline entity is created with user specified size." -> "Each mapper writes a user-specified number of timeline entities with a user-specified size to the timeline server." "Each mappe replay..." -> "Each mapper replays..." "... to be replayed. suggest to launch mappers no more than..." -> "...to be replayed; the number of mappers should be no more than..."
        Hide
        lichangleo Chang Li added a comment -

        Thanks a lot Sangjin Lee for review! updated .3 patch and addressed your suggestions there.

        Show
        lichangleo Chang Li added a comment - Thanks a lot Sangjin Lee for review! updated .3 patch and addressed your suggestions there.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvnsite 0m 23s trunk passed
        +1 mvnsite 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 asflicense 0m 42s Patch does not generate ASF License warnings.
        1m 33s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776620/YARN-4341.3.patch
        JIRA Issue YARN-4341
        Optional Tests asflicense mvnsite
        uname Linux c50fc9bdce27 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 50edcb9
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        Max memory used 29MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9915/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvnsite 0m 23s trunk passed +1 mvnsite 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 asflicense 0m 42s Patch does not generate ASF License warnings. 1m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776620/YARN-4341.3.patch JIRA Issue YARN-4341 Optional Tests asflicense mvnsite uname Linux c50fc9bdce27 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 50edcb9 modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site Max memory used 29MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9915/console This message was automatically generated.
        Hide
        sjlee0 Sangjin Lee added a comment -

        One remaining issue:

        • in usage, "Each mappe" -> "Each mapper"
        Show
        sjlee0 Sangjin Lee added a comment - One remaining issue: in usage, "Each mappe" -> "Each mapper"
        Hide
        lichangleo Chang Li added a comment -

        Thanks for detecting the issue Sangjin Lee. updated .4 patch to fix that

        Show
        lichangleo Chang Li added a comment - Thanks for detecting the issue Sangjin Lee . updated .4 patch to fix that
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 mvnsite 0m 18s trunk passed
        +1 mvnsite 0m 18s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 asflicense 0m 26s Patch does not generate ASF License warnings.
        1m 42s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776690/YARN-4341.4.patch
        JIRA Issue YARN-4341
        Optional Tests asflicense mvnsite
        uname Linux b49a5e7be3a6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 132478e
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        Max memory used 30MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9919/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvnsite 0m 18s trunk passed +1 mvnsite 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 1m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776690/YARN-4341.4.patch JIRA Issue YARN-4341 Optional Tests asflicense mvnsite uname Linux b49a5e7be3a6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 132478e modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site Max memory used 30MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9919/console This message was automatically generated.
        Hide
        lichangleo Chang Li added a comment -

        Sangjin Lee, could you please help look if the latest patch is good? Thanks!

        Show
        lichangleo Chang Li added a comment - Sangjin Lee , could you please help look if the latest patch is good? Thanks!
        Hide
        sjlee0 Sangjin Lee added a comment -

        +1. Committing it shortly.

        Show
        sjlee0 Sangjin Lee added a comment - +1. Committing it shortly.
        Hide
        sjlee0 Sangjin Lee added a comment -

        Committed to trunk and branch-2 (2.9.0). Thanks Chang Li for your contribution!

        Show
        sjlee0 Sangjin Lee added a comment - Committed to trunk and branch-2 (2.9.0). Thanks Chang Li for your contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8956 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8956/)
        YARN-4341. add doc about timeline performance tool usage (Chang Li via (sjlee: rev e8964589de27a507c31203a87c3cbe0a8d68d3dd)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          YARN-4341 addendum. Moved the changelog entry from 2.8.0 to 2.9.0. (sjlee: rev b7b292367ff611f5025605304db9f24a6cfe5653)
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8956 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8956/ ) YARN-4341 . add doc about timeline performance tool usage (Chang Li via (sjlee: rev e8964589de27a507c31203a87c3cbe0a8d68d3dd) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md YARN-4341 addendum. Moved the changelog entry from 2.8.0 to 2.9.0. (sjlee: rev b7b292367ff611f5025605304db9f24a6cfe5653) hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #688 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/688/)
        YARN-4341. add doc about timeline performance tool usage (Chang Li via (sjlee: rev e8964589de27a507c31203a87c3cbe0a8d68d3dd)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
          YARN-4341 addendum. Moved the changelog entry from 2.8.0 to 2.9.0. (sjlee: rev b7b292367ff611f5025605304db9f24a6cfe5653)
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #688 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/688/ ) YARN-4341 . add doc about timeline performance tool usage (Chang Li via (sjlee: rev e8964589de27a507c31203a87c3cbe0a8d68d3dd) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md YARN-4341 addendum. Moved the changelog entry from 2.8.0 to 2.9.0. (sjlee: rev b7b292367ff611f5025605304db9f24a6cfe5653) hadoop-yarn-project/CHANGES.txt

          People

          • Assignee:
            lichangleo Chang Li
            Reporter:
            lichangleo Chang Li
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development