Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5023

History Server Web Services missing Job Counters

    Details

      Description

      The History Server's Job Counters API is not returning all the counters seen on the Job's Counters webpage. Specifically, I'm not seeing any of the counters in the "org.apache.hadoop.mapreduce.JobCounter" group:

      TOTAL_LAUNCHED_MAPS
      TOTAL_LAUNCHED_REDUCES
      OTHER_LOCAL_MAPS
      SLOTS_MILLIS_MAPS
      SLOTS_MILLIS_REDUCES

      1. MAPREDUCE-5023.patch
        2 kB
        Ravi Prakash
      2. MAPREDUCE-5023.patch
        2 kB
        Ravi Prakash

        Issue Links

          Activity

          Hide
          Ravi Prakash added a comment -

          4102 added the job counters to the Web UI but forgot to add it to the Web Services

          Show
          Ravi Prakash added a comment - 4102 added the job counters to the Web UI but forgot to add it to the Web Services
          Hide
          Ravi Prakash added a comment -

          Simple enough patch. There's some code duplication between getCounters() in CountersBlock.java JobCounterInfo.java but I don't know if it can be helped.

          Show
          Ravi Prakash added a comment - Simple enough patch. There's some code duplication between getCounters() in CountersBlock.java JobCounterInfo.java but I don't know if it can be helped.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12572564/MAPREDUCE-5023.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3390//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3390//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12572564/MAPREDUCE-5023.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3390//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3390//console This message is automatically generated.
          Hide
          Thomas Graves added a comment -

          Ravi can you please make sure this works for the AM also.

          Show
          Thomas Graves added a comment - Ravi can you please make sure this works for the AM also.
          Hide
          Ravi Prakash added a comment -

          Thanks Tom. This new patch makes it work for both the HS and AM

          Show
          Ravi Prakash added a comment - Thanks Tom. This new patch makes it work for both the HS and AM
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12572613/MAPREDUCE-5023.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3391//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3391//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12572613/MAPREDUCE-5023.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3391//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3391//console This message is automatically generated.
          Hide
          Thomas Graves added a comment -

          Thanks Ravi! +1 on the first patch. It works for me on AM and HS. I'll commit shortly

          Show
          Thomas Graves added a comment - Thanks Ravi! +1 on the first patch. It works for me on AM and HS. I'll commit shortly
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3437 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3437/)
          MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156)

          Result = SUCCESS
          tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3437 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3437/ ) MAPREDUCE-5023 . History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #149 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/149/)
          MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156)

          Result = SUCCESS
          tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #149 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/149/ ) MAPREDUCE-5023 . History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #547 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/547/)
          MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454158)

          Result = SUCCESS
          tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454158
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #547 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/547/ ) MAPREDUCE-5023 . History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454158) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454158 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1338 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1338/)
          MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156)

          Result = SUCCESS
          tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1338 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1338/ ) MAPREDUCE-5023 . History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1366 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1366/)
          MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156)

          Result = SUCCESS
          tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1366 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1366/ ) MAPREDUCE-5023 . History Server Web Services missing Job Counters (Ravi Prakash via tgraves) (Revision 1454156) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1454156 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/JobCounterInfo.java

            People

            • Assignee:
              Ravi Prakash
              Reporter:
              Kendall Thrapp
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development