Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6377

JHS sorting on state column not working in webUi

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Steps to reproduce
      ================
      1. Install and setup HA cluster with JHS
      2.Create state in in JHS where few jobs are killed and Success

      Check sorting State in JHS WebUI

      Actual
      =====
      Sorting on state column not working in JHS

      Expected
      ======
      Sorting on state column should be working

      1. MAPREDUCE-6377.000.patch
        1 kB
        zhihai xu
      2. Sorting Issue.png
        16 kB
        Bibin A Chundatt
      3. state_sorted1.pdf
        502 kB
        zhihai xu
      4. state_sorted2.pdf
        501 kB
        zhihai xu

        Issue Links

          Activity

          Hide
          zxu zhihai xu added a comment -

          Hi Bibin A Chundatt, thanks for reporting this issue.
          I can work on this issue. It looks like the change in MAPREDUCE-5052 caused this issue.
          In MAPREDUCE-5052, "Submit Time" is added at the first column and all columns should be shifted by one. But HsView#jobsTableInit is not changed to match the new layout.

            private String jobsTableInit() {
              return tableInit().
                  append(", 'aaData': jobsTableData").
                  append(", bDeferRender: true").
                  append(", bProcessing: true").
          
                  // Sort by id upon page load
                  append(", aaSorting: [[2, 'desc']]").
                  append(", aoColumnDefs:[").
                  // Maps Total, Maps Completed, Reduces Total and Reduces Completed
                  append("{'sType':'numeric', 'bSearchable': false, 'aTargets': [ 7, 8, 9, 10 ] }").
                  append("]}").
                  toString();
            }
          

          Based on the above code, you can see column "State" which is 7 now is mistakenly considered as column "Maps Total".

          Show
          zxu zhihai xu added a comment - Hi Bibin A Chundatt , thanks for reporting this issue. I can work on this issue. It looks like the change in MAPREDUCE-5052 caused this issue. In MAPREDUCE-5052 , "Submit Time" is added at the first column and all columns should be shifted by one. But HsView#jobsTableInit is not changed to match the new layout. private String jobsTableInit() { return tableInit(). append( ", 'aaData': jobsTableData" ). append( ", bDeferRender: true " ). append( ", bProcessing: true " ). // Sort by id upon page load append( ", aaSorting: [[2, 'desc']]" ). append( ", aoColumnDefs:[" ). // Maps Total, Maps Completed, Reduces Total and Reduces Completed append( "{'sType':'numeric', 'bSearchable': false , 'aTargets': [ 7, 8, 9, 10 ] }" ). append( "]}" ). toString(); } Based on the above code, you can see column "State" which is 7 now is mistakenly considered as column "Maps Total".
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          zhihai xu thnks a lot for taking up the issue. Its all yours.
          Hoping to get patch soon .

          Show
          bibinchundatt Bibin A Chundatt added a comment - zhihai xu thnks a lot for taking up the issue. Its all yours. Hoping to get patch soon .
          Hide
          zxu zhihai xu added a comment -

          thanks Bibin A Chundatt!
          I uploaded a patch MAPREDUCE-6377.000.patch for review.
          I attached two documents to prove the jobs can be sorted on state column with this patch.

          Show
          zxu zhihai xu added a comment - thanks Bibin A Chundatt ! I uploaded a patch MAPREDUCE-6377 .000.patch for review. I attached two documents to prove the jobs can be sorted on state column with this patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 22s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 31s There were no new javac warning messages.
          +1 javadoc 9m 33s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 28s The applied patch generated 1 new checkstyle issues (total was 4, now 4).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 0m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 5m 52s Tests passed in hadoop-mapreduce-client-hs.
              42m 12s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12736603/MAPREDUCE-6377.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 63e3fee
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-hs.txt
          hadoop-mapreduce-client-hs test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 22s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 31s There were no new javac warning messages. +1 javadoc 9m 33s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 28s The applied patch generated 1 new checkstyle issues (total was 4, now 4). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 5m 52s Tests passed in hadoop-mapreduce-client-hs.     42m 12s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12736603/MAPREDUCE-6377.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 63e3fee checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-hs.txt hadoop-mapreduce-client-hs test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 42s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 45s There were no new javac warning messages.
          +1 javadoc 9m 49s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 28s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 0m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 5m 56s Tests passed in hadoop-mapreduce-client-hs.
              43m 2s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12736617/MAPREDUCE-6377.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 63e3fee
          hadoop-mapreduce-client-hs test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/testReport/
          Java 1.7.0_55
          uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 42s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 45s There were no new javac warning messages. +1 javadoc 9m 49s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 28s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 5m 56s Tests passed in hadoop-mapreduce-client-hs.     43m 2s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12736617/MAPREDUCE-6377.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 63e3fee hadoop-mapreduce-client-hs test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/console This message was automatically generated.
          Hide
          devaraj.k Devaraj K added a comment -

          I verified with the patch, it fixes the issue.

          +1, patch looks good to me.

          Show
          devaraj.k Devaraj K added a comment - I verified with the patch, it fixes the issue. +1, patch looks good to me.
          Hide
          devaraj.k Devaraj K added a comment -

          Thanks zhihai xu for the patch and Bibin A Chundatt for reporting the issue.

          Committed to trunk and branch-2.

          Show
          devaraj.k Devaraj K added a comment - Thanks zhihai xu for the patch and Bibin A Chundatt for reporting the issue. Committed to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7973 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7973/)
          MAPREDUCE-6377. JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7973 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7973/ ) MAPREDUCE-6377 . JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #2147 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2147/)
          MAPREDUCE-6377. JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2147 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2147/ ) MAPREDUCE-6377 . JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/208/)
          MAPREDUCE-6377. JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/208/ ) MAPREDUCE-6377 . JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2165 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2165/)
          MAPREDUCE-6377. JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2165 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2165/ ) MAPREDUCE-6377 . JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #217 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/217/)
          MAPREDUCE-6377. JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #217 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/217/ ) MAPREDUCE-6377 . JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          zxu zhihai xu added a comment -

          thanks Devaraj K for reviewing and committing the patch, thanks Bibin A Chundatt for reporting the issue.

          Show
          zxu zhihai xu added a comment - thanks Devaraj K for reviewing and committing the patch, thanks Bibin A Chundatt for reporting the issue.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #220 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/220/)
          MAPREDUCE-6377. JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #220 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/220/ ) MAPREDUCE-6377 . JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #950 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/950/)
          MAPREDUCE-6377. JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #950 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/950/ ) MAPREDUCE-6377 . JHS sorting on state column not working in webUi. (devaraj: rev 790a861766dcd60212d83f444f2f96d3acf20a94) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java
          Hide
          jlowe Jason Lowe added a comment -

          Thanks, zhihai xu! I committed this to branch-2.7 and branch-2.6 as well.

          Show
          jlowe Jason Lowe added a comment - Thanks, zhihai xu ! I committed this to branch-2.7 and branch-2.6 as well.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Pulled this into 2.7.2 to keep the release up-to-date with 2.6.3. Changing fix-versions to reflect the same.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.7.2 to keep the release up-to-date with 2.6.3. Changing fix-versions to reflect the same.

            People

            • Assignee:
              zxu zhihai xu
              Reporter:
              bibinchundatt Bibin A Chundatt
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development