Hadoop Common
  1. Hadoop Common
  2. HADOOP-5442

The job history display needs to be paged

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently the list of job history will try to render the entire list of jobs that have run. That doesn't scale up as more and more jobs run on a job tracker.

      1. HADOOP-5442-v1.9.patch
        9 kB
        Amar Kamat
      2. HADOOP-5442-v1.6.patch
        8 kB
        Amar Kamat
      3. HADOOP-5442-v1.12.patch
        9 kB
        Amar Kamat

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        7d 2h 47m 1 Arun C Murthy 16/Mar/09 21:01
        Resolved Resolved Closed Closed
        525d 23h 34m 1 Tom White 24/Aug/10 21:36
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #783 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/783/ )
        Arun C Murthy made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 0.21.0 [ 12313563 ]
        Resolution Fixed [ 1 ]
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks, Amar!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks, Amar!
        Hide
        Amar Kamat added a comment -

        Result of test-patch

        [exec] -1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no tests are needed for this patch.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     -1 findbugs.  The patch appears to introduce 1 new Findbugs warnings.
             [exec] 
             [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        

        The findbugs warning is

        HTTP parameter directly written to JSP output, giving reflected XSS vulnerability in org.apache.hadoop.mapred.jobhistory_jsp
        
        In class org.apache.hadoop.mapred.jobhistory_jsp
        In method org.apache.hadoop.mapred.jobhistory_jsp._jspService(HttpServletRequest, HttpServletResponse)
        Parameter 'search' Value generated at jobhistory_jsp.java:[line 146]
        At jobhistory_jsp.java:[line 226] 
        Another occurrence at jobhistory_jsp.java:[line 249] 
        Another occurrence at jobhistory_jsp.java:[line 253]
        Another occurrence at jobhistory_jsp.java:[line 260]
        

        Here the search string is passed across pages.

        Show
        Amar Kamat added a comment - Result of test-patch [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. The findbugs warning is HTTP parameter directly written to JSP output, giving reflected XSS vulnerability in org.apache.hadoop.mapred.jobhistory_jsp In class org.apache.hadoop.mapred.jobhistory_jsp In method org.apache.hadoop.mapred.jobhistory_jsp._jspService(HttpServletRequest, HttpServletResponse) Parameter 'search' Value generated at jobhistory_jsp.java:[line 146] At jobhistory_jsp.java:[line 226] Another occurrence at jobhistory_jsp.java:[line 249] Another occurrence at jobhistory_jsp.java:[line 253] Another occurrence at jobhistory_jsp.java:[line 260] Here the search string is passed across pages.
        Hide
        Jothi Padmanabhan added a comment -

        Patch looks fine.

        Show
        Jothi Padmanabhan added a comment - Patch looks fine.
        Amar Kamat made changes -
        Attachment HADOOP-5442-v1.12.patch [ 12402301 ]
        Hide
        Amar Kamat added a comment -

        Attaching a patch incorporating Jothi's comments and Devaraj's offline comments.

        Show
        Amar Kamat added a comment - Attaching a patch incorporating Jothi's comments and Devaraj's offline comments.
        Hide
        Jothi Padmanabhan added a comment -

        Tested the patch. Looks good. A few minor points

        1. Having a standalone : to imply allusers:alljobs caused an ArrayOutOfBounds Exception
        2. I think the quotes surrounding the job numbers in the display (Displaying '1829' jobs from '1' to '1829' out of '1829' jobs) can be removed
        3. Adding a "First Page" link similar to "Last page" might be useful. This is not important though, we could get back to the first page from the filter if this option is not there.
        4. On Choosing "Last page" option from a "Show all" result page, it gives the first page and not the last page.
        Show
        Jothi Padmanabhan added a comment - Tested the patch. Looks good. A few minor points Having a standalone : to imply allusers:alljobs caused an ArrayOutOfBounds Exception I think the quotes surrounding the job numbers in the display (Displaying '1829' jobs from '1' to '1829' out of '1829' jobs) can be removed Adding a "First Page" link similar to "Last page" might be useful. This is not important though, we could get back to the first page from the filter if this option is not there. On Choosing "Last page" option from a "Show all" result page, it gives the first page and not the last page.
        Amar Kamat made changes -
        Assignee Amar Kamat [ amar_kamat ]
        Amar Kamat made changes -
        Attachment HADOOP-5442-v1.9.patch [ 12402276 ]
        Hide
        Amar Kamat added a comment -

        Attaching a patch incorporating offline comments from Devaraj and Hemanth. The changes are as follows

        1. searching now allows jobname searching
        2. start-index is not longed used. Pageno is used to display pages
        3. A last page link is provided

        Result of test-patch is as follows :

        [exec] -1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no tests are needed for this patch.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     -1 findbugs.  The patch appears to introduce 1 new Findbugs warnings.
             [exec] 
             [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        

        Findbugs warning is because of passing the search string across calls.

        Show
        Amar Kamat added a comment - Attaching a patch incorporating offline comments from Devaraj and Hemanth. The changes are as follows searching now allows jobname searching start-index is not longed used. Pageno is used to display pages A last page link is provided Result of test-patch is as follows : [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. Findbugs warning is because of passing the search string across calls.
        Hide
        Amar Kamat added a comment -

        Currently testing the patch. Patch applies to 0.20. Result of test-patch

        [exec] -1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no tests are needed for this patch.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     -1 findbugs.  The patch appears to introduce 1 new Findbugs warnings.
             [exec] 
             [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        

        Findbugs warning is because the user name is displayed on the webpage.

        Show
        Amar Kamat added a comment - Currently testing the patch. Patch applies to 0.20. Result of test-patch [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. Findbugs warning is because the user name is displayed on the webpage.
        Amar Kamat made changes -
        Field Original Value New Value
        Attachment HADOOP-5442-v1.6.patch [ 12402244 ]
        Hide
        Amar Kamat added a comment -

        Attaching a patch that does

        1. Pagination
        2. Adds search based on user
        3. Back link from jobhistory to jobtracker

        Todo :

        1. Add searching on other parameters too like jobname.
        2. Show users-level grouping by default
        3. Group by date instead of showing all in one table

        For now we feel the current patch should serve the purpose. Todo's can be incorporated later.

        Show
        Amar Kamat added a comment - Attaching a patch that does Pagination Adds search based on user Back link from jobhistory to jobtracker Todo : Add searching on other parameters too like jobname. Show users-level grouping by default Group by date instead of showing all in one table For now we feel the current patch should serve the purpose. Todo's can be incorporated later.
        Hide
        Amar Kamat added a comment -

        HADOOP-4670 should take care of this, no?

        Show
        Amar Kamat added a comment - HADOOP-4670 should take care of this, no?
        Owen O'Malley created issue -

          People

          • Assignee:
            Amar Kamat
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development