Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1612

job conf file is not accessible from job history web page

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed a bug related to access of job_conf.xml from the history web page of a job.

      Description

      Clicking on conf file link from job history web page is causing an NPE if history file(and the job conf file) are stored on DFS. This NPE is from jobconf_history.jsp because jobConf built from path on DFS is not having any properties.

      1. MR-1619-1.patch
        2 kB
        Sreekanth Ramakrishnan
      2. jobconf_history_jsp.fix.20S.patch
        2 kB
        Ravi Gummadi

        Activity

        Ravi Gummadi created issue -
        Hide
        Ravi Gummadi added a comment -

        jobconf_history.jsp is accessing conf file on DFS for reading job view ACLs. Looks like reading these ACLs from history file seems to be a better way.

        Show
        Ravi Gummadi added a comment - jobconf_history.jsp is accessing conf file on DFS for reading job view ACLs. Looks like reading these ACLs from history file seems to be a better way.
        Hide
        Ravi Gummadi added a comment -

        Attaching patch for earlier version of hadoop fixing the issue. Not for commit here.

        Show
        Ravi Gummadi added a comment - Attaching patch for earlier version of hadoop fixing the issue. Not for commit here.
        Ravi Gummadi made changes -
        Field Original Value New Value
        Attachment jobconf_history_jsp.fix.20S.patch [ 12439310 ]
        Hide
        Ravi Gummadi added a comment -

        loadResource(Path) in Configuration.java is assuming that only paths that can come to this method can be on local file system. But jobconf_history.jsp sends Path that is there on DFS causing this NPE. As it is assuming the path to be local file system, looks like fixing Configuration.java seems to be not possible.

        Regarding getting job view ACLs from history file: As getting job view ACLs from conf file is also going to DFS twice(once for loading the JobConf, and one more time for passing it to the xml formatter), getting job view ACLs from history file also seems reasonable. Also, since jobconf_history.jsp is the only JSP that is currently getting ACLs from conf file and every other history related JSP is getting the ACLs from history file itself, making this change would make things consistent also.

        Show
        Ravi Gummadi added a comment - loadResource(Path) in Configuration.java is assuming that only paths that can come to this method can be on local file system. But jobconf_history.jsp sends Path that is there on DFS causing this NPE. As it is assuming the path to be local file system, looks like fixing Configuration.java seems to be not possible. Regarding getting job view ACLs from history file: As getting job view ACLs from conf file is also going to DFS twice(once for loading the JobConf, and one more time for passing it to the xml formatter), getting job view ACLs from history file also seems reasonable. Also, since jobconf_history.jsp is the only JSP that is currently getting ACLs from conf file and every other history related JSP is getting the ACLs from history file itself, making this change would make things consistent also.
        Hide
        Sreekanth Ramakrishnan added a comment -

        Attaching patch for trunk.

        Show
        Sreekanth Ramakrishnan added a comment - Attaching patch for trunk.
        Sreekanth Ramakrishnan made changes -
        Attachment MR-1619-1.patch [ 12442668 ]
        Sreekanth Ramakrishnan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12442668/MR-1619-1.patch
        against trunk revision 937201.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442668/MR-1619-1.patch against trunk revision 937201. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/132/console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Patch looks good. +1.

        TestJobACLs is because of MAPREDUCE-1727.

        Giong to commit this. Will rerun through Hudson to be sure.

        Show
        Vinod Kumar Vavilapalli added a comment - Patch looks good. +1. TestJobACLs is because of MAPREDUCE-1727 . Giong to commit this. Will rerun through Hudson to be sure.
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        TestJobACLs is because of MAPREDUCE-1727.

        I meant TestJobACLs failure is because of MAPREDUCE-1727.

        Show
        Vinod Kumar Vavilapalli added a comment - TestJobACLs is because of MAPREDUCE-1727 . I meant TestJobACLs failure is because of MAPREDUCE-1727 .
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags [Reviewed]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12442668/MR-1619-1.patch
        against trunk revision 937924.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442668/MR-1619-1.patch against trunk revision 937924. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/138/console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Same failure with TestJobACLs. I am going to commit this.

        Show
        Vinod Kumar Vavilapalli added a comment - Same failure with TestJobACLs. I am going to commit this.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        I just committed this. Thanks Ravi and Sreekanth!

        Show
        Vinod Kumar Vavilapalli added a comment - I just committed this. Thanks Ravi and Sreekanth!
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Release Note Fixed a bug related to access of job_conf.xml from the history web page of a job.
        Resolution Fixed [ 1 ]
        Tom White made changes -
        Fix Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.21.0 [ 12314045 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Ravi Gummadi
            Reporter:
            Ravi Gummadi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development