Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1850

Include job submit host information (name and ip) in jobconf and jobdetails display

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Enhancement to identify the source (submit host and ip) of a job request.

      1. mapred-1850-5.patch
        7 kB
        Krishna Ramachandran
      2. mapred-1850-4.patch
        6 kB
        Krishna Ramachandran
      3. mapred-1850-3.patch
        6 kB
        Krishna Ramachandran
      4. mapred-1850-2.patch
        7 kB
        Krishna Ramachandran
      5. mapred-1850-1.patch
        7 kB
        Krishna Ramachandran
      6. mapred-1850.patch
        7 kB
        Krishna Ramachandran
      7. mapred-1850.patch
        6 kB
        Krishna Ramachandran

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
        Hide
        amareshwari Amareshwari Sriramadasu added a comment -

        Test failure is because of MAPREDUCE-1834.

        I just committed this. Thanks Krishna !

        Show
        amareshwari Amareshwari Sriramadasu added a comment - Test failure is because of MAPREDUCE-1834 . I just committed this. Thanks Krishna !
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12448021/mapred-1850-5.patch
        against trunk revision 958279.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448021/mapred-1850-5.patch against trunk revision 958279. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/271/console This message is automatically generated.
        Hide
        amareshwari Amareshwari Sriramadasu added a comment -

        Couldn't view the test report. Re-running through hudson.

        Show
        amareshwari Amareshwari Sriramadasu added a comment - Couldn't view the test report. Re-running through hudson.
        Hide
        ramach Krishna Ramachandran added a comment -

        can you commit and resolve?

        Show
        ramach Krishna Ramachandran added a comment - can you commit and resolve?
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12448021/mapred-1850-5.patch
        against trunk revision 957897.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448021/mapred-1850-5.patch against trunk revision 957897. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/269/console This message is automatically generated.
        Hide
        amareshwari Amareshwari Sriramadasu added a comment -

        Latest patch looks fine

        Show
        amareshwari Amareshwari Sriramadasu added a comment - Latest patch looks fine
        Hide
        ramach Krishna Ramachandran added a comment -

        ah! another inadvertent omission. Sorry. Think I got them all.
        Use of String constant has not been consistent though I have done my part.

        Show
        ramach Krishna Ramachandran added a comment - ah! another inadvertent omission. Sorry. Think I got them all. Use of String constant has not been consistent though I have done my part.
        Hide
        amareshwari Amareshwari Sriramadasu added a comment -

        JobConf still uses the configuration names, not String constants. Also, Javadoc for JobConf.getJobSubmitHostAddress() still has links.
        The new configuration should be added in MRJobConfig.java, not MRConfig.

        Show
        amareshwari Amareshwari Sriramadasu added a comment - JobConf still uses the configuration names, not String constants. Also, Javadoc for JobConf.getJobSubmitHostAddress() still has links. The new configuration should be added in MRJobConfig.java, not MRConfig.
        Hide
        ramach Krishna Ramachandran added a comment -

        I think I got all suggested changes (previous comment)

        Show
        ramach Krishna Ramachandran added a comment - I think I got all suggested changes (previous comment)
        Hide
        amareshwari Amareshwari Sriramadasu added a comment -

        Some comments on the patch:
        1. Can you add the new configuration names to MRJobConfig and use String constants every where?
        2. I don't think links given in javadoc in JobConf methods will work, as there are no such classes. I don't know why javadoc is not complaining. Can you change them to plain text instead of links?
        3. Minor : Can you remove the comment saying "// auto generated stub" from JobInProgress getters?

        Show
        amareshwari Amareshwari Sriramadasu added a comment - Some comments on the patch: 1. Can you add the new configuration names to MRJobConfig and use String constants every where? 2. I don't think links given in javadoc in JobConf methods will work, as there are no such classes. I don't know why javadoc is not complaining. Can you change them to plain text instead of links? 3. Minor : Can you remove the comment saying "// auto generated stub" from JobInProgress getters?
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12447875/mapred-1850-3.patch
        against trunk revision 957283.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12447875/mapred-1850-3.patch against trunk revision 957283. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/263/console This message is automatically generated.
        Hide
        ramach Krishna Ramachandran added a comment -

        revised patch with only JobSubmitter setting these

        Show
        ramach Krishna Ramachandran added a comment - revised patch with only JobSubmitter setting these
        Hide
        ramach Krishna Ramachandran added a comment -

        I missed that. Sorry - tested with only JobSubmitter.java setting the submit host and submit IP

        Show
        ramach Krishna Ramachandran added a comment - I missed that. Sorry - tested with only JobSubmitter.java setting the submit host and submit IP
        Hide
        acmurthy Arun C Murthy added a comment -

        Krishna - I don't see why you need to change both JobClient and JobSubmitter, shouldn't adding this in JobSubmitter suffice?

        Show
        acmurthy Arun C Murthy added a comment - Krishna - I don't see why you need to change both JobClient and JobSubmitter, shouldn't adding this in JobSubmitter suffice?
        Hide
        ramach Krishna Ramachandran added a comment -

        revised patch

        Show
        ramach Krishna Ramachandran added a comment - revised patch
        Hide
        ramach Krishna Ramachandran added a comment -

        submitting a revision

        Show
        ramach Krishna Ramachandran added a comment - submitting a revision
        Hide
        ramach Krishna Ramachandran added a comment -

        revised JobSubmitter after a svn sync

        Show
        ramach Krishna Ramachandran added a comment - revised JobSubmitter after a svn sync
        Hide
        ramach Krishna Ramachandran added a comment -

        revised and renamed

        Show
        ramach Krishna Ramachandran added a comment - revised and renamed
        Hide
        ramach Krishna Ramachandran added a comment -

        revised earlier patch. Modified JobSubmitter to update submit Host and IP

        Show
        ramach Krishna Ramachandran added a comment - revised earlier patch. Modified JobSubmitter to update submit Host and IP
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12447083/mapred-1850.patch
        against trunk revision 954364.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12447083/mapred-1850.patch against trunk revision 954364. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/571/console This message is automatically generated.
        Hide
        ramach Krishna Ramachandran added a comment -

        This is a forward port from a patch for an earlier release

        Fix for deprecated APIs
        For trunk still need to fix Job.java and Configuration

        Show
        ramach Krishna Ramachandran added a comment - This is a forward port from a patch for an earlier release Fix for deprecated APIs For trunk still need to fix Job.java and Configuration

          People

          • Assignee:
            ramach Krishna Ramachandran
            Reporter:
            ramach Krishna Ramachandran
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development