Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.20.3, 0.20.204.0
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      JobContext is used for a read-only view of job's info. Hence all the readonly fields in JobContext are set in the constructor. Job extends JobContext. When a Job is created, jobid is not known and hence there is no way to set JobID once Job is created. JobID is obtained only when the JobClient queries the jobTracker for a job-id., which happens later i.e upon job submission.

      1. patch-118-5.txt
        23 kB
        Amareshwari Sriramadasu
      2. patch-118-4.txt
        23 kB
        Amareshwari Sriramadasu
      3. patch-118-3.txt
        23 kB
        Amareshwari Sriramadasu
      4. patch-118-2.txt
        22 kB
        Amareshwari Sriramadasu
      5. patch-118-1.txt
        23 kB
        Amareshwari Sriramadasu
      6. patch-118-0.21.txt
        10 kB
        Amareshwari Sriramadasu
      7. patch-118-0.20-1.txt
        2 kB
        Amareshwari Sriramadasu
      8. patch-118-0.20.txt
        1 kB
        Amareshwari Sriramadasu
      9. patch-118.txt
        12 kB
        Amareshwari Sriramadasu
      10. MAPREDUCE-118--0-20-3.patch
        3 kB
        Dick King

        Issue Links

          Activity

          Hide
          Owen O'Malley added a comment -

          Hadoop 0.20.204.0 was just released.

          Show
          Owen O'Malley added a comment - Hadoop 0.20.204.0 was just released.
          Hide
          Chris Douglas added a comment -

          I committed the ported fix to branch-0.20-security. Thanks, Dick!

          Show
          Chris Douglas added a comment - I committed the ported fix to branch-0.20-security. Thanks, Dick!
          Hide
          Dick King added a comment -

          for version 20.3

          Show
          Dick King added a comment - for version 20.3
          Hide
          Sharad Agarwal added a comment -

          I just committed this to 0.20, 0.21 and 0.22 branches. Thanks Amareshwari.

          Show
          Sharad Agarwal added a comment - I just committed this to 0.20, 0.21 and 0.22 branches. Thanks Amareshwari.
          Hide
          Sharad Agarwal added a comment -

          +1

          Show
          Sharad Agarwal added a comment - +1
          Hide
          Amareshwari Sriramadasu added a comment -

          The test TestMapredHeartbeat failed with IllegalMonitorException while shutting down DataNode. The failure is not related to the patch.
          The same test passed on my machine.

          Show
          Amareshwari Sriramadasu added a comment - The test TestMapredHeartbeat failed with IllegalMonitorException while shutting down DataNode. The failure is not related to the patch. The same test passed on my machine.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12444776/patch-118-5.txt
          against trunk revision 944427.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 27 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12444776/patch-118-5.txt against trunk revision 944427. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 27 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/539/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Tests failed because of NoClassDefFoundError. Rerunning through hudson.

          Show
          Amareshwari Sriramadasu added a comment - Tests failed because of NoClassDefFoundError. Rerunning through hudson.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12444776/patch-118-5.txt
          against trunk revision 944427.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 27 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12444776/patch-118-5.txt against trunk revision 944427. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 27 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/190/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          patch-118-5.txt applies to trunk and patch-118-4.txt applies to branch 0.21

          Show
          Amareshwari Sriramadasu added a comment - patch-118-5.txt applies to trunk and patch-118-4.txt applies to branch 0.21
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch updated to trunk

          Show
          Amareshwari Sriramadasu added a comment - Patch updated to trunk
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch does not apply to trunk

          Show
          Amareshwari Sriramadasu added a comment - Patch does not apply to trunk
          Hide
          Amareshwari Sriramadasu added a comment -

          Should we override getJobID() in Job and do ensureState before doing super.getJobID() ?

          I had this in my earlier patch. But have seen problems when user calls getJobID() from his InputFormat.getSplis(JobContext) and etc, though the JobID is available by that time.

          Show
          Amareshwari Sriramadasu added a comment - Should we override getJobID() in Job and do ensureState before doing super.getJobID() ? I had this in my earlier patch. But have seen problems when user calls getJobID() from his InputFormat.getSplis(JobContext) and etc, though the JobID is available by that time.
          Hide
          Sharad Agarwal added a comment -

          Should we override getJobID() in Job and do ensureState before doing super.getJobID() ? This will give the consistent error message to user instead of returning null in some cases.

          Show
          Sharad Agarwal added a comment - Should we override getJobID() in Job and do ensureState before doing super.getJobID() ? This will give the consistent error message to user instead of returning null in some cases.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch for branch 0.20.
          Ran test-patch and ant test with the patch. There were a couple of test failure in dfs, they failed without the patch also.

          Show
          Amareshwari Sriramadasu added a comment - Patch for branch 0.20. Ran test-patch and ant test with the patch. There were a couple of test failure in dfs, they failed without the patch also.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12443941/patch-118-4.txt
          against trunk revision 941564.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 27 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443941/patch-118-4.txt against trunk revision 941564. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 27 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/176/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Test failures seem unrelated. The failures are because of NoClassDefFoundError.
          Rerunning through hudson

          Show
          Amareshwari Sriramadasu added a comment - Test failures seem unrelated. The failures are because of NoClassDefFoundError. Rerunning through hudson
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12443941/patch-118-4.txt
          against trunk revision 941564.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 27 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443941/patch-118-4.txt against trunk revision 941564. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 27 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/175/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          patch-118-4.txt applies to branch 0.21 also. All the tests passed on branch 0.21

          Show
          Amareshwari Sriramadasu added a comment - patch-118-4.txt applies to branch 0.21 also. All the tests passed on branch 0.21
          Hide
          Amareshwari Sriramadasu added a comment -

          Missed another test failure in earlier patch. Fixed that.

          Show
          Amareshwari Sriramadasu added a comment - Missed another test failure in earlier patch. Fixed that.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch fixes a test case failure present with earlier patch.

          Show
          Amareshwari Sriramadasu added a comment - Patch fixes a test case failure present with earlier patch.
          Hide
          Amareshwari Sriramadasu added a comment -

          Moved the setting of JobID from Job's constructor to submit method, because after MAPREDUCE-1505, the proxy should be created lazily at the submit time. JobID is not available if the job is not submitted.

          Main code changes are in Job.java and JobContextImpl.java. Other changes are replacing calls to getID() with getJobID().

          Show
          Amareshwari Sriramadasu added a comment - Moved the setting of JobID from Job's constructor to submit method, because after MAPREDUCE-1505 , the proxy should be created lazily at the submit time. JobID is not available if the job is not submitted. Main code changes are in Job.java and JobContextImpl.java. Other changes are replacing calls to getID() with getJobID().
          Hide
          Amareshwari Sriramadasu added a comment -

          The solution proposed for trunk is "Get the jobid for a job in constructor itself", but this contradicts with MAPREDUCE-1505, which creates the RPC client lazily. We have to rethink about the solution here. We might have to set the jobid after the job submission as Thomas suggested.

          Thoughts?

          Show
          Amareshwari Sriramadasu added a comment - The solution proposed for trunk is "Get the jobid for a job in constructor itself", but this contradicts with MAPREDUCE-1505 , which creates the RPC client lazily. We have to rethink about the solution here. We might have to set the jobid after the job submission as Thomas suggested. Thoughts?
          Hide
          Tom White added a comment -

          +1 to Amareshwari's changes. They fix Job#getJobID() within the current (0.20) API. I think patch-118-0.20.txt can be committed to the 0.20 branch, and patch-118-1.txt to trunk (which will go into the forthcoming 0.21 release).

          Show
          Tom White added a comment - +1 to Amareshwari's changes. They fix Job#getJobID() within the current (0.20) API. I think patch-118-0.20.txt can be committed to the 0.20 branch, and patch-118-1.txt to trunk (which will go into the forthcoming 0.21 release).
          Hide
          Thomas Kathmann added a comment -

          Isn't it just possible to add a getter for RunningJob to JobContext or to use the fix I have suggested earlier? (i.e. set the jobId of Job class after Job is submitted)

          Currently I don't know any way in 0.20.x to get the JobID of a job besides manually patching hadoop-core jar

          Show
          Thomas Kathmann added a comment - Isn't it just possible to add a getter for RunningJob to JobContext or to use the fix I have suggested earlier? (i.e. set the jobId of Job class after Job is submitted) Currently I don't know any way in 0.20.x to get the JobID of a job besides manually patching hadoop-core jar
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12436821/patch-118-1.txt
          against trunk revision 915223.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 33 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12436821/patch-118-1.txt against trunk revision 915223. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 33 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/478/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Here is patch fixing the issue with above proposal.

          Main code changes are in Cluster.java, Job.java and JobContextImpl.java. Other changes are replacing calls to getID() with getJobID(); handling interruptedException from Job's constructor. Also, fixes a bug in LocalJobRunner.kill().

          Show
          Amareshwari Sriramadasu added a comment - Here is patch fixing the issue with above proposal. Main code changes are in Cluster.java, Job.java and JobContextImpl.java. Other changes are replacing calls to getID() with getJobID(); handling interruptedException from Job's constructor. Also, fixes a bug in LocalJobRunner.kill().
          Hide
          Amareshwari Sriramadasu added a comment -

          The proposal looks fine.
          But I found small issue implementing it. In 0.21, ClientProtocol.getNewJobID() throws InterruptedException out. The new Job constructors(introduced in 0.21) can be changed to throw InterruptedException. But, the deprecated constructors cannot be changed.

          After discussing with Arun, one solution we could think of is "add a deprecated setJobID in JobContextImpl, which can be called from deprecated constructors. Will remove the newly added method, when we remove the deprecated constructors".

          Show
          Amareshwari Sriramadasu added a comment - The proposal looks fine. But I found small issue implementing it. In 0.21, ClientProtocol.getNewJobID() throws InterruptedException out. The new Job constructors(introduced in 0.21) can be changed to throw InterruptedException. But, the deprecated constructors cannot be changed. After discussing with Arun, one solution we could think of is "add a deprecated setJobID in JobContextImpl, which can be called from deprecated constructors. Will remove the newly added method, when we remove the deprecated constructors".
          Hide
          Arun C Murthy added a comment -

          I think you either need to give it the jobid or it gets one from the job tracker in the constructor.

          Yep, exactly - ClientProtocol.getNewJobID talks to the JT to get the jobid.

          Show
          Arun C Murthy added a comment - I think you either need to give it the jobid or it gets one from the job tracker in the constructor. Yep, exactly - ClientProtocol.getNewJobID talks to the JT to get the jobid.
          Hide
          Owen O'Malley added a comment -

          +1

          I think you either need to give it the jobid or it gets one from the job tracker in the constructor.

          Show
          Owen O'Malley added a comment - +1 I think you either need to give it the jobid or it gets one from the job tracker in the constructor.
          Hide
          Arun C Murthy added a comment -

          I have a slightly simpler proposal: how about calling ClientProtocol.getNewJobID in the constructor of Job and passing that to JobContext instead? That way we do not need any more smarts... Owen?

          Show
          Arun C Murthy added a comment - I have a slightly simpler proposal: how about calling ClientProtocol.getNewJobID in the constructor of Job and passing that to JobContext instead? That way we do not need any more smarts... Owen?
          Hide
          Amareshwari Sriramadasu added a comment -

          Cancelling patch to fix test failures

          Show
          Amareshwari Sriramadasu added a comment - Cancelling patch to fix test failures
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12426883/patch-118.txt
          against trunk revision 887135.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12426883/patch-118.txt against trunk revision 887135. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/290/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch for branch 0.21 and trunk, renaming getID to getJobID, sothat it overrides the method in JobContext.

          Show
          Amareshwari Sriramadasu added a comment - Patch for branch 0.21 and trunk, renaming getID to getJobID, sothat it overrides the method in JobContext.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch for branch 0.20

          Show
          Amareshwari Sriramadasu added a comment - Patch for branch 0.20
          Hide
          YongChul Kwon added a comment -

          With the new API, JobID is only available through JobClient instance which is protected by Job class. The mapreduce.Job.getJobID() should override JobContext.getJobID() to read it form JobClient once it is submitted.

          Show
          YongChul Kwon added a comment - With the new API, JobID is only available through JobClient instance which is protected by Job class. The mapreduce.Job.getJobID() should override JobContext.getJobID() to read it form JobClient once it is submitted.
          Hide
          Amareshwari Sriramadasu added a comment -

          JobID is accessible from job.getID(). Should we remove the method getJobID() from JobContext? and suggest users to use getID?

          Show
          Amareshwari Sriramadasu added a comment - JobID is accessible from job.getID(). Should we remove the method getJobID() from JobContext? and suggest users to use getID?
          Hide
          Amar Kamat added a comment -

          What are we trying to solve here? What feature do we need? Or what bug are we fixing?

          Configuration conf = new Configuration();
          // populate conf for some job
          Job job = new Job(conf);
          job.submit();
          // what should be the expected value of job.getJobID() ?
          // As of today we get null
          
          Show
          Amar Kamat added a comment - What are we trying to solve here? What feature do we need? Or what bug are we fixing? Configuration conf = new Configuration(); // populate conf for some job Job job = new Job(conf); job.submit(); // what should be the expected value of job.getJobID() ? // As of today we get null
          Hide
          Arun C Murthy added a comment -

          What are we trying to solve here? What feature do we need? Or what bug are we fixing?

          Show
          Arun C Murthy added a comment - What are we trying to solve here? What feature do we need? Or what bug are we fixing?

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development