Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1725

Fix MapReduce API incompatibilities between 0.20 and 0.21

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      A few API compatibilities have crept in since 0.20 (they are being tracked in MAPREDUCE-1623). These should be fixed before 0.21 is released.

      1. MAPREDUCE-1725.patch
        8 kB
        Tom White
      2. MAPREDUCE-1725.patch
        8 kB
        Tom White
      3. MAPREDUCE-1725.patch
        6 kB
        Tom White

        Activity

        Tom White created issue -
        Hide
        Tom White added a comment -

        Patch which fixes mapred.FileOutputFormat where a different FileAlreadyExistsException was used (from 0.20 to trunk). Also adds back some deprecated methods that were removed from TaskAttemptID and TaskID. These regressions were found using JDiff in the process of working on MAPREDUCE-1623.

        Show
        Tom White added a comment - Patch which fixes mapred.FileOutputFormat where a different FileAlreadyExistsException was used (from 0.20 to trunk). Also adds back some deprecated methods that were removed from TaskAttemptID and TaskID. These regressions were found using JDiff in the process of working on MAPREDUCE-1623 .
        Tom White made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-1725.patch [ 12442910 ]
        Tom White made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.21.0 [ 12314045 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12442910/MAPREDUCE-1725.patch
        against trunk revision 938249.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442910/MAPREDUCE-1725.patch against trunk revision 938249. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/143/console This message is automatically generated.
        Hide
        Tom White added a comment -

        Test failure is unrelated (MAPREDUCE-1727). No new tests for adding back removed deprecated methods (they delegate to new ones) - I tested for compatibility using JDiff in MAPREDUCE-1623.

        Show
        Tom White added a comment - Test failure is unrelated ( MAPREDUCE-1727 ). No new tests for adding back removed deprecated methods (they delegate to new ones) - I tested for compatibility using JDiff in MAPREDUCE-1623 .
        Hide
        Tom White added a comment -

        Added back some more methods that were removed from 0.20.

        Show
        Tom White added a comment - Added back some more methods that were removed from 0.20.
        Tom White made changes -
        Attachment MAPREDUCE-1725.patch [ 12443140 ]
        Tom White made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Tom White made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Tom White added a comment -

        Updated to apply to trunk. I'd appreciate it if someone could review this. Thanks.

        Show
        Tom White added a comment - Updated to apply to trunk. I'd appreciate it if someone could review this. Thanks.
        Tom White made changes -
        Attachment MAPREDUCE-1725.patch [ 12445805 ]
        Hide
        Chris Douglas added a comment -

        This includes MAPREDUCE-1045, except for the following:

        +  public static Path[] getInputPaths(JobConf job) throws IOException {
        +    return CombineFileInputFormat.getInputPaths(new Job(job));
        +  }
        

        It's probably not worth restoring.

        +1

        Show
        Chris Douglas added a comment - This includes MAPREDUCE-1045 , except for the following: + public static Path[] getInputPaths(JobConf job) throws IOException { + return CombineFileInputFormat.getInputPaths(new Job(job)); + } It's probably not worth restoring. +1
        Hide
        Tom White added a comment -

        I've just committed this.

        BTW I can't see the getInputPaths() method on the 0.20 version of CombineFileInputFormat, so I don't think it's a compatibility issue.

        Show
        Tom White added a comment - I've just committed this. BTW I can't see the getInputPaths() method on the 0.20 version of CombineFileInputFormat, so I don't think it's a compatibility issue.
        Tom White made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        2d 4h 49m 1 Tom White 29/Apr/10 05:10
        Open Open Patch Available Patch Available
        3d 2h 26m 2 Tom White 29/Apr/10 05:10
        Patch Available Patch Available Resolved Resolved
        39d 18h 10m 1 Tom White 07/Jun/10 23:21
        Resolved Resolved Closed Closed
        77d 23h 1 Tom White 24/Aug/10 22:21

          People

          • Assignee:
            Tom White
            Reporter:
            Tom White
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development