Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1196

MAPREDUCE-947 incompatibly changed FileOutputCommitter

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: client
    • Labels:
      None

      Description

      MAPREDUCE-947 unfortunately removed FileOutputCommitter.cleanupJob and doesn't call the deprecated method from the base-class i.e. OutputCommitter.cleanupJob; this means that applications which derive FileOutputCommitter.cleanupJob are now broken.

      1. MAPREDUCE-1196_yhadoop20.patch
        17 kB
        Arun C Murthy
      2. MAPREDUCE-1196_yhadoop20.patch
        3 kB
        Arun C Murthy
      3. MAPREDUCE-1196.patch
        16 kB
        Arun C Murthy
      4. MAPREDUCE-1196.patch
        3 kB
        Arun C Murthy

        Activity

        Hide
        acmurthy Arun C Murthy added a comment -

        I wish I had realised this while committing MAPREDUCE-947, sigh. My bad!

        Show
        acmurthy Arun C Murthy added a comment - I wish I had realised this while committing MAPREDUCE-947 , sigh. My bad!
        Hide
        acmurthy Arun C Murthy added a comment -

        Straight-forward fix while I continue testing.

        Show
        acmurthy Arun C Murthy added a comment - Straight-forward fix while I continue testing.
        Hide
        acmurthy Arun C Murthy added a comment -

        Added a test case to ensure for that deprecated cleanupJob is called from FileOutputFormat. Thanks to Amareshwari for helping to port TestJobCleanup to the new context objects api.

        Show
        acmurthy Arun C Murthy added a comment - Added a test case to ensure for that deprecated cleanupJob is called from FileOutputFormat. Thanks to Amareshwari for helping to port TestJobCleanup to the new context objects api.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12424353/MAPREDUCE-1196.patch
        against trunk revision 833990.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12424353/MAPREDUCE-1196.patch against trunk revision 833990. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/130/console This message is automatically generated.
        Hide
        chris.douglas Chris Douglas added a comment -

        +1

        Show
        chris.douglas Chris Douglas added a comment - +1
        Hide
        acmurthy Arun C Murthy added a comment -

        I just committed this.

        Show
        acmurthy Arun C Murthy added a comment - I just committed this.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #115 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/115/)
        . Fix FileOutputCommitter to use the deprecated cleanupJob api correctly.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #115 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/115/ ) . Fix FileOutputCommitter to use the deprecated cleanupJob api correctly.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #139 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/139/)
        . Fix FileOutputCommitter to use the deprecated cleanupJob api correctly.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #139 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/139/ ) . Fix FileOutputCommitter to use the deprecated cleanupJob api correctly.

          People

          • Assignee:
            acmurthy Arun C Murthy
            Reporter:
            acmurthy Arun C Murthy
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development