Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2727

MR-279: SleepJob throws divide by zero exception when count = 0

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      When the count is 0 for mappers or reducers, a divide-by-zero exception is thrown. There are existing checks to error out when count < 0, which obviously doesn't handle the 0 case. This is causing the MRReliabilityTest to fail.

      1. MAPREDUCE-2727-trunk.patch
        2 kB
        Jeffrey Naisbitt
      2. MAPREDUCE-2727.patch
        3 kB
        Jeffrey Naisbitt

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12487741/MAPREDUCE-2727.patch
        against trunk revision 1150533.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/502//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12487741/MAPREDUCE-2727.patch against trunk revision 1150533. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/502//console This message is automatically generated.
        Hide
        Jeffrey Naisbitt added a comment -

        Patch is for MR-279 branch

        Show
        Jeffrey Naisbitt added a comment - Patch is for MR-279 branch
        Hide
        Robert Joseph Evans added a comment -

        +1 the patch looks fairly simple and straight forward even without tests, but it would be nice to have a unit test or two to verify that it works.

        Show
        Robert Joseph Evans added a comment - +1 the patch looks fairly simple and straight forward even without tests, but it would be nice to have a unit test or two to verify that it works.
        Hide
        Arun C Murthy added a comment -

        Jeffrey, the patch looks good.

        I think this is an appropriate time to remove the copy of SleepJob in mrv2, it's always bugged me.

        Also, will the same patch apply to trunk - can you pls check? Else, I can wait for the merge and just commit to trunk. Thoughts? Thanks.

        Show
        Arun C Murthy added a comment - Jeffrey, the patch looks good. I think this is an appropriate time to remove the copy of SleepJob in mrv2, it's always bugged me. Also, will the same patch apply to trunk - can you pls check? Else, I can wait for the merge and just commit to trunk. Thoughts? Thanks.
        Hide
        Jeffrey Naisbitt added a comment -

        Bobby Evans is actually currently working on removing the extra SleepJob - I believe there are some issues that need to be resolved there (in a separate Jira).

        As far as trunk, it does look like it has the same issue, so I'll post a patch for that as well.

        Show
        Jeffrey Naisbitt added a comment - Bobby Evans is actually currently working on removing the extra SleepJob - I believe there are some issues that need to be resolved there (in a separate Jira). As far as trunk, it does look like it has the same issue, so I'll post a patch for that as well.
        Hide
        Jeffrey Naisbitt added a comment -

        As requested, here is a patch for trunk (which only applies to the one SleepJob.java file)

        Show
        Jeffrey Naisbitt added a comment - As requested, here is a patch for trunk (which only applies to the one SleepJob.java file)
        Hide
        Arun C Murthy added a comment -

        I just committed this to both trunk and MR-279. Thanks Jeffrey!

        Show
        Arun C Murthy added a comment - I just committed this to both trunk and MR-279. Thanks Jeffrey!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #766 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/766/)
        MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals 0. Contributed by Jeffrey Naisbitt.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157422
        Files :

        • /hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/SleepJob.java
        • /hadoop/common/trunk/mapreduce/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #766 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/766/ ) MAPREDUCE-2727 . Fix divide-by-zero error in SleepJob for sleepCount equals 0. Contributed by Jeffrey Naisbitt. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157422 Files : /hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/SleepJob.java /hadoop/common/trunk/mapreduce/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #754 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/754/)
        MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals 0. Contributed by Jeffrey Naisbitt.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157422
        Files :

        • /hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/SleepJob.java
        • /hadoop/common/trunk/mapreduce/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #754 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/754/ ) MAPREDUCE-2727 . Fix divide-by-zero error in SleepJob for sleepCount equals 0. Contributed by Jeffrey Naisbitt. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157422 Files : /hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/SleepJob.java /hadoop/common/trunk/mapreduce/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #742 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/742/)
        MAPREDUCE-2727. Fix divide-by-zero error in SleepJob for sleepCount equals 0. Contributed by Jeffrey Naisbitt.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157422
        Files :

        • /hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/SleepJob.java
        • /hadoop/common/trunk/mapreduce/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #742 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/742/ ) MAPREDUCE-2727 . Fix divide-by-zero error in SleepJob for sleepCount equals 0. Contributed by Jeffrey Naisbitt. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1157422 Files : /hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/SleepJob.java /hadoop/common/trunk/mapreduce/CHANGES.txt

          People

          • Assignee:
            Jeffrey Naisbitt
            Reporter:
            Jeffrey Naisbitt
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development