Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5288

ResourceEstimator#getEstimatedTotalMapOutputSize suffers from divide by zero issues

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.2.0
    • 1.3.0
    • mrv1
    • None
    • Reviewed

    Description

      The computation in the above mentioned class-method is below:

            long estimate = Math.round(((double)inputSize * 
                completedMapsOutputSize * 2.0)/completedMapsInputSize);
      

      Given http://docs.oracle.com/javase/6/docs/api/java/lang/Math.html#round(double), its possible that the returned estimate could be Long.MAX_VALUE if completedMapsInputSize is determined to be zero.

      This can be proven with a simple code snippet:

      class Foo {
          public static void main(String... args) {
              long inputSize = 600L + 2;
              long estimate = Math.round(((double)inputSize *
                                    1L * 2.0)/0L);
              System.out.println(estimate);
          }
      }
      

      The above conveniently prints out: 9223372036854775807, which is Long.MAX_VALUE (or 8 Exbibytes per MapReduce).

      Attachments

        1. mr-5288-1.patch
          5 kB
          Karthik Kambatla

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            kasha Karthik Kambatla
            qwertymaniac Harsh J
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment