Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6714

Refactor UncompressedSplitLineReader.fillBuffer()

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.8.0, 3.0.0-alpha1
    • None
    • None
    • Reviewed

    Description

      MAPREDUCE-6635 made this change:

      -      maxBytesToRead = Math.min(maxBytesToRead,
      -                                (int)(splitLength - totalBytesRead));
      +      long leftBytesForSplit = splitLength - totalBytesRead;
      +      // check if leftBytesForSplit exceed Integer.MAX_VALUE
      +      if (leftBytesForSplit <= Integer.MAX_VALUE) {
      +        maxBytesToRead = Math.min(maxBytesToRead, (int)leftBytesForSplit);
      +      }
      

      The result is one more comparison than necessary and code that's a little convoluted. The code can be simplified as:

            long leftBytesForSplit = splitLength - totalBytesRead;
      
            if (leftBytesForSplit < maxBytesToRead) {
              maxBytesToRead = (int)leftBytesForSplit;
            }
      

      The comparison will auto promote maxBytesToRead, making it safe.

      Attachments

        1. MAPREDUCE-6714.001.patch
          1 kB
          Daniel Templeton

        Activity

          People

            templedf Daniel Templeton
            templedf Daniel Templeton
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: