Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5569

FloatSplitter is not generating correct splits

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.1.0-beta, 1.3.0, 0.23.9
    • 1.3.0, 0.23.10, 2.3.0
    • None
    • None
    • Reviewed

    Description

      The closing split is not calculated correctly:

           // Catch any overage and create the closed interval for the last split.
           if (curLower <= maxVal || splits.size() == 1) {
             splits.add(new DataDrivenDBInputFormat.DataDrivenDBInputSplit(
      -          lowClausePrefix + Double.toString(curUpper),
      +          lowClausePrefix + Double.toString(curLower),
                 colName + " <= " + Double.toString(maxVal)));
           }
      

      For the case of min=5.0, max=7.0, 2 splits, the current code returns splits of (column1 >=5.0, column1 <6.0), (column1 >=7.0, column1 <=7.0). The second split is obviously not correct.

      Attachments

        1. MAPREDUCE-5569-trunk.patch
          1 kB
          Nathan Roberts
        2. MAPREDUCE-5569-branch-1.patch
          7 kB
          Nathan Roberts

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            nroberts Nathan Roberts
            nroberts Nathan Roberts
            Votes:
            0 Vote for this issue
            Watchers:
            3 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment