Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5569

FloatSplitter is not generating correct splits

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta, 1.3.0, 0.23.9
    • Fix Version/s: 1.3.0, 0.23.10, 2.3.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The closing split is not calculated correctly:

           // Catch any overage and create the closed interval for the last split.
           if (curLower <= maxVal || splits.size() == 1) {
             splits.add(new DataDrivenDBInputFormat.DataDrivenDBInputSplit(
      -          lowClausePrefix + Double.toString(curUpper),
      +          lowClausePrefix + Double.toString(curLower),
                 colName + " <= " + Double.toString(maxVal)));
           }
      

      For the case of min=5.0, max=7.0, 2 splits, the current code returns splits of (column1 >=5.0, column1 <6.0), (column1 >=7.0, column1 <=7.0). The second split is obviously not correct.

        Attachments

        1. MAPREDUCE-5569-trunk.patch
          1 kB
          Nathan Roberts
        2. MAPREDUCE-5569-branch-1.patch
          7 kB
          Nathan Roberts

          Issue Links

            Activity

              People

              • Assignee:
                nroberts Nathan Roberts
                Reporter:
                nroberts Nathan Roberts
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: