Uploaded image for project: 'MRUnit'
  1. MRUnit
  2. MRUNIT-127

Key grouping with GroupingComparators is not consistent with MapReduce behaviour

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 1.0.0
    • Labels:
      None

      Description

      In Hadoop MapReduce, for a set of keys to be properly grouped together by a grouping comparator into a reduce call, they need to be in a contiguous range when sorted by the key's ordering comparator.

      MRUnit does not impose this requirement, so if the user's grouping and sorting comparator logic is incorrect, their tests may pass and give the expected result even though the outcome would be different when run as a true MapReduce job.

      (see below for further explanation)

        Attachments

          Activity

            People

            • Assignee:
              dbeech Dave Beech
              Reporter:
              dbeech Dave Beech
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: