MRUnit
  1. MRUnit
  2. MRUNIT-88

MRUnit should support custom sorting and grouping comparators

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not A Problem
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      We're building something that essentially does a secondary sort, to test that we need to be able to specify comparators and partitioners.
      Example:
      the following two tuple keys: (id1, source1), (id1, source2)
      should be grouped together based on the first value of the tuple, and their records should end up in the same reducer

      To do this we have our own custom partitioner / comparator, this is what we need to test through the whole pipeline in this way:

      MapReduceDriver.setPartitioner(p)
      MapReduceDriver.setGroupComparator(c)

      I'm not familiar enough with the MRUnit code to add this easily, but I suspect it would be pretty quick to do.

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Matthew Rathbone
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 24h
              24h
              Remaining:
              Remaining Estimate - 24h
              24h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development