MRUnit
  1. MRUnit
  2. MRUNIT-12

Support MultipleOutputs test, load dataset from text,and some usefull annotation

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.5.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      Some time we need to test reduce/mapreduce which used MultipleOutputs to output,mrunit not support this yet.so, I writed some code support this;
      improvement which make mrunit nicer to use:load test dataset from text and some usefull annotation

      example:

      public void testReduce ()

      { List<TimeInfo> values = new ArrayList<TimeInfo>(); values.add(new TimeInfo(1, 3)); values.add(new TimeInfo(2, 5)); values.add(new TimeInfo(3, 7)); reduceDriver.withReducer(reducer) .withInput(new Text("444"), values) . withMutiOutput ("somePrefix"+444%8,new Text("444"),new LongWritable(2)) .runTest(); }

      load test data from text and use annotation:

      @RunWith(MRUnitJunit4TestClassRunner.class)
      public class XXXMRUseAnnotationTest {
      @MapInputSet
      @MapReduce(mapper = Map.class, reducer = Reduce.class)
      private MapReduceDriver<LongWritable, Text, Text, TimeInfo, Text, LongWritable> mrDriver;

      @Test
      @MapInputSet("ConsignTimeMRUseAnnotationTest.txt")
      public void testMapReduce_3record_1user()

      { mrDriver. withMutiOutput ("somePrefix"+444%8,new Text("444"),new LongWritable(2)) .runTest(); }

      }

      1. mrunit_enhance2.zip
        16 kB
        JenWang
      2. mrunit_enhance.zip
        16 kB
        JenWang

        Activity

        Hide
        JenWang added a comment -

        I have writed code to support this

        Show
        JenWang added a comment - I have writed code to support this
        Hide
        JenWang added a comment -

        the code resolve this issues

        Show
        JenWang added a comment - the code resolve this issues
        Hide
        E. Sammer added a comment -

        Assigning to myself so we can track the patch.

        Show
        E. Sammer added a comment - Assigning to myself so we can track the patch.
        Hide
        E. Sammer added a comment -

        Marking this for the 0.5.0 release.

        Show
        E. Sammer added a comment - Marking this for the 0.5.0 release.
        Hide
        E. Sammer added a comment -

        Jen:

        I haven't yet looked at the patch (on purpose) because you haven't granted the ASF a license. This would prevent us from including the code in mrunit. If you'd like to contribute the code, please reupload the patch(es) with the checkbox labeled "Grant license to ASF for inclusion in ASF works (as per the Apache License §5)" checked.

        Thanks!

        Show
        E. Sammer added a comment - Jen: I haven't yet looked at the patch (on purpose) because you haven't granted the ASF a license. This would prevent us from including the code in mrunit. If you'd like to contribute the code, please reupload the patch(es) with the checkbox labeled "Grant license to ASF for inclusion in ASF works (as per the Apache License §5)" checked. Thanks!
        Hide
        JenWang added a comment -

        ok,i reupload it

        Show
        JenWang added a comment - ok,i reupload it
        Hide
        Jim Donofrio added a comment -

        duplicate of MRUNIT-13

        Show
        Jim Donofrio added a comment - duplicate of MRUNIT-13

          People

          • Assignee:
            E. Sammer
            Reporter:
            JenWang
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development