MRUnit
  1. MRUnit
  2. MRUNIT-91

runTest() should optionally ignore output order

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.8.1
    • Fix Version/s: 0.9.0
    • Labels:

      Description

      Currently MapDriver.runTest() assumes that the order of pairs emitted by the mapper matches the order of the MapDriver.addOutput() calls. However, there are valid mappers that for a given input pair produce output pairs whose order is unspecified for testing purposes. (For example, if the mapper being tested uses a set object for deduplication before emission.) runTest() cannot be used to test these kinds of mappers.

      A workaround is to not use runTest() but instead put the output of run() into a Set and assert that the contents of the set are correct, bypassing MRUnit's validation code.

      A possible improvement would be to add a boolean orderMatters parameter to MapDriver.runTest(), invoking an order-insensitive version of TestDriver.validate() when orderMatters is false and the existing version otherwise.

      For clarity's sake only mappers are discussed in this feature request, but the same applies to reducers as well.

      1. MRUNIT-91.patch
        33 kB
        Dave Beech
      2. MRUNIT-91-1.patch
        33 kB
        Dave Beech
      3. MRUNIT-91-2.patch
        54 kB
        Dave Beech
      4. MRUNIT-91-3.patch
        56 kB
        Dave Beech

        Activity

        Jim Donofrio made changes -
        Assignee Jim Donofrio [ jdonofrio ] Dave Beech [ dbeech ]
        Jim Donofrio made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Jim Donofrio [ jdonofrio ]
        Resolution Fixed [ 1 ]
        Dave Beech made changes -
        Attachment MRUNIT-91-3.patch [ 12520523 ]
        Dave Beech made changes -
        Attachment MRUNIT-91-2.patch [ 12519857 ]
        Dave Beech made changes -
        Attachment MRUNIT-91-1.patch [ 12519811 ]
        Jim Donofrio made changes -
        Fix Version/s 1.0.0 [ 12316360 ]
        Dave Beech made changes -
        Field Original Value New Value
        Attachment MRUNIT-91.patch [ 12519806 ]
        William McNeill created issue -

          People

          • Assignee:
            Dave Beech
            Reporter:
            William McNeill
          • Votes:
            3 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development