Uploaded image for project: 'MRUnit'
  1. MRUnit
  2. MRUNIT-198

Serialization is missing in MockMultipleOutputs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.1.0
    • 1.1.0
    • None

    Description

      Hi,
      with issue MRUNIT-13 MockMultipleOutputs was introduced. Which is great! Unfortunately the inner class MockRecordWriter doesn't serialize the Object and therefore isn't storing a copy of the Object but the Object itself. I would suggest to use org.apache.hadoop.mrunit.internal.output.MockOutputCollector instead of the inner class. This Collector does store a copy of the Object and I see no point in having more or less the same Collector/Writer twice.

      Another thing that bugs me, is that MockMultipleOutputs requires you to use Comparable Objects in your MR Jobs, and I don't see for what reason this restriction is imposed on me.

      I'll provide a patch for this soon and would be glad if someone can comment on it.
      Cheers, Lars

      Attachments

        1. MRUNIT-198.patch
          23 kB
          Lars Grote
        2. MRUNIT-198-1.patch
          23 kB
          Lars Grote

        Activity

          People

            lgrote Lars Grote
            lgrote Lars Grote
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: