Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2225

MultipleOutputs should not require the use of 'Writable'

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.23.0
    • Component/s: job submission
    • Labels:
      None
    • Environment:

      Linux

    • Hadoop Flags:
      Reviewed
    • Release Note:
      MultipleOutputs should not require the use/check of 'Writable' interfaces in key and value classes.
    • Tags:
      multipleoutputs, writable, serialization

      Description

      MultipleOutputs right now requires for Key/Value classes to utilize the Writable and WritableComparable interfaces, and fails if the associated key/value classes aren't doing so.

      With support for alternates like Avro serialization, using Writables isn't necessary and thus the MO class must not strictly check for them.

      And since comparators may be given separately, key class doesn't need to be checked for implementing a comparable (although it is good design if the key class does implement Comparable at least).

      Am not sure if this brings about an incompatible change (does Java have BIC? No idea).

        Activity

        Harsh J created issue -
        Harsh J made changes -
        Field Original Value New Value
        Attachment multipleoutputs.nowritables.r1.diff [ 12466582 ]
        Harsh J made changes -
        Attachment multipleoutputs.nowritables.r2.diff [ 12467578 ]
        Harsh J made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note MultipleOutputs do not require the use of 'Writable' interfaces. MultipleOutputs should not require the use/check of 'Writable' interfaces in key and value classes.
        Harsh J made changes -
        Fix Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.23.0 [ 12315570 ]
        Environment Linux
        Harsh J made changes -
        Priority Major [ 3 ] Blocker [ 1 ]
        Harsh J made changes -
        Attachment multipleoutputs.nowritables.r2.diff [ 12472201 ]
        Tom White made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.23.0 [ 12315570 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Harsh J
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1m
              1m
              Remaining:
              Remaining Estimate - 1m
              1m
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development