Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6332

Provide facility to users for writting custom MergeManager implementation when custom shuffleconsumerPluggin is used

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      MR provides ability to the user for plugin custom ShuffleConsumerPlugin using mapreduce.job.reduce.shuffle.consumer.plugin.class. When the user is allowed to use this configuration as plugin, user also interest in implementing his own MergeManagerImpl.

      But now , user is forced to use MR provided MergeManagerImpl instead of custom MergeManagerImpl when user is using shuffle.consumer.plugin class. There should be well defined API's in MergeManager that can be used for any implementation without much effort to user for custom implementation.

        Attachments

        1. 0004-MAPREDUCE-6332.patch
          14 kB
          Rohith Sharma K S
        2. 0003-MAPREDUCE-6332.patch
          14 kB
          Rohith Sharma K S
        3. 0002-MAPREDUCE-6332.patch
          14 kB
          Rohith Sharma K S
        4. 0001-MAPREDUCE-6332.patch
          13 kB
          Rohith Sharma K S

          Issue Links

            Activity

              People

              • Assignee:
                rohithsharma Rohith Sharma K S
                Reporter:
                rohithsharma Rohith Sharma K S
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: