Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 3.0.0, 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: None
    • Release Note:
      MAPREDUCE-4807 Allow external implementations of the sort phase in a Map task
    • Target Version/s:

      Description

      Define interfaces and some abstract classes in the Hadoop framework to facilitate external sorter plugins both on the Map and Reduce sides.

      1. MapOutputSorter.java
        1 kB
        Mariappan Asokan
      2. ReduceInputSorter.java
        5 kB
        Mariappan Asokan
      3. MapOutputSorterAbstract.java
        5 kB
        Mariappan Asokan
      4. KeyValueIterator.java
        1 kB
        Mariappan Asokan
      5. MR-2454-trunkPatchPreview.gz
        48 kB
        Mariappan Asokan
      6. HadoopSortPlugin.pdf
        61 kB
        Mariappan Asokan
      7. mr-2454-on-mr-279-build82.patch.gz
        44 kB
        Mariappan Asokan
      8. mapreduce-2454.patch
        244 kB
        Mariappan Asokan
      9. mapreduce-2454.patch
        248 kB
        Mariappan Asokan
      10. mapreduce-2454.patch
        250 kB
        Mariappan Asokan
      11. mapreduce-2454.patch
        246 kB
        Mariappan Asokan
      12. HadoopSortPlugin.pdf
        56 kB
        Mariappan Asokan
      13. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      14. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      15. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      16. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      17. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      18. mapreduce-2454.patch
        281 kB
        Mariappan Asokan
      19. mapreduce-2454.patch
        281 kB
        Mariappan Asokan
      20. mapreduce-2454.patch
        281 kB
        Mariappan Asokan
      21. mapreduce-2454.patch
        68 kB
        Mariappan Asokan
      22. mapreduce-2454.patch
        68 kB
        Mariappan Asokan
      23. mapreduce-2454.patch
        69 kB
        Mariappan Asokan
      24. mapreduce-2454.patch
        102 kB
        Mariappan Asokan
      25. mapreduce-2454.patch
        104 kB
        Mariappan Asokan
      26. mapreduce-2454.patch
        104 kB
        Mariappan Asokan
      27. mapreduce-2454.patch
        105 kB
        Mariappan Asokan
      28. mapreduce-2454-modified-code.patch
        67 kB
        Mariappan Asokan
      29. mapreduce-2454-modified-test.patch
        1 kB
        Mariappan Asokan
      30. mapreduce-2454-new-test.patch
        33 kB
        Mariappan Asokan
      31. mapreduce-2454-protection-change.patch
        6 kB
        Mariappan Asokan
      32. mapreduce-2454.patch
        106 kB
        Mariappan Asokan
      33. mapreduce-2454.patch
        106 kB
        Mariappan Asokan

        Issue Links

          Activity

          Mariappan Asokan created issue -
          Chris Douglas made changes -
          Field Original Value New Value
          Affects Version/s 0.21.0 [ 12313563 ]
          Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
          Key HADOOP-7242 MAPREDUCE-2454
          Mariappan Asokan made changes -
          Attachment MapOutputSorter.java [ 12477557 ]
          Attachment ReduceInputSorter.java [ 12477558 ]
          Mariappan Asokan made changes -
          Attachment MapOutputSorterAbstract.java [ 12477710 ]
          Mariappan Asokan made changes -
          Attachment KeyValueIterator.java [ 12477711 ]
          Mariappan Asokan made changes -
          Attachment MR-2454-trunkPatchPreview.gz [ 12480420 ]
          Mariappan Asokan made changes -
          Attachment HadoopSortPlugin.pdf [ 12483610 ]
          Mariappan Asokan made changes -
          Attachment mr-2454-on-mr-279-build82.patch.gz [ 12490568 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12509077 ]
          Jeff Hammerbacher made changes -
          Link This issue relates to MAPREDUCE-4049 [ MAPREDUCE-4049 ]
          Mariappan Asokan made changes -
          Labels features performance
          Mariappan Asokan made changes -
          Labels features performance features performance plugin sort
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 2.0.0-alpha [ 12320354 ]
          Affects Version/s 3.0.0 [ 12320355 ]
          Target Version/s 2.0.0-alpha, 3.0.0 [ 12320354, 12320355 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12530864 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12509077 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531175 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12530864 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531175 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531752 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531752 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12533362 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12533362 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12537540 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12537540 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12537708 ]
          Mariappan Asokan made changes -
          Affects Version/s 2.2.0-alpha [ 12322471 ]
          Mariappan Asokan made changes -
          Link This issue is related to MAPREDUCE-4482 [ MAPREDUCE-4482 ]
          Alejandro Abdelnur made changes -
          Assignee Mariappan Asokan [ masokan ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12538585 ]
          Ahmed Radwan made changes -
          Link This issue relates to MAPREDUCE-4039 [ MAPREDUCE-4039 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542197 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542197 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542643 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542841 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543167 ]
          Attachment HadoopSortPlugin.pdf [ 12543168 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543174 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543167 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543218 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543287 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543452 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543461 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543878 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12545413 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12549046 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552499 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552530 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552689 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552875 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552985 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553151 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553200 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 2.0.0-alpha, 3.0.0 [ 12320354, 12320355 ] 3.0.0 [ 12320355 ]
          Arun C Murthy made changes -
          Comment [ Asokan - thanks again for being patient and working with me through this, and also for spending time with me in person on the design.

          Also, Alejandro, thanks for taking a look.

          Some comments:
          # ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454-modified-code.patch [ 12553727 ]
          Attachment mapreduce-2454-modified-test.patch [ 12553728 ]
          Attachment mapreduce-2454-new-test.patch [ 12553729 ]
          Attachment mapreduce-2454-protection-change.patch [ 12553730 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553731 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553789 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jerry Chen made changes -
          Link This issue blocks MAPREDUCE-1639 [ MAPREDUCE-1639 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 3.0.0 [ 12320355 ]
          Resolution Fixed [ 1 ]
          Jerry Chen made changes -
          Link This issue relates to MAPREDUCE-4891 [ MAPREDUCE-4891 ]
          Alejandro Abdelnur made changes -
          Fix Version/s 2.0.3-alpha [ 12323275 ]
          Fix Version/s 3.0.0 [ 12320355 ]
          Mariappan Asokan made changes -
          Release Note MAPREDUCE-4807 Allow external implementations of the sort phase in a Map task
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-1639 [ MAPREDUCE-1639 ]
          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-1639 [ MAPREDUCE-1639 ]

            People

            • Assignee:
              Mariappan Asokan
              Reporter:
              Mariappan Asokan
            • Votes:
              0 Vote for this issue
              Watchers:
              58 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development