Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 3.0.0, 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: None
    • Release Note:
      MAPREDUCE-4807 Allow external implementations of the sort phase in a Map task
    • Target Version/s:

      Description

      Define interfaces and some abstract classes in the Hadoop framework to facilitate external sorter plugins both on the Map and Reduce sides.

      1. HadoopSortPlugin.pdf
        56 kB
        Mariappan Asokan
      2. HadoopSortPlugin.pdf
        61 kB
        Mariappan Asokan
      3. KeyValueIterator.java
        1 kB
        Mariappan Asokan
      4. MapOutputSorter.java
        1 kB
        Mariappan Asokan
      5. MapOutputSorterAbstract.java
        5 kB
        Mariappan Asokan
      6. mapreduce-2454.patch
        106 kB
        Mariappan Asokan
      7. mapreduce-2454.patch
        106 kB
        Mariappan Asokan
      8. mapreduce-2454.patch
        105 kB
        Mariappan Asokan
      9. mapreduce-2454.patch
        104 kB
        Mariappan Asokan
      10. mapreduce-2454.patch
        104 kB
        Mariappan Asokan
      11. mapreduce-2454.patch
        102 kB
        Mariappan Asokan
      12. mapreduce-2454.patch
        69 kB
        Mariappan Asokan
      13. mapreduce-2454.patch
        68 kB
        Mariappan Asokan
      14. mapreduce-2454.patch
        68 kB
        Mariappan Asokan
      15. mapreduce-2454.patch
        281 kB
        Mariappan Asokan
      16. mapreduce-2454.patch
        281 kB
        Mariappan Asokan
      17. mapreduce-2454.patch
        281 kB
        Mariappan Asokan
      18. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      19. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      20. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      21. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      22. mapreduce-2454.patch
        280 kB
        Mariappan Asokan
      23. mapreduce-2454.patch
        246 kB
        Mariappan Asokan
      24. mapreduce-2454.patch
        250 kB
        Mariappan Asokan
      25. mapreduce-2454.patch
        248 kB
        Mariappan Asokan
      26. mapreduce-2454.patch
        244 kB
        Mariappan Asokan
      27. mapreduce-2454-modified-code.patch
        67 kB
        Mariappan Asokan
      28. mapreduce-2454-modified-test.patch
        1 kB
        Mariappan Asokan
      29. mapreduce-2454-new-test.patch
        33 kB
        Mariappan Asokan
      30. mapreduce-2454-protection-change.patch
        6 kB
        Mariappan Asokan
      31. mr-2454-on-mr-279-build82.patch.gz
        44 kB
        Mariappan Asokan
      32. MR-2454-trunkPatchPreview.gz
        48 kB
        Mariappan Asokan
      33. ReduceInputSorter.java
        5 kB
        Mariappan Asokan

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-1639 [ MAPREDUCE-1639 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-1639 [ MAPREDUCE-1639 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Mariappan Asokan made changes -
          Release Note MAPREDUCE-4807 Allow external implementations of the sort phase in a Map task
          Alejandro Abdelnur made changes -
          Fix Version/s 2.0.3-alpha [ 12323275 ]
          Fix Version/s 3.0.0 [ 12320355 ]
          Jerry Chen made changes -
          Link This issue relates to MAPREDUCE-4891 [ MAPREDUCE-4891 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 3.0.0 [ 12320355 ]
          Resolution Fixed [ 1 ]
          Jerry Chen made changes -
          Link This issue blocks MAPREDUCE-1639 [ MAPREDUCE-1639 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553789 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553731 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454-modified-code.patch [ 12553727 ]
          Attachment mapreduce-2454-modified-test.patch [ 12553728 ]
          Attachment mapreduce-2454-new-test.patch [ 12553729 ]
          Attachment mapreduce-2454-protection-change.patch [ 12553730 ]
          Arun C Murthy made changes -
          Comment [ Asokan - thanks again for being patient and working with me through this, and also for spending time with me in person on the design.

          Also, Alejandro, thanks for taking a look.

          Some comments:
          # ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 2.0.0-alpha, 3.0.0 [ 12320354, 12320355 ] 3.0.0 [ 12320355 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553200 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12553151 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552985 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552875 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552689 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552530 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12552499 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12549046 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12545413 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543878 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543461 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543452 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543287 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543218 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543167 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543174 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12543167 ]
          Attachment HadoopSortPlugin.pdf [ 12543168 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542841 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542643 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542197 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12542197 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Ahmed Radwan made changes -
          Link This issue relates to MAPREDUCE-4039 [ MAPREDUCE-4039 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12538585 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Alejandro Abdelnur made changes -
          Assignee Mariappan Asokan [ masokan ]
          Mariappan Asokan made changes -
          Link This issue is related to MAPREDUCE-4482 [ MAPREDUCE-4482 ]
          Mariappan Asokan made changes -
          Affects Version/s 2.2.0-alpha [ 12322471 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12537708 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12537540 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12537540 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12533362 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12533362 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531752 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531752 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531175 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12530864 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12531175 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Mariappan Asokan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12509077 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12530864 ]
          Mariappan Asokan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 2.0.0-alpha [ 12320354 ]
          Affects Version/s 3.0.0 [ 12320355 ]
          Target Version/s 2.0.0-alpha, 3.0.0 [ 12320354, 12320355 ]
          Mariappan Asokan made changes -
          Labels features performance features performance plugin sort
          Mariappan Asokan made changes -
          Labels features performance
          Jeff Hammerbacher made changes -
          Link This issue relates to MAPREDUCE-4049 [ MAPREDUCE-4049 ]
          Mariappan Asokan made changes -
          Attachment mapreduce-2454.patch [ 12509077 ]
          Mariappan Asokan made changes -
          Attachment mr-2454-on-mr-279-build82.patch.gz [ 12490568 ]
          Mariappan Asokan made changes -
          Attachment HadoopSortPlugin.pdf [ 12483610 ]
          Mariappan Asokan made changes -
          Attachment MR-2454-trunkPatchPreview.gz [ 12480420 ]
          Mariappan Asokan made changes -
          Attachment KeyValueIterator.java [ 12477711 ]
          Mariappan Asokan made changes -
          Attachment MapOutputSorterAbstract.java [ 12477710 ]
          Mariappan Asokan made changes -
          Attachment MapOutputSorter.java [ 12477557 ]
          Attachment ReduceInputSorter.java [ 12477558 ]
          Chris Douglas made changes -
          Field Original Value New Value
          Affects Version/s 0.21.0 [ 12313563 ]
          Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
          Key HADOOP-7242 MAPREDUCE-2454
          Mariappan Asokan created issue -

            People

            • Assignee:
              Mariappan Asokan
              Reporter:
              Mariappan Asokan
            • Votes:
              0 Vote for this issue
              Watchers:
              58 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development