Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2142

Refactor RaidNode to remove dependence on map reduce

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I am refactoring the RaidNode code as follows: The base class RaidNode will contain the common functionality needed for raiding files. The derived class LocalRaidNode contains an implementation of RaidNode that performs raiding locally. The derived class DistRaidNode performs raiding using map reduce jobs. This way, only DistRaidNode has a dependency on map reduce code and RaidNode and LocalRaidNode can be moved to HDFS.

      1. MAPREDUCE-2142.patch
        23 kB
        Patrick Kling
      2. MAPREDUCE-2142.3.patch
        25 kB
        Patrick Kling
      3. MAPREDUCE-2142.2.patch
        24 kB
        Patrick Kling

        Activity

        Patrick Kling created issue -
        Hide
        Patrick Kling added a comment -

        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 12 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system tests framework. The patch passed system tests framework compile.

        Show
        Patrick Kling added a comment - [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 12 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile.
        Patrick Kling made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-2142.patch [ 12457583 ]
        Hide
        Patrick Kling added a comment -

        I have updated this patch (since it no longer applied cleanly) and posted it on the review board: https://reviews.apache.org/r/24/

        Show
        Patrick Kling added a comment - I have updated this patch (since it no longer applied cleanly) and posted it on the review board: https://reviews.apache.org/r/24/
        Patrick Kling made changes -
        Attachment MAPREDUCE-2142.2.patch [ 12458686 ]
        Hide
        Patrick Kling added a comment -

        Use a DummyProgressable instead of null.

        Show
        Patrick Kling added a comment - Use a DummyProgressable instead of null.
        Patrick Kling made changes -
        Attachment MAPREDUCE-2142.3.patch [ 12458690 ]
        Hide
        dhruba borthakur added a comment -

        I just committed it. Thanks Patrick. Please update the wiki too.

        Show
        dhruba borthakur added a comment - I just committed it. Thanks Patrick. Please update the wiki too.
        dhruba borthakur made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.22.0 [ 12314184 ]
        Resolution Fixed [ 1 ]
        dhruba borthakur made changes -
        Assignee Patrick Kling [ pkling ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #531 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/531/)
        MAPREDUCE-2142. Refactor RaidNode so that the map-reduce component is
        clearly separated out. (Patrick Kling via dhruba)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #531 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/531/ ) MAPREDUCE-2142 . Refactor RaidNode so that the map-reduce component is clearly separated out. (Patrick Kling via dhruba)
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )
        Konstantin Shvachko made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Patrick Kling
            Reporter:
            Patrick Kling
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development