Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2142

Refactor RaidNode to remove dependence on map reduce

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I am refactoring the RaidNode code as follows: The base class RaidNode will contain the common functionality needed for raiding files. The derived class LocalRaidNode contains an implementation of RaidNode that performs raiding locally. The derived class DistRaidNode performs raiding using map reduce jobs. This way, only DistRaidNode has a dependency on map reduce code and RaidNode and LocalRaidNode can be moved to HDFS.

      1. MAPREDUCE-2142.patch
        23 kB
        Patrick Kling
      2. MAPREDUCE-2142.2.patch
        24 kB
        Patrick Kling
      3. MAPREDUCE-2142.3.patch
        25 kB
        Patrick Kling

        Activity

        Hide
        Patrick Kling added a comment -

        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 12 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system tests framework. The patch passed system tests framework compile.

        Show
        Patrick Kling added a comment - [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 12 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile.
        Hide
        Patrick Kling added a comment -

        I have updated this patch (since it no longer applied cleanly) and posted it on the review board: https://reviews.apache.org/r/24/

        Show
        Patrick Kling added a comment - I have updated this patch (since it no longer applied cleanly) and posted it on the review board: https://reviews.apache.org/r/24/
        Hide
        Patrick Kling added a comment -

        Use a DummyProgressable instead of null.

        Show
        Patrick Kling added a comment - Use a DummyProgressable instead of null.
        Hide
        dhruba borthakur added a comment -

        I just committed it. Thanks Patrick. Please update the wiki too.

        Show
        dhruba borthakur added a comment - I just committed it. Thanks Patrick. Please update the wiki too.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #531 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/531/)
        MAPREDUCE-2142. Refactor RaidNode so that the map-reduce component is
        clearly separated out. (Patrick Kling via dhruba)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #531 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/531/ ) MAPREDUCE-2142 . Refactor RaidNode so that the map-reduce component is clearly separated out. (Patrick Kling via dhruba)
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )

          People

          • Assignee:
            Patrick Kling
            Reporter:
            Patrick Kling
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development