Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-225

Fault tolerant Hadoop Job Tracker

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      High availability enterprise system

      Description

      The Hadoop framework has been designed, in an eort to enhance perfor-
      mances, with a single JobTracker (master node). It's responsibilities varies
      from managing job submission process, compute the input splits, schedule
      the tasks to the slave nodes (TaskTrackers) and monitor their health.
      In some environments, like the IBM and Google's Internet-scale com-
      puting initiative, there is the need for high-availability, and performances
      becomes a secondary issue. In this environments, having a system with
      a Single Point of Failure (such as Hadoop's single JobTracker) is a major
      concern.
      My proposal is to provide a redundant version of Hadoop by adding
      support for multiple replicated JobTrackers. This design can be approached
      in many dierent ways.

      In the document at: http://sites.google.com/site/hadoopthesis/Home/FaultTolerantHadoop.pdf?attredirects=0

      I wrote an overview of the problem and some approaches to solve it.

      I post this to the community to gather feedback on the best way to proceed in my work.

      Thank you!

        Attachments

        1. HADOOP-4586v0.3.patch
          39 kB
          Francesco Salbaroli
        2. Enhancing the Hadoop MapReduce framework by adding fault.ppt
          511 kB
          Francesco Salbaroli
        3. jgroups-all.jar
          1.92 MB
          Francesco Salbaroli
        4. HADOOP-4586-0.1.patch
          35 kB
          Francesco Salbaroli
        5. FaultTolerantHadoop.pdf
          136 kB
          Francesco Salbaroli

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              fsalbaroli Francesco Salbaroli Assign to me
              Reporter:
              fsalbaroli Francesco Salbaroli

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment