Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-737

Giraph Application Master: move to new and stable YARN API

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: mapreduce
    • Labels:
      None

      Description

      Giraph was the early adopter of Hadoop YARN AM! Eli successfully wrote a Giraph AM based on Hadoop 2.0.x_alpha. However, in last few months, Yarn significantly overhauled its APIs and associated coding patterns. The new beta version is 2.1.x and I was told by Yarn-dev that current APIs will not change much.
      In the above circumstances, we need to substantially overhaul Giraph AM as well to accommodate with the new Yarn API. Moreover, in newer YARN API, supporting kerberos security in AM becomes easier and more transparent.

      Potential impact:
      The upcoming Girpah AM will not work with earlier alpha Hadoop versions such as 2.0.3. I'm not sure if anyone is using Giraph AM in production. However, the more prevalent way of Giraph processing (MR-based) should continue to work.

        Attachments

        1. GIRAPH-737.WIP.patch
          68 kB
          Mohammad Islam
        2. GIRAPH-737-2.patch
          68 kB
          Eli Reisman
        3. GIRAPH-737-3.patch
          69 kB
          Mohammad Islam

          Issue Links

            Activity

              People

              • Assignee:
                kamrul Mohammad Islam
                Reporter:
                kamrul Mohammad Islam
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: