Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4937

[HOD] Include ringmaster RPC port information in the notes attribute

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: contrib/hod
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In large cluster deployments, due to node failures, it sometimes happens that HOD clusters get allocated, but not deallocated even after the idleness limit of the cluster (the time for which no jobs are run) exceeds. One of the main reasons for this is the ringmaster process which is responsible for tracking and cleaning an idle cluster (of which it is a part) itself goes down. To handle such scenarios it makes sense to centrally track the ringmaster nodes for suspicious clusters. But since the information about which port the ringmaster is bound to is not centrally available, this becomes impossible to monitor.

      This issue is an enhancement request to include ringmaster RPC port information along with the JT and NN info as part of the resource manager's notes attribute so that it can be used by any monitoring processes built around it.

        Attachments

        1. hadoop-4937-2.txt
          0.8 kB
          Peeyush Bishnoi
        2. hadoop-4937-1.txt
          0.8 kB
          Peeyush Bishnoi
        3. hadoop-4937.txt
          0.7 kB
          Peeyush Bishnoi

          Activity

            People

            • Assignee:
              peeyushb Peeyush Bishnoi
              Reporter:
              yhemanth Hemanth Yamijala
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: