Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4937

[HOD] Include ringmaster RPC port information in the notes attribute

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.20.0
    • contrib/hod
    • None
    • Reviewed

    Description

      In large cluster deployments, due to node failures, it sometimes happens that HOD clusters get allocated, but not deallocated even after the idleness limit of the cluster (the time for which no jobs are run) exceeds. One of the main reasons for this is the ringmaster process which is responsible for tracking and cleaning an idle cluster (of which it is a part) itself goes down. To handle such scenarios it makes sense to centrally track the ringmaster nodes for suspicious clusters. But since the information about which port the ringmaster is bound to is not centrally available, this becomes impossible to monitor.

      This issue is an enhancement request to include ringmaster RPC port information along with the JT and NN info as part of the resource manager's notes attribute so that it can be used by any monitoring processes built around it.

      Attachments

        1. hadoop-4937.txt
          0.7 kB
          Peeyush Bishnoi
        2. hadoop-4937-1.txt
          0.8 kB
          Peeyush Bishnoi
        3. hadoop-4937-2.txt
          0.8 kB
          Peeyush Bishnoi

        Activity

          People

            peeyushb Peeyush Bishnoi
            yhemanth Hemanth Yamijala
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: