Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3808

[HOD] Include job tracker RPC in notes attribute after job submission

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.18.0
    • 0.18.0
    • contrib/hod
    • None
    • Incompatible change, Reviewed
    • Hide
      Modified HOD to include the RPC port of the JobTracker in the 'notes' attribute of the resource manager. The RPC port is included as the string 'Mapred RPC Port:<port number>'. Tools that depend on the value of the notes attribute must change to parse this new value.
      Show
      Modified HOD to include the RPC port of the JobTracker in the 'notes' attribute of the resource manager. The RPC port is included as the string 'Mapred RPC Port:<port number>'. Tools that depend on the value of the notes attribute must change to parse this new value.

    Description

      Currently, after a job is submitted, HOD records the Jobtracker URL and the NameNode URL in the 'notes' attribute of the Torque resource manager. This helps in building centralized administration tools that need to get a full picture of all allocated clusters etc. Including the Jobtracker RPC port in this attribute would be useful to build such similar tools. One of them is an idle job tracker that can be centrally run to clear up any clusters that aren't automatically cleaned up due to some faulty nodes in the cluster or resource manager problems.

      Attachments

        1. HADOOP-3808.1.patch
          1 kB
          Hemanth Yamijala
        2. HADOOP-3808.patch
          3 kB
          Hemanth Yamijala

        Activity

          People

            yhemanth Hemanth Yamijala
            yhemanth Hemanth Yamijala
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: