Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3178

Capacity Schedular shows incorrect cluster information in the RM logs

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Duplicate
    • Affects Version/s: 0.23.0
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      When we start the NM, after stopping it (in a quick session) CS shows incorrect information about clusterResource in the logs.

      I have encountered this issue in a pseudo cluster mode and steps to reproduce are

      1) start the YARN cluster
      2) stop a NM and start the NM again (in a quick session)

      There should be a NM running in the cluster however as I observed RM detects NM as dead, after default time since its actual unavailability(In this case NM has been stopped).

      If you start your NM before this time (default time), ResourceTracker throws IOEx, however, CS adds the NM's capacity to the clusterResource.

      After elapsed time (default time) when RM detects NM as dead, RM removes the NM and hence capacity of the cluster will be subtracted by the amount NM capacity.

      Eventually there is no NM running in the cluster, but capacity of the cluster is NM's capacity (by default)

        Attachments

        1. MAPREDUCE-3178.patch
          2 kB
          Bhallamudi Venkata Siva Kamesh

          Activity

            People

            • Assignee:
              kamesh Bhallamudi Venkata Siva Kamesh
              Reporter:
              kam_iitkgp Bhallamudi Venkata Siva Kamesh
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: