Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2492 (Clone of YARN-796) Allow for (admin) labels on nodes and resource-requests
  3. YARN-2744

Under some scenario, it is possible to end up with capacity scheduler configuration that uses labels that no longer exist

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.5.1
    • Fix Version/s: 2.6.0
    • Component/s: capacityscheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Use the following steps:

      • Ensure default in-memory storage is configured for labels
      • Define some labels and assign nodes to labels (e.g. define two labels and assign both labels to the host on a one host cluster)
      • Invoke refreshQueues
      • Modify capacity scheduler to create two top level queues and allow access to the labels from both the queues
      • Assign appropriate "label + queue" specific capacities
      • Restart resource manager

      Noticed that RM starts without any issues. The labels are not preserved across restart and thus the capacity-scheduler ends up using labels that are no longer present.

      At this point submitting an application to YARN will not succeed as there are no resources available with the labels.

        Attachments

        1. YARN-2744-20141025-2.patch
          10 kB
          Wangda Tan
        2. YARN-2744-20141025-1.patch
          8 kB
          Wangda Tan

          Activity

            People

            • Assignee:
              leftnoteasy Wangda Tan
              Reporter:
              sumitmohanty Sumit Mohanty
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: