Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6834

A container request with only racks specified and relax locality set to false is never honoured

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: capacity scheduler
    • Labels:
      None

      Description

      A patch for a unit test is attached to reproduce the issue. It creates a container request with only racks specified (nodes=null) and relax locality set to false. With the node-locality-delay conf set appropriately, we wait indefinitely for a container allocation and the test will timeout.

      My understanding of what causes this issue is as follows. The RegularContainerAllocator delays a rack local allocation based on the node-locality-delay parameter. This delay is based on missed opportunities. However, the corresponding off-switch request is skipped but does not count towards a missed opportunity (because relax locality is set to false). So the allocator waits indefinitely.

      1. yarn-6834-unittest.patch
        4 kB
        Muhammad Samir Khan

        Activity

        There are no comments yet on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            samkhan Muhammad Samir Khan
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development