Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3926 [Umbrella] Extend the YARN resource model for easier resource-type management and profiles
  3. YARN-7158

Address performance issue when 3+ resource types configured in the system

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Duplicate
    • None
    • None
    • nodemanager, resourcemanager
    • None

    Description

      In existing YARN-3926 branch, we use BaseResource to handle 2 mandatory resource types (memory+cpu) configured in the system. Once we have 3 or more resource types configured in the system. YARN will use ResourcePBImpl to initialize resource objects, which will be very expensive. We should update BaseResource when do resource calculation in RM instead of using PB-backed resource implementation.

      Attachments

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue YARN-6909 Use LightWeightedResource when number of resource types more than two

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              leftnoteasy Wangda Tan
              leftnoteasy Wangda Tan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: