Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5764

NUMA awareness support for launching containers

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0, 3.2.0
    • Component/s: nodemanager, yarn
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The purpose of this feature is to improve Hadoop performance by minimizing costly remote memory accesses on non SMP systems. Yarn containers, on launch, will be pinned to a specific NUMA node and all subsequent memory allocations will be served by the same node, reducing remote memory accesses. The current default behavior is to spread memory across all NUMA nodes.

        Attachments

        1. NUMA Awareness for YARN Containers.pdf
          288 kB
          Devaraj K
        2. NUMA Performance Results.pdf
          594 kB
          Devaraj K
        3. YARN-5764-v0.patch
          45 kB
          Devaraj K
        4. YARN-5764-v1.patch
          45 kB
          Devaraj K
        5. YARN-5764-v10.patch
          65 kB
          Devaraj K
        6. YARN-5764-v11.patch
          64 kB
          Devaraj K
        7. YARN-5764-v2.patch
          45 kB
          Devaraj K
        8. YARN-5764-v3.patch
          47 kB
          Devaraj K
        9. YARN-5764-v4.patch
          63 kB
          Devaraj K
        10. YARN-5764-v5.patch
          63 kB
          Devaraj K
        11. YARN-5764-v6.patch
          62 kB
          Devaraj K
        12. YARN-5764-v7.patch
          62 kB
          Devaraj K
        13. YARN-5764-v8.patch
          62 kB
          Devaraj K
        14. YARN-5764-v9.patch
          62 kB
          Devaraj K

          Issue Links

            Activity

              People

              • Assignee:
                devaraj.k Devaraj K
                Reporter:
                olasoji Olasoji
              • Votes:
                0 Vote for this issue
                Watchers:
                28 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: