Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1747 Better physical memory monitoring for containers
  3. YARN-4599

Set OOM control for memory cgroups

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.9.0
    • 3.2.0
    • nodemanager
    • Reviewed

    Description

      YARN-1856 adds memory cgroups enforcing support. We should also explicitly set OOM control so that containers are not killed as soon as they go over their usage. Today, one could set the swappiness to control this, but clusters with swap turned off exist.

      Attachments

        1. Elastic Memory Control in YARN.pdf
          92 kB
          Miklos Szegedi
        2. YARN-4599.000.patch
          95 kB
          Miklos Szegedi
        3. YARN-4599.001.patch
          96 kB
          Miklos Szegedi
        4. YARN-4599.002.patch
          96 kB
          Miklos Szegedi
        5. YARN-4599.003.patch
          98 kB
          Miklos Szegedi
        6. YARN-4599.004.patch
          101 kB
          Miklos Szegedi
        7. YARN-4599.005.patch
          101 kB
          Miklos Szegedi
        8. YARN-4599.006.patch
          102 kB
          Miklos Szegedi
        9. YARN-4599.007.patch
          114 kB
          Miklos Szegedi
        10. YARN-4599.008.patch
          115 kB
          Miklos Szegedi
        11. YARN-4599.009.patch
          116 kB
          Miklos Szegedi
        12. YARN-4599.010.patch
          116 kB
          Miklos Szegedi
        13. YARN-4599.011.patch
          116 kB
          Miklos Szegedi
        14. YARN-4599.012.patch
          116 kB
          Miklos Szegedi
        15. YARN-4599.013.patch
          116 kB
          Miklos Szegedi
        16. YARN-4599.014.patch
          117 kB
          Miklos Szegedi
        17. YARN-4599.015.patch
          117 kB
          Miklos Szegedi
        18. YARN-4599.016.patch
          117 kB
          Miklos Szegedi
        19. YARN-4599.sandflee.patch
          33 kB
          sandflee
        20. yarn-4599-not-so-useful.patch
          9 kB
          Karthik Kambatla

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. YARN-8498 Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8375 TestCGroupElasticMemoryController fails surefire build

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8388 TestCGroupElasticMemoryController.testNormalExit() hangs on Linux

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-8437 Build oom-listener fails on older versions

          • Major - Major loss of function.
          • Resolved
          Dependency

          Improvement - An improvement or enhancement to an existing feature or task. YARN-6677 Preempt opportunistic containers when root container cgroup goes over memory limit

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue YARN-1856 cgroups based memory monitoring for containers

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-4048 Linux kernel panic under strict CPU limits(on CentOS/RHEL 6.x)

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Activity

            People

              miklos.szegedi@cloudera.com Miklos Szegedi
              kasha Karthik Kambatla
              Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: