Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8851 [Umbrella] A pluggable device plugin framework to ease vendor plugin development
  3. YARN-9060

[YARN-8851] Phase 1 - Support device isolation and use the Nvidia GPU plugin as an example

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0
    • None
    • None
    • Reviewed

    Description

      Due to the cgroups v1 implementation policy in linux kernel, we cannot update the value of the device cgroups controller unless we have the root permission (here). So we need to support this in container-executor for Java layer to invoke.

      This Jira will have three parts:

      1. native c-e module
      2. Java layer code to isolate devices for container (docker and non-docker)
      3. A sample Nvidia GPU plugin

      Attachments

        1. YARN-9060-trunk.001.patch
          22 kB
          Zhankun Tang
        2. YARN-9060-trunk.002.patch
          28 kB
          Zhankun Tang
        3. YARN-9060-trunk.003.patch
          28 kB
          Zhankun Tang
        4. YARN-9060-trunk.004.patch
          28 kB
          Zhankun Tang
        5. YARN-9060-trunk.005.patch
          28 kB
          Zhankun Tang
        6. YARN-9060-trunk.006.patch
          62 kB
          Zhankun Tang
        7. YARN-9060-trunk.007.patch
          72 kB
          Zhankun Tang
        8. YARN-9060-trunk.008.patch
          86 kB
          Zhankun Tang
        9. YARN-9060-trunk.009.patch
          108 kB
          Zhankun Tang
        10. YARN-9060-trunk.010.patch
          108 kB
          Zhankun Tang
        11. YARN-9060-trunk.011.patch
          108 kB
          Zhankun Tang
        12. YARN-9060-trunk.012.patch
          108 kB
          Zhankun Tang
        13. YARN-9060-trunk.013.patch
          112 kB
          Zhankun Tang
        14. YARN-9060-trunk.014.patch
          112 kB
          Zhankun Tang
        15. YARN-9060-trunk.015.patch
          113 kB
          Zhankun Tang
        16. YARN-9060-trunk.016.patch
          113 kB
          Zhankun Tang
        17. YARN-9060-trunk.017.patch
          113 kB
          Zhankun Tang
        18. YARN-9060-trunk.018.patch
          114 kB
          Zhankun Tang

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. YARN-9319 Fix compilation issue of handling typedef an existing name by gcc compiler

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          is depended upon by

          Sub-task - The sub-task of the issue YARN-8821 [YARN-8851] GPU hierarchy/topology scheduling support based on pluggable device framework

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              tangzhankun Zhankun Tang
              tangzhankun Zhankun Tang
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: