Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-91 DFIP aka 'NodeManager should handle Disk-Failures In Place'
  3. YARN-90

NodeManager should identify failed disks becoming good again

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.6.0
    • nodemanager
    • None
    • Reviewed

    Description

      MAPREDUCE-3121 makes NodeManager identify disk failures. But once a disk goes down, it is marked as failed forever. To reuse that disk (after it becomes good), NodeManager needs restart. This JIRA is to improve NodeManager to reuse good disks(which could be bad some time back).

      Attachments

        1. apache-yarn-90.0.patch
          48 kB
          Varun Vasudev
        2. apache-yarn-90.1.patch
          93 kB
          Varun Vasudev
        3. apache-yarn-90.10.patch
          109 kB
          Varun Vasudev
        4. apache-yarn-90.2.patch
          93 kB
          Varun Vasudev
        5. apache-yarn-90.3.patch
          95 kB
          Varun Vasudev
        6. apache-yarn-90.4.patch
          95 kB
          Varun Vasudev
        7. apache-yarn-90.5.patch
          99 kB
          Varun Vasudev
        8. apache-yarn-90.6.patch
          99 kB
          Varun Vasudev
        9. apache-yarn-90.7.patch
          108 kB
          Varun Vasudev
        10. apache-yarn-90.8.patch
          109 kB
          Varun Vasudev
        11. apache-yarn-90.9.patch
          109 kB
          Varun Vasudev
        12. YARN-90.1.patch
          2 kB
          nijel
        13. YARN-90.patch
          7 kB
          Ravi Prakash
        14. YARN-90.patch
          7 kB
          Hou Song
        15. YARN-90.patch
          11 kB
          Hou Song
        16. YARN-90.patch
          2 kB
          nijel

        Issue Links

          breaks

          Sub-task - The sub-task of the issue YARN-4393 TestResourceLocalizationService#testFailedDirsResourceRelease fails intermittently

          • Major - Major loss of function.
          • Closed
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. YARN-1196 LocalDirsHandlerService never change failedDirs back to normal even when these disks turn good

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-1380 Enable NM to automatically reuse failed local dirs after they are available again

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. YARN-2473 YARN never cleans up container directories from a full disk

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-4011 Jobs fail since nm-local-dir not cleaned up when rogue job fills up disk

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-2488 Nodemanager should support check if bad disks have turned to good ones

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. YARN-2839 YARN minicluster doesn't bail out if all the NM disks are dead

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-2799 cleanup TestLogAggregationService based on the change in YARN-90

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              vvasudev Varun Vasudev
              ravidotg Ravi Gummadi
              Votes:
              1 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: