Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-522

[Umbrella] Better reporting for crashed/Killed AMs and Containers

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Crashing AMs has been a real pain for users since the beginning. And there are already a few tickets floating around, filing this to consolidate them.

      Attachments

        Issue Links

          depends upon

          Bug - A problem which impairs or prevents the functions of the product. YARN-2560 Diagnostics is delayed to passed to ApplicationReport

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. YARN-2688 Better diagnostics on Container Launch failures

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-4065 container-executor error should include effective user id

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. YARN-1441 The diagnostic shows empty information in the console when Application failed to complete successfully in Distributed Shell

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-3820 Improve logging when containers run beyond memory limits

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-1551 Allow user-specified reason for killApplication

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue YARN-649 Make container logs available over HTTP in plain text

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-3688 Need better Error message if AM is killed/throws exception

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-4362 If possible, we should get back the feature of propagating task logs back to JobClient

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. YARN-560 If AM fails due to overrunning resource limits, error not visible through UI sometimes

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. YARN-1438 When a container fails, the text of the exception isn't included in the diagnostics

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-564 Job history logs do not log anything when JVM fails to start

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-4937 MR AM handles an oversized split metainfo file poorly

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-10312 Shell.ExitCodeException to have more useful toString

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. YARN-257 NM should gracefully handle a full local disk

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-2934 Improve handling of container's stderr

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          1.
          		 On container failure, include logs in diagnostics Sub-task Resolved Sandy Ryza
          2.
          		 Container localization failures aren't reported from NM to RM Sub-task Closed Jian He
          3.
          		 In YarnClient, pull AM logs on AM container failure Sub-task Open Li Lu
          4.
          		 Failures in container launches due to issues like disk failure are difficult to diagnose Sub-task Resolved Unassigned
          5.
          		 Difficult to diagnose a failed container launch when error due to invalid environment variable Sub-task Closed Jian He
          6.
          		 Localization failures should be available in container diagnostics Sub-task Resolved Vinod Kumar Vavilapalli
          7.
          		 Improve exception information on AM launch crashes Sub-task Closed Li Lu
          8.
          		 The diagnostics is always the ExitCodeException stack when the container crashes Sub-task Closed Tsuyoshi Ozawa
          9.
          		 add more diags about app retry limits on AM failures Sub-task Resolved Steve Loughran
          10.
          		 Include command line, localization info and env vars on AM launch failure Sub-task Resolved Unassigned
          11.
          		 Log the command of launching containers Sub-task Resolved Jeff Zhang
          12.
          		 Add tail of stderr to diagnostics if container fails to launch or it container logs are empty Sub-task Resolved Unassigned
          13.
          		 LinuxContainerExecutor loses info when forwarding ResourceHandlerException Sub-task Resolved Bibin Chundatt
          14.
          		 add a way for an attempt to report an attempt failure Sub-task Open Sunil G
          15.
          		 create a yarn.troubleshooting log for logging app launching problems Sub-task Open Unassigned
          16.
          		 Add container launch related debug information to container logs when a container fails Sub-task Resolved Varun Vasudev
          17.
          		 Print enough container-executor logs for troubleshooting container launch failures Sub-task Open Unassigned
          18.
          		 Capture launch_container.sh logs to a separate log file Sub-task Resolved Suma Shivaprasad
          19.
          		 When an export var command fails in launch_container.sh, the full container launch should fail Sub-task Resolved Sunil G

          Activity

            People

              Unassigned Unassigned
              vinodkv Vinod Kumar Vavilapalli
              Votes:
              1 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

                Created:
                Updated: