Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-431 [Umbrella] Complete/Stabilize YARN application log-handling
  3. YARN-171

NodeManager should serve logs directly if log-aggregation is not enabled

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.23.3
    • None
    • nodemanager
    • None

    Description

      NodeManagers never serve logs for completed applications. If log-aggregation is not enabled, in the interim, due to bugs like YARN-162, this is a serious problem for users as logs are necessarily not available.

      We should let nodes serve logs directly if YarnConfiguration.LOG_AGGREGATION_ENABLED is set. This should be okay as NonAggregatingLogHandler can retain logs upto YarnConfiguration.NM_LOG_RETAIN_SECONDS.

      Attachments

        1. YARN-171_3.txt
          53 kB
          Siddharth Seth
        2. YARN171_WIP.txt
          42 kB
          Siddharth Seth

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-4734 The history server should link back to NM logs if aggregation is incomplete / disabled

          • Major - Major loss of function.
          • Open
          is related to

          Sub-task - The sub-task of the issue YARN-162 nodemanager log aggregation has scaling issues with namenode

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Activity

            People

              sseth Siddharth Seth
              vinodkv Vinod Kumar Vavilapalli
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: