Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-431

[Umbrella] Complete/Stabilize YARN application log-handling

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Attachments

        Issue Links

        relates to

        Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-3143 Complete aggregation of user-logs spit out by containers onto DFS

        • Major - Major loss of function.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. YARN-4029 Update LogAggregationStatus to store on finish

        • Major - Major loss of function.
        • Patch Available
        1.
        		 Allow per job log aggregation configuration Sub-task Open Unassigned
        2.
        		 NodeManager should serve logs directly if log-aggregation is not enabled Sub-task Open Siddharth Seth
        3.
        		 Log aggregation should not assume an AppMaster will have containerId 1 Sub-task Open Unassigned
        4.
        		 yarn log does not output all needed information, and is in a binary format Sub-task Closed Ravi Prakash
        5.
        		 Documentation for Log Aggregation and log retrieval. Sub-task Open Vinod Kumar Vavilapalli
        6.
        		 NM should provide a way for AM to tell it not to aggregate logs. Sub-task Resolved Ming Ma
        7.
        		 NM should aggregate logs when application finishes. Sub-task Closed Robert Joseph Evans
        8.
        		 NM should limit number of applications who's logs are being aggregated Sub-task Open Unassigned
        9.
        		 nodemanager log aggregation has scaling issues with namenode Sub-task Open Kuhu Shukla
        10.
        		 Make container logs available over HTTP in plain text Sub-task Closed Sandy Ryza
        11.
        		 Page navigation support for container logs page and the logs web-service on NMs Sub-task Open Omkar Vinit Joshi
        12.
        		 NM containerlogs servlet can't handle logs of more than a GB Sub-task Closed Unassigned
        13.
        		 $yarn logs command should return an appropriate error message if YARN application is still running Sub-task Closed Siddharth Seth
        14.
        		 Expose a client API to allow clients to figure if log aggregation is complete Sub-task Resolved Xuan Gong
        15.
        		 Log Aggregation generates a storm of fsync() for namenode Sub-task Closed Kihwal Lee
        16.
        		 NM need to notify the log aggregation status to RM through Node heartbeat Sub-task Resolved Xuan Gong
        17.
        		 Related Web UI, CLI changes on exposing client API to check log aggregation status Sub-task Resolved Xuan Gong
        18.
        		 Yarn aggregated logs are difficult for external tools to understand Sub-task Open Unassigned
        19.
        		 Enhance yarn logs CLI to allow logs to be pulled for a given containerId without specifying nodeAddress Sub-task Resolved Xuan Gong
        20.
        		 Enhance yarn logs command to specify the filename to be retrieved Sub-task Resolved Xuan Gong
        21.
        		 IOException in AppLogAggregatorImpl does not give stacktrace and leaves aggregated TFile in a bad state. Sub-task Open Xuan Gong
        22.
        		 Improve YARN log command to get AMContainer logs as well as running containers logs Sub-task Resolved Xuan Gong
        23.
        		 Ability to limit or avoid aggregating logs beyond a certain size Sub-task Open Adam Antal
        24.
        		 Node's Log Aggregation Report with SUCCEED should not cached in RMApps Sub-task Resolved Xuan Gong
        25.
        		 Log aggregation finish time should get logged for trouble shooting. Sub-task Open Junping Du

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vinodkv Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated: