Hadoop YARN
  1. Hadoop YARN
  2. YARN-431 [Umbrella] Complete/Stabilize YARN application log-handling
  3. YARN-85

Allow per job log aggregation configuration

        Details

        • Type: Sub-task Sub-task
        • Status: Open
        • Priority: Critical Critical
        • Resolution: Unresolved
        • Affects Version/s: None
        • Fix Version/s: None
        • Component/s: nodemanager
        • Labels:
          None

          Description

          Currently, if log aggregation is enabled for a cluster - logs for all jobs will be aggregated - leading to a whole bunch of files on hdfs which users may not want.
          Users should be able to control this along with the aggregation policy - failed only, all, etc.

            Issue Links

            blocks

            Bug - A problem which impairs or prevents the functions of the product. YARN-386 [Umbrella] YARN API Changes

            • Major - Major loss of function.
            • Open
            is related to

            Sub-task - The sub-task of the issue YARN-221 NM should provide a way for AM to tell it not to aggregate logs.

            • Major - Major loss of function.
            • Patch Available

            Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-3143 Complete aggregation of user-logs spit out by containers onto DFS

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Siddharth Seth created issue -
              Vinod Kumar Vavilapalli made changes -
              Field Original Value New Value
              Link This issue is related to MAPREDUCE-3143 [ MAPREDUCE-3143 ]
              Robert Joseph Evans made changes -
              Target Version/s 0.23.2 [ 12319851 ] 0.23.3, 2.0.0, 3.0.0 [ 12320060, 12320354, 12320355 ]
              Vinod Kumar Vavilapalli made changes -
              Project Hadoop Map/Reduce [ 12310941 ] Hadoop YARN [ 12313722 ]
              Key MAPREDUCE-3905 YARN-85
              Affects Version/s 0.23.0 [ 12315570 ]
              Target Version/s 0.23.3, 2.0.0-alpha, 3.0.0 [ 12320060, 12320354, 12320355 ]
              Component/s nodemanager [ 12319323 ]
              Component/s mrv2 [ 12314301 ]
              Robert Joseph Evans made changes -
              Link This issue is related to YARN-221 [ YARN-221 ]
              Hitesh Shah made changes -
              Priority Major [ 3 ] Critical [ 2 ]
              Vinod Kumar Vavilapalli made changes -
              Parent YARN-431 [ 12634475 ]
              Issue Type Improvement [ 4 ] Sub-task [ 7 ]
              Vinod Kumar Vavilapalli made changes -
              Link This issue blocks YARN-386 [ YARN-386 ]
              Chris Trezzo made changes -
              Assignee Siddharth Seth [ sseth ] Chris Trezzo [ ctrezzo ]

                People

                • Assignee:
                  Chris Trezzo
                  Reporter:
                  Siddharth Seth
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  7 Start watching this issue

                  Dates

                  • Created:
                    Updated:

                    Development