Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-431 [Umbrella] Complete/Stabilize YARN application log-handling
  3. YARN-950

Ability to limit or avoid aggregating logs beyond a certain size

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.23.9, 2.6.0
    • None
    • log-aggregation, nodemanager
    • None

    Description

      It would be nice if ops could configure a cluster such that any container log beyond a configured size would either only have a portion of the log aggregated or not aggregated at all. This would help speed up the recovery path for cases where a container creates an enormous log and fills a disk, as currently it tries to aggregate the entire, enormous log rather than only aggregating a small portion or simply deleting it.

      Attachments

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. YARN-10471 Prevent logs for any container from becoming larger than a configurable size.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: