Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10471

Prevent logs for any container from becoming larger than a configurable size.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Done
    • 3.2.1, 3.1.4
    • 3.2.2, 3.4.0, 3.3.1, 3.2.3
    • None
    • None

    Description

      Configure a cluster such that a task attempt will be killed if any container log exceeds a configured size. This would help prevent logs from filling disks and also prevent the need to aggregate enormous logs.

      Attachments

        1. YARN.10471.branch-3.2.005.patch
          25 kB
          Eric Payne
        2. YARN.10471.005.patch
          26 kB
          Eric Payne
        3. YARN.10471.004.patch
          26 kB
          Eric Payne
        4. YARN.10471.branch-3.2.003.patch
          24 kB
          Eric Payne
        5. YARN.10471.003.patch
          24 kB
          Eric Payne
        6. YARN.10471.002.patch
          24 kB
          Eric Payne
        7. YARN.10471.001.patch
          24 kB
          Eric Payne

        Issue Links

          is related to

          Sub-task - The sub-task of the issue YARN-950 Ability to limit or avoid aggregating logs beyond a certain size

          • Major - Major loss of function.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. YARN-10743 Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              epayne Eric Payne
              epayne Eric Payne
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: