Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10471

Prevent logs for any container from becoming larger than a configurable size.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Done
    • 3.2.1, 3.1.4
    • 3.2.2, 3.4.0, 3.3.1, 3.2.3
    • None
    • None

    Description

      Configure a cluster such that a task attempt will be killed if any container log exceeds a configured size. This would help prevent logs from filling disks and also prevent the need to aggregate enormous logs.

      Attachments

        1. YARN.10471.001.patch
          24 kB
          Eric Payne
        2. YARN.10471.002.patch
          24 kB
          Eric Payne
        3. YARN.10471.003.patch
          24 kB
          Eric Payne
        4. YARN.10471.004.patch
          26 kB
          Eric Payne
        5. YARN.10471.005.patch
          26 kB
          Eric Payne
        6. YARN.10471.branch-3.2.003.patch
          24 kB
          Eric Payne
        7. YARN.10471.branch-3.2.005.patch
          25 kB
          Eric Payne

        Issue Links

          is related to

          Sub-task - The sub-task of the issue YARN-950 Ability to limit or avoid aggregating logs beyond a certain size

          • Major - Major loss of function.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. YARN-10743 Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              epayne Eric Payne
              epayne Eric Payne
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: