Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10471

Prevent logs for any container from becoming larger than a configurable size.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: 3.2.1, 3.1.4
    • Fix Version/s: 3.2.2, 3.4.0, 3.3.1, 3.2.3
    • Component/s: None
    • Labels:
      None

      Description

      Configure a cluster such that a task attempt will be killed if any container log exceeds a configured size. This would help prevent logs from filling disks and also prevent the need to aggregate enormous logs.

        Attachments

        1. YARN.10471.001.patch
          24 kB
          Eric Payne
        2. YARN.10471.002.patch
          24 kB
          Eric Payne
        3. YARN.10471.003.patch
          24 kB
          Eric Payne
        4. YARN.10471.004.patch
          26 kB
          Eric Payne
        5. YARN.10471.005.patch
          26 kB
          Eric Payne
        6. YARN.10471.branch-3.2.003.patch
          24 kB
          Eric Payne
        7. YARN.10471.branch-3.2.005.patch
          25 kB
          Eric Payne

          Issue Links

          is related to

          Sub-task - The sub-task of the issue YARN-950 Ability to limit or avoid aggregating logs beyond a certain size

          • Major - Major loss of function.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. YARN-10743 Add a policy for not aggregating for containers which are killed because exceeding container log size limit.

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                epayne Eric Payne
                Reporter:
                epayne Eric Payne
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: