Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7124

LogAggregationTFileController deletes/renames while file is open

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.9.0, 3.0.0-beta1
    • Fix Version/s: 2.9.0, 3.0.0, 3.1.0
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      YARN-6288 changes the log aggregation writer to be an autoclosable. Unfortunately the try-with-resources block for the writer will either rename or delete the log while open.

      Assuming the NM's behavior is correct, deleting open files only results in ominous WARNs in the nodemanager log and increases the rate of logging in the NN when the implicit try-with-resource close fails. These red herrings complicate debugging efforts.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jlowe Jason Lowe
                Reporter:
                daryn Daryn Sharp
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: