Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7124

LogAggregationTFileController deletes/renames while file is open

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.9.0, 3.0.0-beta1
    • 2.9.0, 3.0.0, 3.1.0
    • nodemanager
    • None
    • Reviewed

    Description

      YARN-6288 changes the log aggregation writer to be an autoclosable. Unfortunately the try-with-resources block for the writer will either rename or delete the log while open.

      Assuming the NM's behavior is correct, deleting open files only results in ominous WARNs in the nodemanager log and increases the rate of logging in the NN when the implicit try-with-resource close fails. These red herrings complicate debugging efforts.

      Attachments

        1. YARN-7124.001.patch
          1 kB
          Jason Darrell Lowe

        Issue Links

          Activity

            People

              jlowe Jason Darrell Lowe
              daryn Daryn Sharp
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: