Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10648

NM local logs are not cleared after uploading to hdfs

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.2.0
    • Fix Version/s: None
    • Component/s: log-aggregation
    • Labels:
      None

      Description

      YARN-8273 has induced the following issues.

      1. The delService.delete(deletionTask) has been removed from the for loop, and added at the end in finally block. Inside the for loop  we are creating FileDeletionTask for each container, but not storing it, due to this, only the last container log files will be present in the deletionTask and only those files will be removed. Ideally all the container log files which are uploaded must be deleted.
      2. The LogAggregationDFSException is caught in the closeswriter, but when we configure LogAggregationTFileController as logAggregationFileController,  this.logAggregationFileController.closeWriter()  itself calls closeWriter, which throws LogAggregationDFSException if any, and the exception is not saved. Again when we try to do closeWriter we dont get any exception and, we are not throwing the LogAggregationDFSException in this scenario.

        Attachments

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              dmmkr D M Murali Krishna Reddy Assign to me
              Reporter:
              dmmkr D M Murali Krishna Reddy

              Dates

              • Created:
                Updated:

                Issue deployment