Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6889

Add Job#close API to shutdown MR client services.

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-beta1
    • None
    • None
    • Reviewed

    Description

      ATS1.5 uses FileSystemTimelineWriter which creates FS object on every writer initialization. If writer is not closed, then there is possibility of OOM see YARN-5438 fixes closing FS object.

      TimelineClient is used by YarnClient. So all the user who uses YarnClient with ATS1.5 need to stop service properly. Otherwise there is big chance of FS object leak.

      Of course MR uses YARN client submit job. If MR do not stop YarnClient then there is FS object leak.

      JobClient provides a API to stop all these service using JobClient#close. But many MR clients uses Job object to submit a job. But do not stop started services by default.

      So, provide a API Job#close to shutdown MR client services. This API can be utilized by caller explicitly to stop service which avoids FS leak.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rohithsharma Rohith Sharma K S Assign to me
            rohithsharma Rohith Sharma K S
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment