Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12352

Delay in checkpointing Trash can leave trash for 2 intervals before deleting

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 2.7.1
    • 2.8.0, 3.0.0-alpha1
    • trash
    • None
    • Reviewed
    • Fixes an Trash related issue wherein a delay in the periodic checkpointing of one user's directory causes the subsequent user directory checkpoints to carry a newer timestamp, thereby delaying their eventual deletion.

    Description

      When the fs.trash.checkpoint.interval and the fs.trash.interval are set non-zero and the same, it is possible for trash to be left for two intervals.

      The TrashPolicyDefault will use a floor and ceiling function to ensure that the Trash will be checkpointed every "interval" of minutes.

      Each user's trash is checkpointed individually. The time resolution of the checkpoint timestamp is to the second.

      If the seconds switch while one user is checkpointing, then the next user's timestamp will be later.

      This will cause the next user's checkpoint to not be deleted at the next interval.

      I have recreated this in a lab cluster

      I also have a suggestion for a patch that I can upload later tonight after testing it further.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            caseyjbrotherton Casey Brotherton Assign to me
            caseyjbrotherton Casey Brotherton
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment