Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21070

SnapshotFileCache won't update for snapshots stored in S3

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      The SnapshotFileCache depends on last modified time to determine whether to update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. S3 filesystems create a dummy file in place of a folder, but the dummy file last modified time is not updated when files are changed 'under' it. This means that the SnapshotFileCache doesn't pick up new snapshot HFiles and these files aren't removed from the HFileCleaner and can be eligible for deletion.

       

      My patch removes the lastmodified assumption.

      Attachments

        1. HBASE-21070.master.001.patch
          7 kB
          Zach York
        2. HBASE-21070.master.002.patch
          7 kB
          Zach York
        3. HBASE-21070.master.003.patch
          14 kB
          Zach York

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            zyork Zach York
            zyork Zach York
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment