Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11203

Rename support during re-encrypt EDEK

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • encryption
    • None

    Description

      Currently HDFS-10899 disables renames within the EZ if it's under re-encryption. (similar to current cross-zone rename checks).

      We'd like to support rename in the long run, so cluster is fully functioning during re-encryption.

      The reason rename is particularly difficult is:

      • We want to re-encrypt all files under an EZ in one pass, without missing any
      • We want to iterate through the files and keep track of where we are (i.e. a cursor), so in case of NN failover/crash, we can resume from fsimage/edits.
      • We cannot guarantee namespace is not changed during re-encryption. Newly created files automatically has new edek, deleted files we don't care. But if a file is renamed from behind the cursor to before, it may be missed in the re-encryption.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            xiaochen Xiao Chen
            xiaochen Xiao Chen

            Dates

              Created:
              Updated:

              Slack

                Issue deployment