Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11203

Rename support during re-encrypt EDEK

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: encryption
    • Labels:
      None
    • Target Version/s:

      Description

      Currently HDFS-10899 disables renames within the EZ if it's under re-encryption. (similar to current cross-zone rename checks).

      We'd like to support rename in the long run, so cluster is fully functioning during re-encryption.

      The reason rename is particularly difficult is:

      • We want to re-encrypt all files under an EZ in one pass, without missing any
      • We want to iterate through the files and keep track of where we are (i.e. a cursor), so in case of NN failover/crash, we can resume from fsimage/edits.
      • We cannot guarantee namespace is not changed during re-encryption. Newly created files automatically has new edek, deleted files we don't care. But if a file is renamed from behind the cursor to before, it may be missed in the re-encryption.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                xiaochen Xiao Chen
                Reporter:
                xiaochen Xiao Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: