Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11197

Listing encryption zones fails when deleting a EZ that is on a snapshotted directory

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.6.0
    • 2.8.0, 3.0.0-alpha2
    • hdfs
    • None
    • Reviewed


      If a EZ directory is under a snapshotable directory, and a snapshot has been taking, then if this EZ is permanently deleted, it causes hdfs crypto listZones command to fail without showing any of the still available zones.

      This happens only after the EZ is removed from Trash folder. For example, considering /test-snap folder is snapshotable and there is already an snapshot for it:

      $ hdfs crypto -listZones
      /user/systest           my-key
      /test-snap/EZ-1       my-key
      $ hdfs dfs -rmr /test-snap/EZ-1
      INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/test-snap/EZ-1' to trash at: hdfs://ns1/user/hdfs/.Trash/Current/test-snap/EZ-1
      $ hdfs crypto -listZones
      /user/systest           my-key
      /user/hdfs/.Trash/Current/test-snap/EZ-1  my-key 
      $ hdfs dfs -rmr /user/hdfs/.Trash/Current/test-snap/EZ-1
      Deleted /user/hdfs/.Trash/Current/test-snap/EZ-1
      $ hdfs crypto -listZones
      RemoteException: Absolute path required

      Once this error happens, hdfs crypto -listZones only works again if we remove the snapshot:

      $ hdfs dfs -deleteSnapshot /test-snap snap1
      $ hdfs crypto -listZones
      /user/systest           my-key

      If we instead delete the EZ using skipTrash option, hdfs crypto -listZones does not break:

      $ hdfs crypto -listZones
      /user/systest           my-key
      /test-snap/EZ-2  my-key
      $ hdfs dfs -rmr -skipTrash /test-snap/EZ-2
      Deleted /test-snap/EZ-2
      $ hdfs crypto -listZones
      /user/systest           my-key

      The different behaviour seems to be because when removing the EZ trash folder, it's related INode is left with no parent INode. This causes EncryptionZoneManager.listEncryptionZones to throw the seen error, when trying to resolve the inodes in the given path.

      Am proposing a patch that fixes this issue by simply performing an additional check on EncryptionZoneManager.listEncryptionZones for the case an inode has no parent, so that it would be skipped on the list without trying to resolve it. Feedback on the proposal is appreciated.


        1. HDFS-11197-1.patch
          6 kB
          Wellington Chevreuil
        2. HDFS-11197-2.patch
          7 kB
          Wellington Chevreuil
        3. HDFS-11197-3.patch
          8 kB
          Wellington Chevreuil
        4. HDFS-11197-4.patch
          8 kB
          Wellington Chevreuil
        5. HDFS-11197-5.patch
          7 kB
          Wellington Chevreuil
        6. HDFS-11197-6.patch
          7 kB
          Wellington Chevreuil
        7. HDFS-11197-7.patch
          14 kB
          Wellington Chevreuil

        Issue Links


          This comment will be Viewable by All Users Viewable by All Users


            wchevreuil Wellington Chevreuil
            wchevreuil Wellington Chevreuil
            0 Vote for this issue
            6 Start watching this issue




                Issue deployment