Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11197

Listing encryption zones fails when deleting a EZ that is on a snapshotted directory

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If a EZ directory is under a snapshotable directory, and a snapshot has been taking, then if this EZ is permanently deleted, it causes hdfs crypto listZones command to fail without showing any of the still available zones.

      This happens only after the EZ is removed from Trash folder. For example, considering /test-snap folder is snapshotable and there is already an snapshot for it:

      $ hdfs crypto -listZones
      /user/systest           my-key
      /test-snap/EZ-1       my-key
      
      $ hdfs dfs -rmr /test-snap/EZ-1
      INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/test-snap/EZ-1' to trash at: hdfs://ns1/user/hdfs/.Trash/Current/test-snap/EZ-1
      
      $ hdfs crypto -listZones
      /user/systest           my-key
      /user/hdfs/.Trash/Current/test-snap/EZ-1  my-key 
      
      $ hdfs dfs -rmr /user/hdfs/.Trash/Current/test-snap/EZ-1
      Deleted /user/hdfs/.Trash/Current/test-snap/EZ-1
      
      $ hdfs crypto -listZones
      RemoteException: Absolute path required
      

      Once this error happens, hdfs crypto -listZones only works again if we remove the snapshot:

      $ hdfs dfs -deleteSnapshot /test-snap snap1
      $ hdfs crypto -listZones
      /user/systest           my-key
      

      If we instead delete the EZ using skipTrash option, hdfs crypto -listZones does not break:

      $ hdfs crypto -listZones
      /user/systest           my-key
      /test-snap/EZ-2  my-key
      
      $ hdfs dfs -rmr -skipTrash /test-snap/EZ-2
      Deleted /test-snap/EZ-2
      
      $ hdfs crypto -listZones
      /user/systest           my-key
      

      The different behaviour seems to be because when removing the EZ trash folder, it's related INode is left with no parent INode. This causes EncryptionZoneManager.listEncryptionZones to throw the seen error, when trying to resolve the inodes in the given path.

      Am proposing a patch that fixes this issue by simply performing an additional check on EncryptionZoneManager.listEncryptionZones for the case an inode has no parent, so that it would be skipped on the list without trying to resolve it. Feedback on the proposal is appreciated.

        Attachments

        1. HDFS-11197-7.patch
          14 kB
          Wellington Chevreuil
        2. HDFS-11197-6.patch
          7 kB
          Wellington Chevreuil
        3. HDFS-11197-5.patch
          7 kB
          Wellington Chevreuil
        4. HDFS-11197-4.patch
          8 kB
          Wellington Chevreuil
        5. HDFS-11197-3.patch
          8 kB
          Wellington Chevreuil
        6. HDFS-11197-2.patch
          7 kB
          Wellington Chevreuil
        7. HDFS-11197-1.patch
          6 kB
          Wellington Chevreuil

          Issue Links

            Activity

              People

              • Assignee:
                wchevreuil Wellington Chevreuil
                Reporter:
                wchevreuil Wellington Chevreuil
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: