Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13021

Incorrect storage policy of snapshot file was returned by getStoragePolicy command

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 3.1.0
    • None
    • hdfs, snapshots
    • None

    Description

      Snapshots are supposed to be immutable and read only, so the file status which in a snapshot path shouldn't follow the original file's change.

      The StoragePolicy in snapshot situation acts like a bug now.

      -----------

      Reproduction:Operation on snapshottable dir /storagePolicy

      before make snapshot:

      
       [bin]# hdfs storagepolicies -setStoragePolicy -path /storagePolicy -policy PROVIDED
       Set storage policy PROVIDED on /storagePolicy
      
       [bin]# hadoop fs -put /home/file /storagePolicy/file_PROVIDED
      
       [bin]# hdfs storagepolicies -getStoragePolicy -path /storagePolicy/file_PROVIDED
       The storage policy of /storagePolicy/file_PROVIDED:
       BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
      
      

      make snapshot and check:

      
      [bin]# hdfs dfs -createSnapshot /storagePolicy s3_PROVIDED
      Created snapshot /storagePolicy/.snapshot/s3_PROVIDED
      
      [bin]# hdfs storagepolicies -getStoragePolicy -path /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED
      The storage policy of /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED:
      BlockStoragePolicy{PROVIDED:1, storageTypes=[PROVIDED, DISK], creationFallbacks=[PROVIDED, DISK], replicationFallbacks=[PROVIDED, DISK]} 
      
      

      change the StroagePolicy and check again:

      [bin]# hdfs storagepolicies -setStoragePolicy -path /storagePolicy -policy HOT
      Set storage policy HOT on /storagePolicy
      
      [bin]# hdfs storagepolicies -getStoragePolicy -path /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED
      The storage policy of /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED:
      BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}   ---- It shouldn't be HOT
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            GeLiXin LiXin Ge Assign to me
            GeLiXin LiXin Ge

            Dates

              Created:
              Updated:

              Slack

                Issue deployment