Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13021

Incorrect storage policy of snapshot file was returned by getStoragePolicy command

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0
    • Fix Version/s: None
    • Component/s: hdfs, snapshots
    • Labels:
      None

      Description

      Snapshots are supposed to be immutable and read only, so the file status which in a snapshot path shouldn't follow the original file's change.

      The StoragePolicy in snapshot situation acts like a bug now.

      -----------

      Reproduction´╝ÜOperation on snapshottable dir /storagePolicy

      before make snapshot:

      
       [bin]# hdfs storagepolicies -setStoragePolicy -path /storagePolicy -policy PROVIDED
       Set storage policy PROVIDED on /storagePolicy
      
       [bin]# hadoop fs -put /home/file /storagePolicy/file_PROVIDED
      
       [bin]# hdfs storagepolicies -getStoragePolicy -path /storagePolicy/file_PROVIDED
       The storage policy of /storagePolicy/file_PROVIDED:
       BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
      
      

      make snapshot and check:

      
      [bin]# hdfs dfs -createSnapshot /storagePolicy s3_PROVIDED
      Created snapshot /storagePolicy/.snapshot/s3_PROVIDED
      
      [bin]# hdfs storagepolicies -getStoragePolicy -path /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED
      The storage policy of /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED:
      BlockStoragePolicy{PROVIDED:1, storageTypes=[PROVIDED, DISK], creationFallbacks=[PROVIDED, DISK], replicationFallbacks=[PROVIDED, DISK]} 
      
      

      change the StroagePolicy and check again:

      [bin]# hdfs storagepolicies -setStoragePolicy -path /storagePolicy -policy HOT
      Set storage policy HOT on /storagePolicy
      
      [bin]# hdfs storagepolicies -getStoragePolicy -path /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED
      The storage policy of /storagePolicy/.snapshot/s3_PROVIDED/file_PROVIDED:
      BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}   ---- It shouldn't be HOT
      

        Attachments

        1. HDFS-13021.001.patch
          16 kB
          LiXin Ge

          Activity

            People

            • Assignee:
              GeLiXin LiXin Ge
              Reporter:
              GeLiXin LiXin Ge
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: