Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8219

setStoragePolicy with folder behavior is different after cluster restart

    XMLWordPrintableJSON

Details

    Description

      Reproduce steps.
      1) mkdir named /temp
      2) put one file A under /temp
      3) change /temp storage policy to COLD
      4) use -getStoragePolicy to query file A's storage policy, it is same with /temp
      5) change /temp folder storage policy again, will see file A's storage policy keep same with parent folder.

      then restart the cluster.
      do 3) 4) again, will find file A's storage policy is not change while parent folder's storage policy changes. It behaves different.

      As i debugged, found the code:
      in INodeFile.getStoragePolicyID

        public byte getStoragePolicyID() {
          byte id = getLocalStoragePolicyID();
          if (id == BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) {
            return this.getParent() != null ?
                this.getParent().getStoragePolicyID() : id;
          }
          return id;
        }
      

      If the file do not have its storage policy, it will use parent's. But after cluster restart, the file turns to have its own storage policy.

      Attachments

        1. HDFS-8219.patch
          3 kB
          Surendra Singh Lilhore
        2. HDFS-8219.unittest-norepro.patch
          3 kB
          Xiaoyu Yao

        Activity

          People

            surendralilhore Surendra Singh Lilhore
            shihaoliang Peter Shi
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: