Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13724

Storage Tiering Show Paths with Policies applied

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Improvement Request to add an hdfs storagepolicies command to find paths for which storage policies have been applied.

      Right now you must explicitly query a single directory to get its policy to determine if one has been applied, but if another hadoop admin has configured policies on anything but trivially obvious paths such as /archive then there is no way to find which paths have policies applied to them other than by querying every single directory and subdirectory one by one which might potentially have a policy, eg:

      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1
      hdfs storagepolicies -getStoragePolicy -path /dir2
      hdfs storagepolicies -getStoragePolicy -path /dir3
      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1
      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir2
      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir3
      ...
      hdfs storagepolicies -getStoragePolicy -path /dirN
      ...
      hdfs storagepolicies -getStoragePolicy -path /dirN/subdirN/subsubdirN
      ...

      In my current environment for example, a policy was configured for /data/blah which doesn't show when trying

       hdfs storagepolicies -getStoragePolicy -path /data

      and I had no way of knowing that I had to do:

       hdfs storagepolicies -getStoragePolicy -path /data/blah

      other than trial and error of trying every directory and every subdirectory in hdfs.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              harisekhon Hari Sekhon
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: