Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13724

Storage Tiering Show Paths with Policies applied

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Improvement Request to add an hdfs storagepolicies command to find paths for which storage policies have been applied.

      Right now you must explicitly query a single directory to get its policy to determine if one has been applied, but if another hadoop admin has configured policies on anything but trivially obvious paths such as /archive then there is no way to find which paths have policies applied to them other than by querying every single directory and subdirectory one by one which might potentially have a policy, eg:

      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1
      hdfs storagepolicies -getStoragePolicy -path /dir2
      hdfs storagepolicies -getStoragePolicy -path /dir3
      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1
      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir2
      hdfs storagepolicies -getStoragePolicy -path /dir3/subdir3
      ...
      hdfs storagepolicies -getStoragePolicy -path /dirN
      ...
      hdfs storagepolicies -getStoragePolicy -path /dirN/subdirN/subsubdirN
      ...

      In my current environment for example, a policy was configured for /data/blah which doesn't show when trying

       hdfs storagepolicies -getStoragePolicy -path /data

      and I had no way of knowing that I had to do:

       hdfs storagepolicies -getStoragePolicy -path /data/blah

      other than trial and error of trying every directory and every subdirectory in hdfs.

      Attachments

        Activity

          People

            Unassigned Unassigned
            harisekhon Hari Sekhon
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: