Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Improvement Request to add an hdfs storagepolicies command to find paths for which storage policies have been applied.
Right now you must explicitly query a single directory to get its policy to determine if one has been applied, but if another hadoop admin has configured policies on anything but trivially obvious paths such as /archive then there is no way to find which paths have policies applied to them other than by querying every single directory and subdirectory one by one which might potentially have a policy, eg:
hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1 hdfs storagepolicies -getStoragePolicy -path /dir2 hdfs storagepolicies -getStoragePolicy -path /dir3 hdfs storagepolicies -getStoragePolicy -path /dir3/subdir1 hdfs storagepolicies -getStoragePolicy -path /dir3/subdir2 hdfs storagepolicies -getStoragePolicy -path /dir3/subdir3 ... hdfs storagepolicies -getStoragePolicy -path /dirN ... hdfs storagepolicies -getStoragePolicy -path /dirN/subdirN/subsubdirN ...
In my current environment for example, a policy was configured for /data/blah which doesn't show when trying
hdfs storagepolicies -getStoragePolicy -path /data
and I had no way of knowing that I had to do:
hdfs storagepolicies -getStoragePolicy -path /data/blah
other than trial and error of trying every directory and every subdirectory in hdfs.