Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-253

Method to retrieve all quotas active on HDFS

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: namenode
    • Labels:

      Description

      Currently the only way to view quota information on an HDFS is via dfs -count -q, which is fine when an admin is examining a specific directory for quota status.

      It would also be good to do full HDFS quota audits, by pulling all HDFS quotas currently set on the system. This is especially important when trying to do capacity management ("OK, how much quota have we allotted so far?"). I think the only way to do this now is via lsr | count -q, which is pretty cumbersome.

        Issue Links

          Activity

          Marco Nicosia created issue -
          Owen O'Malley made changes -
          Field Original Value New Value
          Project Hadoop Common [ 12310240 ] HDFS [ 12310942 ]
          Key HADOOP-5290 HDFS-253
          Component/s dfs [ 12310710 ]
          Hide
          Allen Wittenauer added a comment -

          One of my favorite JIRAs. Still open! I think I'll add the newbie tag.

          Show
          Allen Wittenauer added a comment - One of my favorite JIRAs. Still open! I think I'll add the newbie tag.
          Allen Wittenauer made changes -
          Labels newbie
          Allen Wittenauer made changes -
          Component/s namenode [ 12312926 ]
          Allen Wittenauer made changes -
          Link This issue duplicates HDFS-2711 [ HDFS-2711 ]
          Surendra Singh Lilhore made changes -
          Assignee surendra singh lilhore [ surendrasingh ]
          Hide
          Surendra Singh Lilhore added a comment -

          Hi All,

          I am working on this jira and planning to do something like this.

          1. Adding one new admin command for list quota information from given path.

          	hdfs dfsadmin -listQuota -r -h <path>
          	

          This command will list the directories with quota information under the given path. Here "-r" is for recursively list the quota information.

          2. Adding one new API listQuotaInfo() in ClientProtocol.java for retrieving quota information from namenode , It will return DirectoriesQuotaInfo.

          	    public DirectoriesQuotaInfo listQuotaInfo(String path, int next) throws IOException;
                      
          	    public class DirectoriesQuotaInfo {  
          		private final ContentSummary[] contentSummarys;
          		private final int next;
          	    }
          	    

          3. One API in FileSystem.java, it will return RemoteIterator<ContentSummary>

          		public RemoteIterator<ContentSummary> listQuotaInfo(Path path) throws IOException;
          	    

          All the recursive logic I am implementing in admin command, listQuotaInfo() API only return the list of ContentSummary from the give path for the directories which has Quota.

          Is this correct implementation? Please give suggestion.

          Show
          Surendra Singh Lilhore added a comment - Hi All, I am working on this jira and planning to do something like this. 1. Adding one new admin command for list quota information from given path. hdfs dfsadmin -listQuota -r -h <path> This command will list the directories with quota information under the given path. Here "-r" is for recursively list the quota information. 2. Adding one new API listQuotaInfo() in ClientProtocol.java for retrieving quota information from namenode , It will return DirectoriesQuotaInfo . public DirectoriesQuotaInfo listQuotaInfo(String path, int next) throws IOException; public class DirectoriesQuotaInfo { private final ContentSummary[] contentSummarys; private final int next; } 3. One API in FileSystem.java, it will return RemoteIterator<ContentSummary> public RemoteIterator<ContentSummary> listQuotaInfo(Path path) throws IOException; All the recursive logic I am implementing in admin command, listQuotaInfo() API only return the list of ContentSummary from the give path for the directories which has Quota. Is this correct implementation? Please give suggestion.

            People

            • Assignee:
              Surendra Singh Lilhore
              Reporter:
              Marco Nicosia
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:

                Development