Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13096

HDFS group quota

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.8.3, 2.7.5, 3.0.0
    • None
    • datanode, fs, hdfs, nn
    • None

    Description

      We have groups of people that have their own set of HDFS directories.
      For example, they have HDFS staging place for new files:
      /datascience
      /analysts
      ...
      but at the same time they have Hive warehouse directory
      /hivewarehouse/datascience
      /hivewarehouse/analysts
      ...
      on top of that they also have some files stored under /user/${username}/

      It's always been a challenge to maintain a combined quota on all HDFS locations a particular group of people owns. As we're currently forced to put a particular quota for each directory independently.

      It would be great if HDFS would have a quota tied either

      • to a set of HDFS locations ;
      • or to a group of people (where `group`is defined as which HDFS group a particular file/directory belongs to).

      Linux allows to define quotas at group level, i.e. `edquota -g devel` etc.. would be great to have the same at HDFS level.

      Other thoughts and ideas?

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            Tagar Ruslan Dautkhanov

            Dates

              Created:
              Updated:

              Slack

                Issue deployment