Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-882

Specify per user quota for private/application cache and user log files

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      At present there is no limit on the number of files / size of the files localized by single user. Similarly there is no limit on the size of the log files created by user via running containers.

      We need to restrict the user for this.
      For LocalizedResources; this has serious concerns in case of secured environment where malicious user can start one container and localize resources whose total size >= DEFAULT_NM_LOCALIZER_CACHE_TARGET_SIZE_MB. Thereafter it will either fail (if no extra space is present on disk) or deletion service will keep removing localized files for other containers/applications.
      The limit for logs/localized resources should be decided by RM and sent to NM via secured containerToken. All these configurations should per container instead of per user or per nm.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ojoshi Omkar Vinit Joshi
            ojoshi Omkar Vinit Joshi

            Dates

              Created:
              Updated:

              Slack

                Issue deployment