Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2314

ContainerManagementProtocolProxy can create thousands of threads for a large cluster

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.1.0-beta
    • 2.6.0
    • client
    • None
    • Reviewed

    Description

      ContainerManagementProtocolProxy has a cache of NM proxies, and the size of this cache is configurable. However the cache can grow far beyond the configured size when running on a large cluster and blow AM address/container limits. More details in the first comment.

      Attachments

        1. disable-cm-proxy-cache.patch
          7 kB
          Jason Darrell Lowe
        2. nmproxycachefix.prototype.patch
          3 kB
          Jason Darrell Lowe
        3. tez-yarn-2314.xlsx
          64 kB
          Rajesh Balamohan
        4. YARN-2314.patch
          12 kB
          Jason Darrell Lowe
        5. YARN-2314v2.patch
          12 kB
          Jason Darrell Lowe

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jlowe Jason Darrell Lowe
            jlowe Jason Darrell Lowe
            Votes:
            1 Vote for this issue
            Watchers:
            25 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment