Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8766 Support for Node Attribute Functionality, Phase 2
  3. YARN-8925

Updating distributed node attributes only when necessary

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Currently if distributed node attributes exist, even though there is no change, updating for distributed node attributes will happen in every heartbeat between NM and RM. Updating process will hold NodeAttributesManagerImpl#writeLock and may have some influence in a large cluster. We have found nodes UI of a large cluster is opened slowly and most time it's waiting for the lock in NodeAttributesManagerImpl. I think this updating should be called only when necessary to enhance the performance of related process.

      Attachments

        1. YARN-8925.001.patch
          12 kB
          Tao Yang
        2. YARN-8925.002.patch
          69 kB
          Tao Yang
        3. YARN-8925.003.patch
          82 kB
          Tao Yang
        4. YARN-8925.004.patch
          86 kB
          Tao Yang
        5. YARN-8925.005.patch
          86 kB
          Tao Yang
        6. YARN-8925.006.patch
          86 kB
          Tao Yang
        7. YARN-8925.007.patch
          86 kB
          Tao Yang
        8. YARN-8925.008.patch
          87 kB
          Tao Yang
        9. YARN-8925.009.patch
          87 kB
          Tao Yang
        10. YARN-8925.010.patch
          91 kB
          Tao Yang
        11. YARN-8925-branch-3.2.001.patch
          91 kB
          Tao Yang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Tao Yang Tao Yang
            Tao Yang Tao Yang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment