Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5551

Ignore file backed pages from memory computation when smaps is enabled

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha2
    • None
    • None
    • Reviewed

    Description

      Currently deleted file mappings are also included in the memory computation when SMAP is enabled. For e.g

      7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513                 /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185 (deleted)
      Size:                  8 kB
      Rss:                   4 kB
      Pss:                   2 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          4 kB
      Private_Clean:         0 kB
      Private_Dirty:         0 kB
      Referenced:            4 kB
      Anonymous:             0 kB
      AnonHugePages:         0 kB
      Swap:                  0 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      
      
      7fbf28000000-7fbf68000000 rw-s 00000000 08:02 11927571                   /tmp/7298569189125604642/arena-1291157252088664681.cache (deleted)
      Size:            1048576 kB
      Rss:               17288 kB
      Pss:               17288 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          0 kB
      Private_Clean:       232 kB
      Private_Dirty:     17056 kB
      Referenced:        17288 kB
      Anonymous:             0 kB
      AnonHugePages:         0 kB
      Swap:                  0 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      

      It would be good to exclude these from getSmapBasedRssMemorySize() computation.

      Attachments

        1. YARN-5551.branch-2.001.patch
          9 kB
          Rajesh Balamohan
        2. YARN-5551.branch-2.002.patch
          8 kB
          Rajesh Balamohan
        3. YARN-5551.branch-2.003.patch
          9 kB
          Rajesh Balamohan

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rajesh.balamohan Rajesh Balamohan
            rajesh.balamohan Rajesh Balamohan
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment