Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5551

Ignore file backed pages from memory computation when smaps is enabled

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently deleted file mappings are also included in the memory computation when SMAP is enabled. For e.g

      7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513                 /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185 (deleted)
      Size:                  8 kB
      Rss:                   4 kB
      Pss:                   2 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          4 kB
      Private_Clean:         0 kB
      Private_Dirty:         0 kB
      Referenced:            4 kB
      Anonymous:             0 kB
      AnonHugePages:         0 kB
      Swap:                  0 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      
      
      7fbf28000000-7fbf68000000 rw-s 00000000 08:02 11927571                   /tmp/7298569189125604642/arena-1291157252088664681.cache (deleted)
      Size:            1048576 kB
      Rss:               17288 kB
      Pss:               17288 kB
      Shared_Clean:          0 kB
      Shared_Dirty:          0 kB
      Private_Clean:       232 kB
      Private_Dirty:     17056 kB
      Referenced:        17288 kB
      Anonymous:             0 kB
      AnonHugePages:         0 kB
      Swap:                  0 kB
      KernelPageSize:        4 kB
      MMUPageSize:           4 kB
      

      It would be good to exclude these from getSmapBasedRssMemorySize() computation.

        Attachments

        1. YARN-5551.branch-2.001.patch
          9 kB
          Rajesh Balamohan
        2. YARN-5551.branch-2.002.patch
          8 kB
          Rajesh Balamohan
        3. YARN-5551.branch-2.003.patch
          9 kB
          Rajesh Balamohan

          Activity

            People

            • Assignee:
              rajesh.balamohan Rajesh Balamohan
              Reporter:
              rajesh.balamohan Rajesh Balamohan
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: