Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4768

getAvailablePhysicalMemorySize can be inaccurate on linux

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.2, 3.0.0-alpha1
    • 2.8.0, 3.0.0-alpha1
    • nodemanager
    • None
    • Linux

    Description

      Algorithm currently uses "MemFree" + "Inactive" from /proc/meminfo

      "Inactive" may not be a very good indication of how much memory can be readily freed because it contains both:

      • Pages mapped with MAP_SHARED|MAP_ANONYMOUS (regardless of whether they're being actively accessed or not. Unclear to me why this is the case...)
      • Pages mapped MAP_PRIVATE|MAP_ANONYMOUS that have not been accessed recently

      Both of these types of pages probably shouldn't be considered "Available".

      "Inactive(file)" would seem more accurate but it's not available in all kernel versions. To keep things simple, maybe just use "Inactive(file)" if available, otherwise fallback to "Inactive".

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            nroberts Nathan Roberts Assign to me
            nroberts Nathan Roberts
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment