Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5660

Log info about possible thrashing (when using memory-based scheduling in Capacity Scheduler) is not printed

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 1.2.1
    • 1.3.0
    • None
    • Reviewed

    Description

      There is a tiny, but confusing when troubleshooting, bug in TaskTracker code:

      if (totalMemoryAllottedForTasks > totalPhysicalMemoryOnTT) {
        LOG.info("totalMemoryAllottedForTasks > totalPhysicalMemoryOnTT."
            + " Thrashing might happen.");
      } else if (totalMemoryAllottedForTasks > totalVirtualMemoryOnTT) {
        LOG.info("totalMemoryAllottedForTasks > totalVirtualMemoryOnTT."
            + " Thrashing might happen.");
      }
      

      totalMemoryAllottedForTasks is calculated in megabytes, while totalPhysicalMemoryOnTT (and totalVirtualMemoryOnTT) is calculated in bytes. totalMemoryAllottedForTasks should be converted to bytes for a correct comparison.

      Attachments

        1. MAPREDUCE-5660.1.patch.txt
          1 kB
          Adam Kawa

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            kawaa Adam Kawa
            kawaa Adam Kawa
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment