Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5660

Log info about possible thrashing (when using memory-based scheduling in Capacity Scheduler) is not printed

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.2.1
    • Fix Version/s: 1.3.0
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There is a tiny, but confusing when troubleshooting, bug in TaskTracker code:

      if (totalMemoryAllottedForTasks > totalPhysicalMemoryOnTT) {
        LOG.info("totalMemoryAllottedForTasks > totalPhysicalMemoryOnTT."
            + " Thrashing might happen.");
      } else if (totalMemoryAllottedForTasks > totalVirtualMemoryOnTT) {
        LOG.info("totalMemoryAllottedForTasks > totalVirtualMemoryOnTT."
            + " Thrashing might happen.");
      }
      

      totalMemoryAllottedForTasks is calculated in megabytes, while totalPhysicalMemoryOnTT (and totalVirtualMemoryOnTT) is calculated in bytes. totalMemoryAllottedForTasks should be converted to bytes for a correct comparison.

        Attachments

        1. MAPREDUCE-5660.1.patch.txt
          1 kB
          Adam Kawa

          Activity

            People

            • Assignee:
              kawaa Adam Kawa
              Reporter:
              kawaa Adam Kawa
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: