Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3104

MultithreadMapRunner keeps consuming records even if trheads are not available

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.16.1
    • Fix Version/s: 0.16.2
    • Component/s: None
    • Labels:
      None
    • Environment:

      all

      Description

      The ExecutorService execute() method does not block when there are not threads available, it queues up the runnables until there are threads.

      The problem is that all key/values are read and kept in memory for the task, with large datasets this will create a OOM exception.

      Have to figure out how to use the execute in blocking fashion.

        Attachments

        1. patch3104.txt
          7 kB
          Alejandro Abdelnur
        2. patch3104.txt
          5 kB
          Alejandro Abdelnur
        3. patch3104-2.txt
          7 kB
          Owen O'Malley
        4. patch3104-3.txt
          7 kB
          Owen O'Malley

          Activity

            People

            • Assignee:
              tucu00 Alejandro Abdelnur
              Reporter:
              tucu00 Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: