Hadoop Common
  1. Hadoop Common
  2. HADOOP-205

the job tracker does not schedule enough map on the cluster

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.2.0
    • Fix Version/s: 0.3.0
    • Component/s: None
    • Labels:
      None

      Description

      The job tracker during my big sort runs only has 80-120 maps running at a time with 2 tasks/node and 179 nodes and a large queue of map tasks. It seems to be caused by the load balancing using the current load rather than the required load to run all of the queued tasks.

      1. maplaunch.patch
        9 kB
        Mahadev konar

        Activity

        Hide
        Mahadev konar added a comment -

        This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.

        Show
        Mahadev konar added a comment - This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Mahadev!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Mahadev!

          People

          • Assignee:
            Mahadev konar
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development