Hadoop Common
  1. Hadoop Common
  2. HADOOP-205

the job tracker does not schedule enough map on the cluster

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.2.0
    • Fix Version/s: 0.3.0
    • Component/s: None
    • Labels:
      None

      Description

      The job tracker during my big sort runs only has 80-120 maps running at a time with 2 tasks/node and 179 nodes and a large queue of map tasks. It seems to be caused by the load balancing using the current load rather than the required load to run all of the queued tasks.

      1. maplaunch.patch
        9 kB
        Mahadev konar

        Activity

        Owen O'Malley created issue -
        Hide
        Mahadev konar added a comment -

        This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.

        Show
        Mahadev konar added a comment - This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.
        Mahadev konar made changes -
        Field Original Value New Value
        Attachment maplaunch.patch [ 12334430 ]
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Mahadev!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Mahadev!
        Doug Cutting made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Fix Version/s 0.4 [ 12311021 ]
        Fix Version/s 0.3 [ 12310930 ]
        Assignee Owen O'Malley [ owen.omalley ] Mahadev konar [ mahadev ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Doug Cutting made changes -
        Workflow jira [ 12362341 ] no reopen closed [ 12373119 ]
        Doug Cutting made changes -
        Workflow no reopen closed [ 12373119 ] no-reopen-closed [ 12373455 ]
        Doug Cutting made changes -
        Workflow no-reopen-closed [ 12373455 ] no-reopen-closed, patch-avail [ 12377767 ]
        Owen O'Malley made changes -
        Component/s mapred [ 12310690 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        14d 1h 59m 1 Doug Cutting 24/May/06 05:56
        Resolved Resolved Closed Closed
        13d 10m 1 Doug Cutting 06/Jun/06 06:06

          People

          • Assignee:
            Mahadev konar
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development