Hadoop Common
  1. Hadoop Common
  2. HADOOP-205

the job tracker does not schedule enough map on the cluster

        Details

        • Type: Bug Bug
        • Status: Closed
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: 0.2.0
        • Fix Version/s: 0.3.0
        • Component/s: None
        • Labels:
          None

          Description

          The job tracker during my big sort runs only has 80-120 maps running at a time with 2 tasks/node and 179 nodes and a large queue of map tasks. It seems to be caused by the load balancing using the current load rather than the required load to run all of the queued tasks.

          1. Text File
            maplaunch.patch
            9 kB
            Mahadev konar

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Mahadev konar added a comment -

            This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.

            Show
            Mahadev konar added a comment - This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.
            Hide
            Permalink
            Doug Cutting added a comment -

            I just committed this. Thanks, Mahadev!

            Show
            Doug Cutting added a comment - I just committed this. Thanks, Mahadev!

              People

              • Assignee:
                Mahadev konar
                Reporter:
                Owen O'Malley
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development