Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-205

the job tracker does not schedule enough map on the cluster

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 0.2.0
        • Fix Version/s: 0.3.0
        • Component/s: None
        • Labels:
          None

          Description

          The job tracker during my big sort runs only has 80-120 maps running at a time with 2 tasks/node and 179 nodes and a large queue of map tasks. It seems to be caused by the load balancing using the current load rather than the required load to run all of the queued tasks.

          1. Text File
            maplaunch.patch
            9 kB
            Mahadev konar

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            mahadev Mahadev konar added a comment -

            This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.

            Show
            mahadev Mahadev konar added a comment - This patch allows map/reduce jobs to be scheduled using the pending load rather than the current load. Also, I have made changes in the JobInProgress to keep a running count of number of running/finished maps/reduces.
            Hide
            Permalink
            cutting Doug Cutting added a comment -

            I just committed this. Thanks, Mahadev!

            Show
            cutting Doug Cutting added a comment - I just committed this. Thanks, Mahadev!

              People

              • Assignee:
                mahadev Mahadev konar
                Reporter:
                owen.omalley Owen O'Malley
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development