Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-224

limit running tasks per job

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      It should be possible to specify a limit to the number of tasks per job permitted to run simultaneously. If, for example, you have a cluster of 50 nodes, with 100 map task slots and 100 reduce task slots, and the configured limit is 25 simultaneous tasks/job, then four or more jobs will be able to run at a time. This will permit short jobs to pass longer-running jobs. This also avoids some problems we've seen with HOD, where nodes are underutilized in their tail, and it should permit improved input locality.

      Attachments

        Issue Links

          is part of

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-3412 Refactor the scheduler out of the JobTracker

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5583 Ability to limit running map and reduce tasks

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              cutting Doug Cutting
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: