Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      observing the cluster over the last day - one thing i noticed is that small jobs (single digit tasks) are not doing a good job competing against large jobs. what seems to happen is that:

      • large job comes along and needs to wait for a while for other large jobs.
      • slots are slowly transfered from one large job to another
      • small tasks keep waiting forever.

      is this an artifact of deficit based scheduling? it seems that long pending large jobs are out-scheduling small jobs

        Activity

        Joydeep Sen Sarma created issue -
        Joydeep Sen Sarma made changes -
        Field Original Value New Value
        Assignee Matei Zaharia [ matei@eecs.berkeley.edu ]
        Owen O'Malley made changes -
        Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
        Key HADOOP-4803 MAPREDUCE-543
        Component/s contrib/fair-share [ 12312456 ]
        Matei Zaharia made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 0.21.0 [ 12314045 ]
        Resolution Duplicate [ 3 ]
        Matei Zaharia made changes -
        Assignee Matei Zaharia [ matei@eecs.berkeley.edu ] Matei Zaharia [ matei ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Matei Zaharia
            Reporter:
            Joydeep Sen Sarma
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development