Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1974

FairScheduler can preempt the same task many times

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0, 0.21.1, 0.22.0, 0.23.0
    • Fix Version/s: 0.22.0
    • Component/s: contrib/fair-share
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In FairScheduler.preemptTasks(), tasks are collected from JobInProgress.runningMapCache.
      But tasks repeat multiple times in JobInProgress.runningMapCache (on rack, node and cluster).
      This makes FairScheduler preempt the same task many times.

        Activity

        Hide
        Matei Zaharia added a comment -

        The patch looks good to me. Is there any way to test this? (Did you at least try to print the list before and see that a task is in there multiple times?)

        Show
        Matei Zaharia added a comment - The patch looks good to me. Is there any way to test this? (Did you at least try to print the list before and see that a task is in there multiple times?)
        Hide
        Scott Chen added a comment -

        Hi Matei,
        I see multiple same preemption in the log. I think I should add a test here to verify that.

        Scott

        Show
        Scott Chen added a comment - Hi Matei, I see multiple same preemption in the log. I think I should add a test here to verify that. Scott
        Hide
        Matei Zaharia added a comment -

        Did you guys fix this one at Facebook? It would be nice to put it into trunk and 0.21.1 too.

        Show
        Matei Zaharia added a comment - Did you guys fix this one at Facebook? It would be nice to put it into trunk and 0.21.1 too.
        Hide
        Scott Chen added a comment -

        Hey Matei,

        Sorry for not responding on this for a while.
        We have put this in production for a long while and it fixes the problem.
        I think it is OK to commit this.

        Show
        Scott Chen added a comment - Hey Matei, Sorry for not responding on this for a while. We have put this in production for a long while and it fixes the problem. I think it is OK to commit this.
        Hide
        Joydeep Sen Sarma added a comment -

        +1. this seems like a no-brainer!

        Show
        Joydeep Sen Sarma added a comment - +1. this seems like a no-brainer!
        Hide
        Scott Chen added a comment -
        
             [exec]
             [exec] BUILD SUCCESSFUL
             [exec] Total time: 2 minutes 15 seconds
             [exec]
             [exec]
             [exec]
             [exec]
             [exec] +1 overall.
             [exec]
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec]
             [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
             [exec]
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec]
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec]
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.
             [exec]
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
             [exec]
             [exec]     +1 system test framework.  The patch passed system test framework compile.
             [exec]
             [exec]
             [exec]
             [exec]
             [exec] ======================================================================
             [exec] ======================================================================
             [exec]     Finished build.
             [exec] ======================================================================
             [exec] ======================================================================
        
        Show
        Scott Chen added a comment - [exec] [exec] BUILD SUCCESSFUL [exec] Total time: 2 minutes 15 seconds [exec] [exec] [exec] [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ======================================================================
        Hide
        Scott Chen added a comment -

        I posted the result on the wrong patch. It should not have a +1 on test included.
        Will run this again.

        Show
        Scott Chen added a comment - I posted the result on the wrong patch. It should not have a +1 on test included. Will run this again.
        Hide
        Scott Chen added a comment -

        test-patch result

        [exec]
        [exec]
        [exec] -1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
        [exec] Please justify why no new tests are needed for this patch.
        [exec] Also please list what manual steps were performed to verify this patch.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.
        [exec]
        [exec]
        [exec]
        [exec]
        [exec] ======================================================================
        [exec] ======================================================================
        [exec] Finished build.
        [exec] ======================================================================
        [exec] ======================================================================

        Show
        Scott Chen added a comment - test-patch result [exec] [exec] [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ======================================================================
        Hide
        Priyo Mustafi added a comment -

        I have tested this as well and all looks good

        Show
        Priyo Mustafi added a comment - I have tested this as well and all looks good
        Hide
        Konstantin Shvachko added a comment -

        I just committed this. Thank you Scott.

        Show
        Konstantin Shvachko added a comment - I just committed this. Thank you Scott.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #603 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/603/)
        MAPREDUCE-1974. Fix multiple preemtions of the same task in FairScheduler. Contributed by Scott Chen.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #603 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/603/ ) MAPREDUCE-1974 . Fix multiple preemtions of the same task in FairScheduler. Contributed by Scott Chen.
        Hide
        Scott Chen added a comment -

        Thanks for the help, guys

        Show
        Scott Chen added a comment - Thanks for the help, guys
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-22-branch #33 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/33/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-22-branch #33 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/33/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )

          People

          • Assignee:
            Scott Chen
            Reporter:
            Scott Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development