Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5686

DefaultContainerExecutor random working dir algorigthm skews results

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      long randomPosition = RandomUtils.nextLong() % totalAvailable;
          ...
          while (randomPosition > availableOnDisk[dir]) {
            randomPosition -= availableOnDisk[dir++];
          }
      

      The code above selects a disk based on the random number weighted by the free space on each disk respectively. For example, if I have two disks with 100 bytes each, totalAvailable is 200. The value of randomPosition will be 0..199. 0..99 should select the first disk, 100..199 should select the second disk inclusively. Random number 100 should select the second disk to be fair but this is not the case right now.

      We need to use

      while (randomPosition >= availableOnDisk[dir])
      

      instead of

      while (randomPosition > availableOnDisk[dir])
      
      1. YARN-5686.001.patch
        5 kB
        Vrushali C
      2. YARN-5686.002.patch
        5 kB
        Vrushali C

        Activity

        Hide
        vrushalic Vrushali C added a comment -


        Uploading patch v001. Also added unit tests for checking the example case mentioned as well as other cases.

        Show
        vrushalic Vrushali C added a comment - Uploading patch v001. Also added unit tests for checking the example case mentioned as well as other cases.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 6s trunk passed
        +1 compile 0m 26s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 26s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 42s trunk passed
        +1 javadoc 0m 16s trunk passed
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 24s the patch passed
        +1 javac 0m 24s the patch passed
        -1 checkstyle 0m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 1 new + 22 unchanged - 0 fixed = 23 total (was 22)
        +1 mvnsite 0m 24s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 45s the patch passed
        +1 javadoc 0m 13s the patch passed
        -1 unit 14m 51s hadoop-yarn-server-nodemanager in the patch failed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        27m 52s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834272/YARN-5686.001.patch
        JIRA Issue YARN-5686
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ea7841391cf0 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e9c4616
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13448/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13448/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13448/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 6s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 42s trunk passed +1 javadoc 0m 16s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 24s the patch passed +1 javac 0m 24s the patch passed -1 checkstyle 0m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 1 new + 22 unchanged - 0 fixed = 23 total (was 22) +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 45s the patch passed +1 javadoc 0m 13s the patch passed -1 unit 14m 51s hadoop-yarn-server-nodemanager in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 27m 52s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834272/YARN-5686.001.patch JIRA Issue YARN-5686 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ea7841391cf0 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e9c4616 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13448/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13448/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13448/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you, for the patch Vrushali C!

        I think it would make sense to order these by the random number specified for better readability.

        497	    assertEquals(1, executor.pickDirectory(100L, availableOnDisk));
        498	    assertEquals(0, executor.pickDirectory(99L, availableOnDisk));
        499	    assertEquals(1, executor.pickDirectory(101L, availableOnDisk));
        500	    assertEquals(0, executor.pickDirectory(0L, availableOnDisk));
        501	    assertEquals(1, executor.pickDirectory(199L, availableOnDisk));
        
        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you, for the patch Vrushali C ! I think it would make sense to order these by the random number specified for better readability. 497 assertEquals(1, executor.pickDirectory(100L, availableOnDisk)); 498 assertEquals(0, executor.pickDirectory(99L, availableOnDisk)); 499 assertEquals(1, executor.pickDirectory(101L, availableOnDisk)); 500 assertEquals(0, executor.pickDirectory(0L, availableOnDisk)); 501 assertEquals(1, executor.pickDirectory(199L, availableOnDisk));
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Miklos Szegedi, yes will reorder those and also fix the checkstyle warning about ending the comment with a period shortly.

        Show
        vrushalic Vrushali C added a comment - Thanks Miklos Szegedi , yes will reorder those and also fix the checkstyle warning about ending the comment with a period shortly.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch 002 that reorders the test case data and fixes the checkstyle comment.

        Show
        vrushalic Vrushali C added a comment - Uploading patch 002 that reorders the test case data and fixes the checkstyle comment.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        +1 (non-binding) It looks good to me. Thank you, Vrushali C!

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - +1 (non-binding) It looks good to me. Thank you, Vrushali C !
        Hide
        varun_saxena Varun Saxena added a comment -

        +1. Will commit this pending Jenkins.

        Show
        varun_saxena Varun Saxena added a comment - +1. Will commit this pending Jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 10s trunk passed
        +1 compile 0m 29s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 28s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 43s trunk passed
        +1 javadoc 0m 18s trunk passed
        +1 mvninstall 0m 24s the patch passed
        +1 compile 0m 25s the patch passed
        +1 javac 0m 25s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 27s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 51s the patch passed
        +1 javadoc 0m 15s the patch passed
        -1 unit 15m 9s hadoop-yarn-server-nodemanager in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        29m 17s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue YARN-5686
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834492/YARN-5686.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ffdcf3092e85 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a895c66
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13538/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13538/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13538/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 10s trunk passed +1 compile 0m 29s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 28s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 43s trunk passed +1 javadoc 0m 18s trunk passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 25s the patch passed +1 javac 0m 25s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 51s the patch passed +1 javadoc 0m 15s the patch passed -1 unit 15m 9s hadoop-yarn-server-nodemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 29m 17s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-5686 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834492/YARN-5686.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ffdcf3092e85 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a895c66 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/13538/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13538/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13538/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        varun_saxena Varun Saxena added a comment -

        Test failure unrelated.
        Will commit it shortly.

        Show
        varun_saxena Varun Saxena added a comment - Test failure unrelated. Will commit it shortly.
        Hide
        varun_saxena Varun Saxena added a comment -

        Committed to trunk,branch-2.
        Thanks Vrushali C for your contribution.
        Thanks Miklos Szegedi for raising the issue and subsequent reviews.

        Show
        varun_saxena Varun Saxena added a comment - Committed to trunk,branch-2. Thanks Vrushali C for your contribution. Thanks Miklos Szegedi for raising the issue and subsequent reviews.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10700 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10700/)
        YARN-5686. DefaultContainerExecutor random working dir algorigthm skews (varunsaxena: rev 7b4e9ec3b03dfd8aa9375f823e3e8f4aebe86e6b)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10700 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10700/ ) YARN-5686 . DefaultContainerExecutor random working dir algorigthm skews (varunsaxena: rev 7b4e9ec3b03dfd8aa9375f823e3e8f4aebe86e6b) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java

          People

          • Assignee:
            vrushalic Vrushali C
            Reporter:
            miklos.szegedi@cloudera.com Miklos Szegedi
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development