Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6673

Add a test example job that grows in memory usage over time

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      While working on YARN-1011, I needed to put together an example that would have tasks increase their resource usage deterministically over time. It would be useful for any other utilization related work or stress tests.

      1. mr-6673-1.patch
        5 kB
        Karthik Kambatla
      2. MAPREDUCE-6673.2.patch
        5 kB
        Karthik Kambatla

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11592 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11592/)
        MAPREDUCE-6673. Add a test example job that grows in memory usage over (jlowe: rev ac3cfdf3e5927eade0e325d4d5fe392ddcc667b0)

        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/test/MapredTestDriver.java
        • (add) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/GrowingSleepJob.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11592 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11592/ ) MAPREDUCE-6673 . Add a test example job that grows in memory usage over (jlowe: rev ac3cfdf3e5927eade0e325d4d5fe392ddcc667b0) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/test/MapredTestDriver.java (add) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/GrowingSleepJob.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11589 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11589/)
        MAPREDUCE-6673. Add a test example job that grows in memory usage over (haibochen: rev 25ac44709b4bbed78b607ea48021237b64e01b9f)

        • (add) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/GrowingSleepJob.java
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/test/MapredTestDriver.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11589 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11589/ ) MAPREDUCE-6673 . Add a test example job that grows in memory usage over (haibochen: rev 25ac44709b4bbed78b607ea48021237b64e01b9f) (add) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/GrowingSleepJob.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/test/MapredTestDriver.java
        Hide
        haibochen Haibo Chen added a comment -

        Thanks Karthik Kambatla for the contribution! I committed it to trunk and branch-2.

        Show
        haibochen Haibo Chen added a comment - Thanks Karthik Kambatla for the contribution! I committed it to trunk and branch-2.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 14m 12s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 26s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 24s trunk passed
        +1 javadoc 0m 12s trunk passed
        +1 mvninstall 0m 23s the patch passed
        +1 compile 0m 20s the patch passed
        +1 javac 0m 20s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 23s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 29s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 99m 45s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 27s The patch does not generate ASF License warnings.
        119m 36s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:612578f
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863511/MAPREDUCE-6673.2.patch
        JIRA Issue MAPREDUCE-6673
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 105f619af691 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4168805
        Default Java 1.8.0_121
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6960/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6960/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6960/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 14m 12s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 24s trunk passed +1 javadoc 0m 12s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 15s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 29s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 99m 45s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 119m 36s Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863511/MAPREDUCE-6673.2.patch JIRA Issue MAPREDUCE-6673 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 105f619af691 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4168805 Default Java 1.8.0_121 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6960/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6960/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6960/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 12m 46s trunk passed
        +1 compile 0m 21s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 26s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 21s trunk passed
        +1 javadoc 0m 9s trunk passed
        +1 mvninstall 0m 20s the patch passed
        +1 compile 0m 22s the patch passed
        +1 javac 0m 22s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 23s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 30s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 110m 56s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 26s The patch does not generate ASF License warnings.
        128m 56s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:612578f
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863511/MAPREDUCE-6673.2.patch
        JIRA Issue MAPREDUCE-6673
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c699aa51262c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8a1d748
        Default Java 1.8.0_121
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6959/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6959/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6959/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 12m 46s trunk passed +1 compile 0m 21s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 21s trunk passed +1 javadoc 0m 9s trunk passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 15s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 30s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 110m 56s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 128m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863511/MAPREDUCE-6673.2.patch JIRA Issue MAPREDUCE-6673 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c699aa51262c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8a1d748 Default Java 1.8.0_121 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6959/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6959/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6959/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        -1 mvninstall 5m 11s root in trunk failed.
        +1 compile 0m 34s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 35s trunk passed
        +1 mvneclipse 0m 21s trunk passed
        +1 findbugs 0m 46s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 29s the patch passed
        +1 compile 0m 24s the patch passed
        +1 javac 0m 24s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 18s the patch passed
        -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 31s the patch passed
        +1 javadoc 0m 9s the patch passed
        +1 unit 108m 20s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        120m 27s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:612578f
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863511/MAPREDUCE-6673.2.patch
        JIRA Issue MAPREDUCE-6673
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 08f5a4c0e30f 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a41f8dd
        Default Java 1.8.0_121
        mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/artifact/patchprocess/branch-mvninstall-root.txt
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. -1 mvninstall 5m 11s root in trunk failed. +1 compile 0m 34s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 35s trunk passed +1 mvneclipse 0m 21s trunk passed +1 findbugs 0m 46s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 29s the patch passed +1 compile 0m 24s the patch passed +1 javac 0m 24s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 18s the patch passed -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 31s the patch passed +1 javadoc 0m 9s the patch passed +1 unit 108m 20s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 120m 27s Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863511/MAPREDUCE-6673.2.patch JIRA Issue MAPREDUCE-6673 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 08f5a4c0e30f 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a41f8dd Default Java 1.8.0_121 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6958/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 23s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        -1 mvninstall 14m 26s root in trunk failed.
        +1 compile 0m 27s trunk passed
        +1 checkstyle 0m 20s trunk passed
        +1 mvnsite 0m 37s trunk passed
        +1 mvneclipse 0m 23s trunk passed
        +1 findbugs 0m 33s trunk passed
        +1 javadoc 0m 15s trunk passed
        +1 mvninstall 0m 37s the patch passed
        +1 compile 0m 33s the patch passed
        +1 javac 0m 33s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 1m 1s the patch passed
        +1 javadoc 0m 14s the patch passed
        -1 unit 1m 13s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 asflicense 0m 25s The patch does not generate ASF License warnings.
        23m 43s



        Reason Tests
        Failed junit tests hadoop.fs.TestFileSystem



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:612578f
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863507/MAPREDUCE-6673.2.patch
        JIRA Issue MAPREDUCE-6673
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b56da7a379b5 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a41f8dd
        Default Java 1.8.0_121
        mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/branch-mvninstall-root.txt
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. -1 mvninstall 14m 26s root in trunk failed. +1 compile 0m 27s trunk passed +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 0m 33s the patch passed +1 javac 0m 33s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 22s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 1m 1s the patch passed +1 javadoc 0m 14s the patch passed -1 unit 1m 13s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 23m 43s Reason Tests Failed junit tests hadoop.fs.TestFileSystem Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863507/MAPREDUCE-6673.2.patch JIRA Issue MAPREDUCE-6673 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b56da7a379b5 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a41f8dd Default Java 1.8.0_121 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6956/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 13m 4s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 26s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 26s trunk passed
        +1 javadoc 0m 14s trunk passed
        -1 mvninstall 0m 19s hadoop-mapreduce-client-jobclient in the patch failed.
        -1 compile 0m 18s hadoop-mapreduce-client-jobclient in the patch failed.
        -1 javac 0m 19s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 checkstyle 0m 14s the patch passed
        -1 mvnsite 0m 21s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 mvneclipse 0m 15s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 findbugs 0m 16s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 javadoc 0m 10s the patch passed
        -1 unit 0m 20s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        18m 37s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:612578f
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863507/MAPREDUCE-6673.2.patch
        JIRA Issue MAPREDUCE-6673
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 8c3b9d67bb2f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a41f8dd
        Default Java 1.8.0_121
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        compile https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-compile-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        javac https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-compile-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        mvnsite https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/whitespace-eol.txt
        findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 13m 4s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 26s trunk passed +1 javadoc 0m 14s trunk passed -1 mvninstall 0m 19s hadoop-mapreduce-client-jobclient in the patch failed. -1 compile 0m 18s hadoop-mapreduce-client-jobclient in the patch failed. -1 javac 0m 19s hadoop-mapreduce-client-jobclient in the patch failed. +1 checkstyle 0m 14s the patch passed -1 mvnsite 0m 21s hadoop-mapreduce-client-jobclient in the patch failed. +1 mvneclipse 0m 15s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 0m 16s hadoop-mapreduce-client-jobclient in the patch failed. +1 javadoc 0m 10s the patch passed -1 unit 0m 20s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 18m 37s Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863507/MAPREDUCE-6673.2.patch JIRA Issue MAPREDUCE-6673 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8c3b9d67bb2f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a41f8dd Default Java 1.8.0_121 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt compile https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-compile-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt javac https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-compile-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt mvnsite https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6957/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        The removed import is needed, MapredTestDriver does not compile now

        Show
        haibochen Haibo Chen added a comment - The removed import is needed, MapredTestDriver does not compile now
        Hide
        kasha Karthik Kambatla added a comment -

        Thanks for the review, Haibo. Updated patch address the suggestions, and also cleans up the imports.

        Show
        kasha Karthik Kambatla added a comment - Thanks for the review, Haibo. Updated patch address the suggestions, and also cleans up the imports.
        Hide
        haibochen Haibo Chen added a comment -

        Thanks [~kasha@apache.org] for the patch. Two nits,
        1) We should probably add description of the behavior of growing memory usage over time in the class javadoc
        2)

        LOG.info("Free = " + free + ". Creating 1 MB on the heap.");

        can be LOG.info("Free memory = " + free + " bytes. Creating 1MB on the heap");
        The patch no longer applies cleanly to trunk, otherwise LGTM.

        Show
        haibochen Haibo Chen added a comment - Thanks [~kasha@apache.org] for the patch. Two nits, 1) We should probably add description of the behavior of growing memory usage over time in the class javadoc 2) LOG.info("Free = " + free + ". Creating 1 MB on the heap."); can be LOG.info("Free memory = " + free + " bytes. Creating 1MB on the heap"); The patch no longer applies cleanly to trunk, otherwise LGTM.

          People

          • Assignee:
            kasha Karthik Kambatla
            Reporter:
            kasha Karthik Kambatla
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development