Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.3-alpha, 0.23.6
    • Component/s: applications
    • Labels:
      None

      Description

      Added some tests for org.apache.hadoop.tools

      1. YARN-204-branch-0.23.patch
        29 kB
        Aleksey Gorshkov
      2. YARN-204-branch-2.patch
        20 kB
        Aleksey Gorshkov
      3. YARN-204-trunk.patch
        29 kB
        Aleksey Gorshkov
      4. YARN-204-branch-0.23-a.patch
        31 kB
        Aleksey Gorshkov
      5. YARN-204-branch-2-a.patch
        22 kB
        Aleksey Gorshkov
      6. YARN-204-trunk-a.patch
        31 kB
        Aleksey Gorshkov
      7. YARN-204-branch-0.23-b.patch
        28 kB
        Aleksey Gorshkov
      8. YARN-204-branch-2-b.patch
        28 kB
        Aleksey Gorshkov
      9. YARN-204-trunk-b.patch
        28 kB
        Aleksey Gorshkov

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #449 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/449/)
        svn merge -c 1414239 FIXES: YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby) (Revision 1414243)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414243
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-archives/src/test/java/org/apache/hadoop/tools/TestHadoopArchives.java
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestExternalCall.java
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestCopyFiles.java
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestDistCh.java
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestLogalyzer.java
        • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #449 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/449/ ) svn merge -c 1414239 FIXES: YARN-204 . test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby) (Revision 1414243) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414243 Files : /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-archives/src/test/java/org/apache/hadoop/tools/TestHadoopArchives.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestExternalCall.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestCopyFiles.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestDistCh.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestLogalyzer.java /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #50 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/50/)
        YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby) (Revision 1414239)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414239
        Files :

        • /hadoop/common/trunk/hadoop-tools/hadoop-archives/src/test/java/org/apache/hadoop/tools/TestHadoopArchives.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestExternalCall.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestCopyFiles.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestDistCh.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestLogalyzer.java
        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Yarn-trunk #50 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/50/ ) YARN-204 . test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby) (Revision 1414239) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414239 Files : /hadoop/common/trunk/hadoop-tools/hadoop-archives/src/test/java/org/apache/hadoop/tools/TestHadoopArchives.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestExternalCall.java /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestCopyFiles.java /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestDistCh.java /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestLogalyzer.java /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3061 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3061/)
        YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby) (Revision 1414239)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414239
        Files :

        • /hadoop/common/trunk/hadoop-tools/hadoop-archives/src/test/java/org/apache/hadoop/tools/TestHadoopArchives.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestExternalCall.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestCopyFiles.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestDistCh.java
        • /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestLogalyzer.java
        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - Integrated in Hadoop-trunk-Commit #3061 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3061/ ) YARN-204 . test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby) (Revision 1414239) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414239 Files : /hadoop/common/trunk/hadoop-tools/hadoop-archives/src/test/java/org/apache/hadoop/tools/TestHadoopArchives.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestExternalCall.java /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestCopyFiles.java /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestDistCh.java /hadoop/common/trunk/hadoop-tools/hadoop-extras/src/test/java/org/apache/hadoop/tools/TestLogalyzer.java /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        Hide
        revans2 Robert Joseph Evans added a comment -

        Thanks Aleksey,

        I put this into trunk, branch-2 and branch-0.23

        Show
        revans2 Robert Joseph Evans added a comment - Thanks Aleksey, I put this into trunk, branch-2 and branch-0.23
        Hide
        revans2 Robert Joseph Evans added a comment -

        +1 the new changes look better. I'll check this in.

        Show
        revans2 Robert Joseph Evans added a comment - +1 the new changes look better. I'll check this in.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12554664/YARN-204-trunk-b.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-archives hadoop-tools/hadoop-distcp hadoop-tools/hadoop-extras.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/162//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/162//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554664/YARN-204-trunk-b.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-tools/hadoop-archives hadoop-tools/hadoop-distcp hadoop-tools/hadoop-extras. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/162//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/162//console This message is automatically generated.
        Hide
        aleksgor Aleksey Gorshkov added a comment -

        OK. I've fixed code .
        About DistCh. The code with Random data have been wrote early. I've rewrote this code

        Show
        aleksgor Aleksey Gorshkov added a comment - OK. I've fixed code . About DistCh. The code with Random data have been wrote early. I've rewrote this code
        Hide
        revans2 Robert Joseph Evans added a comment -

        I have a few more comments. Sorry to keep giving you more things to do. These are mostly just minor cleanup things. I am a bit confused by TestExternalCall and what exactly it is trying to test. I think I understand it, but it would be nice to have a comment in there that explains what it is intended to test.

        Also inside the tests in several places in the test you catch(Exception e) and then fail the test. I personally think it is cleaner to just let the test throw the exception and have JUnit report the "Error".

        In Test DistCh you are using a Random. I am glad that there is a way to override the seed, but I think it would be best to have a hard coded seed instead of just outputting it in the logs.

        There are a few places that there is old code that is commented out. Please just remove that code.

        Show
        revans2 Robert Joseph Evans added a comment - I have a few more comments. Sorry to keep giving you more things to do. These are mostly just minor cleanup things. I am a bit confused by TestExternalCall and what exactly it is trying to test. I think I understand it, but it would be nice to have a comment in there that explains what it is intended to test. Also inside the tests in several places in the test you catch(Exception e) and then fail the test. I personally think it is cleaner to just let the test throw the exception and have JUnit report the "Error". In Test DistCh you are using a Random. I am glad that there is a way to override the seed, but I think it would be best to have a hard coded seed instead of just outputting it in the logs. There are a few places that there is old code that is commented out. Please just remove that code.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12554163/YARN-204-trunk-a.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-archives hadoop-tools/hadoop-distcp hadoop-tools/hadoop-extras.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/159//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/159//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554163/YARN-204-trunk-a.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-tools/hadoop-archives hadoop-tools/hadoop-distcp hadoop-tools/hadoop-extras. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/159//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/159//console This message is automatically generated.
        Hide
        aleksgor Aleksey Gorshkov added a comment -

        YARN-204-trunk-a.patch - patch for trunk .
        YARN-204-branch-2-a.patch - patch for branch-2
        YARN-204-branch-0.23-a.patch - patch for branch-0.23
        OK. I've fixed testMapCount in TestCopyFiles.
        patches ready for commit

        Show
        aleksgor Aleksey Gorshkov added a comment - YARN-204 -trunk-a.patch - patch for trunk . YARN-204 -branch-2-a.patch - patch for branch-2 YARN-204 -branch-0.23-a.patch - patch for branch-0.23 OK. I've fixed testMapCount in TestCopyFiles. patches ready for commit
        Hide
        revans2 Robert Joseph Evans added a comment -

        I have one comment testMapCount in TestCopyFiles got turned into tesMapCount. Without the t at the beginning the test is not run. What is more this test fails with this patch and the t, but passed previously. Please fix it.

        Show
        revans2 Robert Joseph Evans added a comment - I have one comment testMapCount in TestCopyFiles got turned into tesMapCount. Without the t at the beginning the test is not run. What is more this test fails with this patch and the t, but passed previously. Please fix it.
        Hide
        revans2 Robert Joseph Evans added a comment -

        Sorry I was off a bit on the last comment TestCopyFiles appears to have been a random error of some sort where the MR job did not launch. I reran it and it passed both with your patch and without it. TestCopyListing is failing on trunk right now and is probably not related.

        Show
        revans2 Robert Joseph Evans added a comment - Sorry I was off a bit on the last comment TestCopyFiles appears to have been a random error of some sort where the MR job did not launch. I reran it and it passed both with your patch and without it. TestCopyListing is failing on trunk right now and is probably not related.
        Hide
        revans2 Robert Joseph Evans added a comment -

        Both of those tests seem to be failing in trunk right now anyways. they were not caused by your patches.

        Show
        revans2 Robert Joseph Evans added a comment - Both of those tests seem to be failing in trunk right now anyways. they were not caused by your patches.
        Hide
        aleksgor Aleksey Gorshkov added a comment -

        I do not understand why org.apache.hadoop.tools.TestCopyListing.testDuplicates is failed
        I have not changed the org.apache.hadoop.tools.TestCopyListing .

        Show
        aleksgor Aleksey Gorshkov added a comment - I do not understand why org.apache.hadoop.tools.TestCopyListing.testDuplicates is failed I have not changed the org.apache.hadoop.tools.TestCopyListing .
        Hide
        aleksgor Aleksey Gorshkov added a comment -

        YARN-204-trunk.patch - patch for trunk .
        YARN-204-branch-2.patch - patch for branch-2
        YARN-204-branch-0.23.patch - patch for branch-0.23

        Show
        aleksgor Aleksey Gorshkov added a comment - YARN-204 -trunk.patch - patch for trunk . YARN-204 -branch-2.patch - patch for branch-2 YARN-204 -branch-0.23.patch - patch for branch-0.23
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12552294/YARN-204-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-tools/hadoop-archives hadoop-tools/hadoop-distcp hadoop-tools/hadoop-extras:

        org.apache.hadoop.tools.TestCopyListing
        org.apache.hadoop.tools.TestCopyFiles

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/135//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/135//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12552294/YARN-204-trunk.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-tools/hadoop-archives hadoop-tools/hadoop-distcp hadoop-tools/hadoop-extras: org.apache.hadoop.tools.TestCopyListing org.apache.hadoop.tools.TestCopyFiles +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/135//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/135//console This message is automatically generated.

          People

          • Assignee:
            aleksgor Aleksey Gorshkov
            Reporter:
            aleksgor Aleksey Gorshkov
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development