Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 0.23.10, 2.3.0
    • Fix Version/s: 3.0.0, 0.23.10, 2.3.0
    • Component/s: None
    • Labels:
      None

      Description

      the test method run order on JDK7 is not fixed causing test method inter-dependencies to show themselves.

      1. MAPREDUCE-5587.patch
        6 kB
        Jonathan Eagles

        Activity

        Hide
        Jonathan Eagles added a comment -

        Through experimentation, only two different test method orders were shown to succeed.

        testFormatWithCustomSeparator
        testCompress
        testFormat
        testFormat
        testFormatWithCustomSeparator
        testCompress

        All other orders produced failures. Thus the inter-dependency is that testCompress must be run immediately after testFormatWithCustomSeparator.

        Dependencies Identified

        • All tests verify success by reading input from the same test file test.txt
        • testCompress relies on having the same custom separator as testFormatWithCustomSeparator
        • testCompress actually writes a file name test.txt.deflate
        Show
        Jonathan Eagles added a comment - Through experimentation, only two different test method orders were shown to succeed. testFormatWithCustomSeparator testCompress testFormat testFormat testFormatWithCustomSeparator testCompress All other orders produced failures. Thus the inter-dependency is that testCompress must be run immediately after testFormatWithCustomSeparator . Dependencies Identified All tests verify success by reading input from the same test file test.txt testCompress relies on having the same custom separator as testFormatWithCustomSeparator testCompress actually writes a file name test.txt.deflate
        Hide
        Ravi Prakash added a comment -

        Thanks Jon. Minor nits:

        • in testCompress(), you didn't have to remove separator. I'd rather check \u0001 than \t.
        •  output += out + "\n"; 
        • Do you think we should clean up the output files in an @After ? Or do you think its more useful to have them around after the test for debugging? I could go either way.

        Otherwise patch looks good.

        Show
        Ravi Prakash added a comment - Thanks Jon. Minor nits: in testCompress(), you didn't have to remove separator. I'd rather check \u0001 than \t. output += out + "\n" ; Do you think we should clean up the output files in an @After ? Or do you think its more useful to have them around after the test for debugging? I could go either way. Otherwise patch looks good.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12609020/MAPREDUCE-5587.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

        org.apache.hadoop.mapred.TestClientRedirect

        The following test timeouts occurred in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

        org.apache.hadoop.mapreduce.v2.TestUberAM
        org.apache.hadoop.conf.TestNoDefaultsJobConf
        org.apache.hadoop.mapred.TestClusterMapReduceTestCase

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4123//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4123//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12609020/MAPREDUCE-5587.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: org.apache.hadoop.mapred.TestClientRedirect The following test timeouts occurred in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: org.apache.hadoop.mapreduce.v2.TestUberAM org.apache.hadoop.conf.TestNoDefaultsJobConf org.apache.hadoop.mapred.TestClusterMapReduceTestCase +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4123//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4123//console This message is automatically generated.
        Hide
        Jonathan Eagles added a comment -

        Thanks for the review, Ravi.

        in testCompress(), you didn't have to remove separator. I'd rather check \u0001 than \t.

        It's true that removing the separator wasn't technically part of this JIRA, but I chose to take it out since the custom separator test was already testing that.

        Do you think we should clean up the output files in an @After ? Or do you think its more useful to have them around after the test for debugging? I could go either way.

        Cleaned up temp files from this test

        Show
        Jonathan Eagles added a comment - Thanks for the review, Ravi. in testCompress(), you didn't have to remove separator. I'd rather check \u0001 than \t. It's true that removing the separator wasn't technically part of this JIRA, but I chose to take it out since the custom separator test was already testing that. Do you think we should clean up the output files in an @After ? Or do you think its more useful to have them around after the test for debugging? I could go either way. Cleaned up temp files from this test
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12609203/MAPREDUCE-5587.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 javac. The patch appears to cause the build to fail.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4126//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12609203/MAPREDUCE-5587.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4126//console This message is automatically generated.
        Hide
        Ravi Prakash added a comment -

        Hi Jon!

        It's true that removing the separator wasn't technically part of this JIRA, but I chose to take it out since the custom separator test was already testing that.

        That sounds reasonable. Ok. Lets just go with the first patch. It looks good to me. +1. Thanks for it. Please check it in.

        Show
        Ravi Prakash added a comment - Hi Jon! It's true that removing the separator wasn't technically part of this JIRA, but I chose to take it out since the custom separator test was already testing that. That sounds reasonable. Ok. Lets just go with the first patch. It looks good to me. +1. Thanks for it. Please check it in.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #4631 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4631/)
        MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629)

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #4631 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4631/ ) MAPREDUCE-5587 . TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #367 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/367/)
        MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629)

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #367 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/367/ ) MAPREDUCE-5587 . TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-0.23-Build #765 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/765/)
        MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533634)

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-0.23-Build #765 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/765/ ) MAPREDUCE-5587 . TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533634 ) /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #1557 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1557/)
        MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629)

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1557 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1557/ ) MAPREDUCE-5587 . TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1583 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1583/)
        MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629)

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1583 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1583/ ) MAPREDUCE-5587 . TestTextOutputFormat fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1533629 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTextOutputFormat.java

          People

          • Assignee:
            Jonathan Eagles
            Reporter:
            Jonathan Eagles
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development