Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5625

TestFixedLengthInputFormat fails in jdk7 environment

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: None
    • Labels:
    • Target Version/s:

      Activity

      Hide
      Jonathan Eagles added a comment -

      In jdk7, I believe the way that junit does reflection on a class to get the test methods now returns the tests in random order (not file order. changed in jdk7). In jdk6, the tests were always return in file order(top to bottom.)

      Show
      Jonathan Eagles added a comment - In jdk7, I believe the way that junit does reflection on a class to get the test methods now returns the tests in random order (not file order. changed in jdk7). In jdk6, the tests were always return in file order(top to bottom.)
      Hide
      Hudson added a comment -

      SUCCESS: Integrated in Hadoop-Hdfs-trunk #1586 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1586/)
      MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733)

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Show
      Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1586 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1586/ ) MAPREDUCE-5625 . TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Hide
      Hudson added a comment -

      FAILURE: Integrated in Hadoop-Mapreduce-trunk #1612 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1612/)
      MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733)

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Show
      Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1612 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1612/ ) MAPREDUCE-5625 . TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Hide
      Hudson added a comment -

      SUCCESS: Integrated in Hadoop-Yarn-trunk #395 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/395/)
      MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733)

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Show
      Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #395 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/395/ ) MAPREDUCE-5625 . TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Hide
      Mariappan Asokan added a comment -

      Hi Jonathan,
      Thanks for committing this. I am wondering how the tests were successful consistently under JDK 1.6. Anyway, I will keep in mind to build and run tests with JDK 7 for future Jiras I will be working on.

      Show
      Mariappan Asokan added a comment - Hi Jonathan, Thanks for committing this. I am wondering how the tests were successful consistently under JDK 1.6. Anyway, I will keep in mind to build and run tests with JDK 7 for future Jiras I will be working on.
      Hide
      Jonathan Eagles added a comment -

      I verified this fix works successfully in my jdk7 environment. Until apache builds start running with jdk7, I feel we will continue to see issue like this.

      Show
      Jonathan Eagles added a comment - I verified this fix works successfully in my jdk7 environment. Until apache builds start running with jdk7, I feel we will continue to see issue like this.
      Hide
      Hudson added a comment -

      SUCCESS: Integrated in Hadoop-trunk-Commit #4750 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4750/)
      MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733)

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Show
      Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #4750 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4750/ ) MAPREDUCE-5625 . TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542733 ) /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFixedLengthInputFormat.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestFixedLengthInputFormat.java
      Hide
      Jonathan Eagles added a comment -

      +1. lgtm. Thanks for fixing this so quickly, Mariappan Asokan.

      Show
      Jonathan Eagles added a comment - +1. lgtm. Thanks for fixing this so quickly, Mariappan Asokan .
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12614253/mapreduce-5625_v1.patch
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 2 new or modified test files.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 eclipse:eclipse. The patch built with eclipse:eclipse.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

      org.apache.hadoop.mapred.TestJobCleanup

      The following test timeouts occurred in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

      org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl

      +1 contrib tests. The patch passed contrib unit tests.

      Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4208//testReport/
      Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4208//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614253/mapreduce-5625_v1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: org.apache.hadoop.mapred.TestJobCleanup The following test timeouts occurred in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4208//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4208//console This message is automatically generated.
      Hide
      Mariappan Asokan added a comment -

      Hi Sandy,
      The problem was caused by improper use of a static Configuration object across different tests. I fixed it by making sure that the Configuration is cloned for each job run in the tests.

      Please review the fix.

      – Asokan

      Show
      Mariappan Asokan added a comment - Hi Sandy, The problem was caused by improper use of a static Configuration object across different tests. I fixed it by making sure that the Configuration is cloned for each job run in the tests. Please review the fix. – Asokan
      Hide
      Sandy Ryza added a comment -
      Show
      Sandy Ryza added a comment - Thanks Mariappan Asokan
      Hide
      Mariappan Asokan added a comment -

      Sandy,
      I downloaded JDK 1.7 and ran the test and got it failing. I am looking into it.
      Thanks.

      Show
      Mariappan Asokan added a comment - Sandy, I downloaded JDK 1.7 and ran the test and got it failing. I am looking into it. Thanks.
      Hide
      Mariappan Asokan added a comment -

      Sandy,
      Where can I access the log files that show failures. I can take a look at them.
      Thanks.

      Show
      Mariappan Asokan added a comment - Sandy, Where can I access the log files that show failures. I can take a look at them. Thanks.
      Hide
      Sandy Ryza added a comment -

      I recently committed this in MAPREDUCE-1176. Mariappan Asokan, any idea why these tests might be failing if run out of the order given in the file?

      Show
      Sandy Ryza added a comment - I recently committed this in MAPREDUCE-1176 . Mariappan Asokan , any idea why these tests might be failing if run out of the order given in the file?

        People

        • Assignee:
          Mariappan Asokan
          Reporter:
          Jonathan Eagles
        • Votes:
          0 Vote for this issue
          Watchers:
          5 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development