Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: 0.98.0, 0.94.8, 0.95.1
    • Fix Version/s: None
    • Component/s: mapreduce, test
    • Labels:
      None

      Description

      It looks like TestImportTsv can be fixed once HBASE-8453 is applied with this additional delta:

      diff --git a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java b/src/test/java/org/
      index 77d044b..0da6f0a 100644
      --- a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
      +++ b/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
      @@ -26,6 +26,7 @@ import java.util.ArrayList;
       import org.apache.commons.logging.Log;
       import org.apache.commons.logging.LogFactory;
       import org.apache.hadoop.hbase.*;
      +import org.apache.hadoop.mapred.JobConf;
       import org.apache.hadoop.mapreduce.Job;
       import org.apache.hadoop.fs.FSDataOutputStream;
       import org.apache.hadoop.fs.Path;
      @@ -289,7 +290,10 @@ public class TestImportTsv {
               LOG.info("set the hbaseAdmin");
               ImportTsv.createHbaseAdmin(conf);
             }
      -      Job job = ImportTsv.createSubmittableJob(conf, args);
      +
      +      JobConf jobConf = htu1.getMRCluster().createJobConf();
      +      HBaseConfiguration.merge(jobConf, conf);
      +      Job job = ImportTsv.createSubmittableJob(jobConf, args);
             job.waitForCompletion(false);
             assertTrue(job.isSuccessful());
      

      Tested with Hadoop 2.0.4 + HBase 0.94.7.

      1. 8454-branch-0.94.patch
        1 kB
        Andrew Purtell
      2. 8454.patch
        2 kB
        Andrew Purtell

        Activity

        Hide
        Andrew Purtell added a comment -

        Maybe not. It worked locally but I got a failure report back from a private Jenkins. Will look into it.

        Show
        Andrew Purtell added a comment - Maybe not. It worked locally but I got a failure report back from a private Jenkins. Will look into it.
        Hide
        Andrew Purtell added a comment -

        My bad. The patch is a fix. The (other) failure is an integration test, now that all unit tests are passing.

        Show
        Andrew Purtell added a comment - My bad. The patch is a fix. The (other) failure is an integration test, now that all unit tests are passing.
        Hide
        stack added a comment -

        Go for it Andrew Purtell

        Show
        stack added a comment - Go for it Andrew Purtell
        Hide
        Andrew Purtell added a comment -

        Alright, I'll put this and HBASE-8453 in through trunk today.

        Show
        Andrew Purtell added a comment - Alright, I'll put this and HBASE-8453 in through trunk today.
        Hide
        Andrew Purtell added a comment -

        Have to do it a bit differently for trunk. Patch includes the HTU change from HBASE-8453 so it can be independently testable in precommit. Tested locally with 2.0.4-alpha. Let's see if HadoopQA picks it up (for Hadoop 1).

        Show
        Andrew Purtell added a comment - Have to do it a bit differently for trunk. Patch includes the HTU change from HBASE-8453 so it can be independently testable in precommit. Tested locally with 2.0.4-alpha. Let's see if HadoopQA picks it up (for Hadoop 1).
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12581026/8454.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.backup.TestHFileArchiving
        org.apache.hadoop.hbase.util.TestHBaseFsck

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//testReport/
        Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12581026/8454.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.backup.TestHFileArchiving org.apache.hadoop.hbase.util.TestHBaseFsck Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5489//console This message is automatically generated.
        Hide
        Andrew Purtell added a comment -

        TestHFileArchiving or TestHBaseFsck may need a look but this change is good. Will commit soon.

        Show
        Andrew Purtell added a comment - TestHFileArchiving or TestHBaseFsck may need a look but this change is good. Will commit soon.
        Hide
        Andrew Purtell added a comment -

        Committed.

        Show
        Andrew Purtell added a comment - Committed.
        Hide
        Andrew Purtell added a comment -

        Reopening, see HBASE-8453.

        Show
        Andrew Purtell added a comment - Reopening, see HBASE-8453 .
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95 #168 (See https://builds.apache.org/job/hbase-0.95/168/)
        HBASE-8454. TestImportTsv failing due to configuration issues (Revision 1477378)

        Result = SUCCESS
        apurtell :
        Files :

        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in hbase-0.95 #168 (See https://builds.apache.org/job/hbase-0.95/168/ ) HBASE-8454 . TestImportTsv failing due to configuration issues (Revision 1477378) Result = SUCCESS apurtell : Files : /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Andrew Purtell added a comment -

        Reverting, see HBASE-8453.

        Show
        Andrew Purtell added a comment - Reverting, see HBASE-8453 .
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #4086 (See https://builds.apache.org/job/HBase-TRUNK/4086/)
        HBASE-8454. TestImportTsv failing due to configuration issues (Revision 1477377)

        Result = FAILURE
        apurtell :
        Files :

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #4086 (See https://builds.apache.org/job/HBase-TRUNK/4086/ ) HBASE-8454 . TestImportTsv failing due to configuration issues (Revision 1477377) Result = FAILURE apurtell : Files : /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #970 (See https://builds.apache.org/job/HBase-0.94/970/)
        Revert HBASE-8453, HBASE-8454. (Revision 1477450)
        HBASE-8454. TestImportTsv failing due to configuration issues (Revision 1477379)

        Result = FAILURE
        apurtell :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java

        apurtell :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #970 (See https://builds.apache.org/job/HBase-0.94/970/ ) Revert HBASE-8453 , HBASE-8454 . (Revision 1477450) HBASE-8454 . TestImportTsv failing due to configuration issues (Revision 1477379) Result = FAILURE apurtell : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java apurtell : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95-on-hadoop2 #85 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/85/)
        Revert HBASE-8453, HBASE-8454. (Revision 1477449)
        HBASE-8454. TestImportTsv failing due to configuration issues (Revision 1477378)

        Result = FAILURE
        apurtell :
        Files :

        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java

        apurtell :
        Files :

        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #85 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/85/ ) Revert HBASE-8453 , HBASE-8454 . (Revision 1477449) HBASE-8454 . TestImportTsv failing due to configuration issues (Revision 1477378) Result = FAILURE apurtell : Files : /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java apurtell : Files : /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #515 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/515/)
        Revert HBASE-8453, HBASE-8454. (Revision 1477448)
        HBASE-8454. TestImportTsv failing due to configuration issues (Revision 1477377)

        Result = FAILURE
        apurtell :
        Files :

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java

        apurtell :
        Files :

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #515 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/515/ ) Revert HBASE-8453 , HBASE-8454 . (Revision 1477448) HBASE-8454 . TestImportTsv failing due to configuration issues (Revision 1477377) Result = FAILURE apurtell : Files : /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java apurtell : Files : /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95 #169 (See https://builds.apache.org/job/hbase-0.95/169/)
        Revert HBASE-8453, HBASE-8454. (Revision 1477449)

        Result = SUCCESS
        apurtell :
        Files :

        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in hbase-0.95 #169 (See https://builds.apache.org/job/hbase-0.95/169/ ) Revert HBASE-8453 , HBASE-8454 . (Revision 1477449) Result = SUCCESS apurtell : Files : /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #4087 (See https://builds.apache.org/job/HBase-TRUNK/4087/)
        Revert HBASE-8453, HBASE-8454. (Revision 1477448)

        Result = FAILURE
        apurtell :
        Files :

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #4087 (See https://builds.apache.org/job/HBase-TRUNK/4087/ ) Revert HBASE-8453 , HBASE-8454 . (Revision 1477448) Result = FAILURE apurtell : Files : /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Andrew Purtell added a comment -

        Unassigning for now, will circle back to this next week.

        Show
        Andrew Purtell added a comment - Unassigning for now, will circle back to this next week.
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #141 (See https://builds.apache.org/job/HBase-0.94-security/141/)
        Revert HBASE-8453, HBASE-8454. (Revision 1477450)
        HBASE-8454. TestImportTsv failing due to configuration issues (Revision 1477379)

        Result = FAILURE
        apurtell :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java

        apurtell :
        Files :

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #141 (See https://builds.apache.org/job/HBase-0.94-security/141/ ) Revert HBASE-8453 , HBASE-8454 . (Revision 1477450) HBASE-8454 . TestImportTsv failing due to configuration issues (Revision 1477379) Result = FAILURE apurtell : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java apurtell : Files : /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
        Hide
        Lars Hofhansl added a comment -

        Moving to 0.94.9

        Show
        Lars Hofhansl added a comment - Moving to 0.94.9
        Hide
        Lars Hofhansl added a comment -

        Removed from 0.94. I have not seen this failing recently.

        Show
        Lars Hofhansl added a comment - Removed from 0.94. I have not seen this failing recently.
        Hide
        Andrew Purtell added a comment -

        Effectively resolved by revert.

        Show
        Andrew Purtell added a comment - Effectively resolved by revert.

          People

          • Assignee:
            Unassigned
            Reporter:
            Andrew Purtell
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development