This patch fixes the NoSuchMethodError.
A similar patch fixed the issue I found in Flume.

I'm running on a mac and the tests seem to not terminate - this seems to be some issue in the minicluster code, and not a compatibility issue. It'd be great if someone could apply this patch and see if it works on Linux.

Forgot to mention this. To verify:

• rm -rf ~/.ivy2 (This is required, else any previous version of hbase in the ivy cache will stay and will not pull the new version).
• Clone hbase repo, checkout 0.94.2 tag, and build using mvn clean install -DskipTests -Dhadoop.profile=2.0
• Apply patch to sqoop trunk and build using hadoopversion=200

Seems like it is hitting gc during the run, at least it runs – it basically ends up pausing in the middle of the test

So this patch fixes the tests - they are being run, but this is what the JStack looks like:

"main" prio=5 tid=7fa142800000 nid=0x110391000 waiting on condition [11038f000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at java.lang.Thread.sleep(Thread.java:302)
	at java.util.concurrent.TimeUnit.sleep(TimeUnit.java:328)
	at org.apache.hadoop.hbase.util.RetryCounter.sleepUntilNextRetry(RetryCounter.java:54)
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:179)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:230)
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:82)
	- locked <77d5563f8> (a org.apache.hadoop.hbase.MasterAddressTracker)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:591)
	- locked <77d528ec8> (a org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:650)
	at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:110)
	at org.apache.sqoop.mapreduce.HBaseImportJob.jobSetup(HBaseImportJob.java:143)
	at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:209)
	at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:465)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
	at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:45)
	at com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:215)
	at com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:230)
	at com.cloudera.sqoop.hbase.TestHBaseImport.testBasicUsage(TestHBaseImport.java:38)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at junit.framework.TestCase.runTest(TestCase.java:168)
	at junit.framework.TestCase.runBare(TestCase.java:134)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at junit.framework.TestSuite.runTest(TestSuite.java:232)
	at junit.framework.TestSuite.run(TestSuite.java:227)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)

It looks like the test is not really able to run properly on the minicluster and may never have been able to. I am not familiar with the code in question here, but it looks like it ends up indefinitely for the master. Maybe this needs to be looked at: org.apache.sqoop.mapreduce.HBaseImportJob.jobSetup(HBaseImportJob.java:143).

Precise steps that I've done:

svn checkout http://svn.apache.org/repos/asf/hbase/tags/0.94.2 hbase-0.94.2
cd hbase-0.94.2
rm -rf ~/.m2 ~/.ivy2
mvn clean install -DskipTests -Dhadoop.profile=2.0

cd /path/to/sqoop
ant clean test -Dhadoopversion=200 -Dtestcase=TestHBaseImport                                                  

Unfortunately as was suggested in previous comments this tests has timed out for me (after 20 minutes).

It looks like I'm hitting the same issue in hadoop-20 profile too.

I'm not sure how to selectively disable certain tests when hadoop-23 and hadoop-2. If you do, please feel free to add to the patch and commit.

Hi Hari,
it seems really working on profile 200. Thank you very much for your effort to make this happen, I'm excited to see HBase tests working again.

I'm unfortunately getting compilation errors in profile 100:

    [javac] /home/jarcec/apache/repos/sqoop/src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java:230: cannot find symbol
    [javac] symbol  : method getTempDirectory()
    [javac] location: class org.apache.commons.io.FileUtils
    [javac]     File baseDir = FileUtils.getTempDirectory();
    [javac]                             ^

Would you mind taking a look?

Looks like there is something wrong with the commons io versions - probably due to the different versions required by Hadoop. I will remove that dependency by using the java.io.tempdir system property

The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=33a7a81412d789371a235323ac15f875e1870894

Thank you very much for your contribution Hari!

Jarcec

I've created SQOOP-731 to move out HBase tests to manual test set.

Integrated in Sqoop-ant-jdk-1.6-hadoop20 #337 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/337/)
SQOOP-725: Enable HBase tests in sqoop 1 (Revision 33a7a81412d789371a235323ac15f875e1870894)

Result = SUCCESS
jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=33a7a81412d789371a235323ac15f875e1870894
Files :

• src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java
• build.xml
• src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java
• src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java
• ivy/ivysettings.xml

Integrated in Sqoop-ant-jdk-1.6-hadoop23 #506 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/506/)
SQOOP-725: Enable HBase tests in sqoop 1 (Revision 33a7a81412d789371a235323ac15f875e1870894)

Result = SUCCESS
jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=33a7a81412d789371a235323ac15f875e1870894
Files :

• src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java
• build.xml
• src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java
• src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java
• ivy/ivysettings.xml

Integrated in Sqoop-ant-jdk-1.6-hadoop200 #340 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/340/)
SQOOP-725: Enable HBase tests in sqoop 1 (Revision 33a7a81412d789371a235323ac15f875e1870894)

Result = FAILURE
jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=33a7a81412d789371a235323ac15f875e1870894
Files :

• build.xml
• ivy/ivysettings.xml
• src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java
• src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java
• src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java

The failure on profile hadoop 200 is expected and will be fixed by SQOOP-731. I'm surprised though to see the profile hadoop 23 succeeded as I was expecting it's failure as well.

Integrated in Sqoop-ant-jdk-1.6-hadoop100 #329 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/329/)
SQOOP-725: Enable HBase tests in sqoop 1 (Revision 33a7a81412d789371a235323ac15f875e1870894)

Result = FAILURE
jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=33a7a81412d789371a235323ac15f875e1870894
Files :

• src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java
• src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java
• src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java
• ivy/ivysettings.xml
• build.xml

Jarek Jarcec Cecho Could you please re-run the hadoop-1 build? The build is weird, since some HBase tests are successful, and just one fails.

Yes of course, I've run it and it seems to be successful: https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/331/console

Thanks!