A dep. problem?

[ivy:resolve] 	==== maven2: tried
[ivy:resolve] 	  http://repo1.maven.org/maven2/org/apache/velocity/velocity/1.7/velocity-1.7.pom
[ivy:resolve] 		::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] 		::          UNRESOLVED DEPENDENCIES         ::
[ivy:resolve] 		::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] 		:: org.apache.velocity#velocity;1.7: not found
[ivy:resolve] 		::::::::::::::::::::::::::::::::::::::::::::::

Is this for 0.23 or 0.22?

This is .23 related. The root cause as far as I can see is the Mavenization of Hadoop. Ivy gets confused because of extra transitive dependencies that it fetches from the .23 pom files.

Roman: have you tried excluding the extra transitive dependencies?

Here's a patch to demonstrate the problem. Changing the Hadoop dependency to 0.23 necessitates an incompatible change to MockObjectFactory. This is similar to the changes covered in BIGTOP-162, and which are handled either via shims (an interface whose implementation is conditionally compiled according to the Hadoop target version) or via mock object libraries. Since the changes to Sqoop are only in the tests, using a mock object library is probably most appropriate. Are folks open to adding Mockito (say) as a Sqoop dependency?

The other problem, not fixed in the patch, is that the Hadoop test classes (in particular HadoopTestCase) are not yet published to a Maven repository, so it's not possible to compile TestDataDrivenDBInputFormat.

More precisely, the Hadoop test JARs are being published, but the HadoopTestCase class has not been moved to the Maven part of the build, so is missing from the test JARs. See HADOOP-7590.

It turns out that Mockito isn't needed, since MockObjectFactory was only being used by TestLargeObjectLoader, and then only to configure the output committer. I've simplified the test so that it doesn't use an output committer (since that isn't a part of the behaviour being tested), and now MockObjectFactory is no longer needed.

Also, HadoopTestCase wasn't really being used, so I've removed that dependency too. The tests that I've changed work on both 0.20 and 0.23, so I think this change can be committed. Please review.

Having a switch to build on either 0.20 or 0.23 should be tackled in another issue.

Reviewboard: https://reviews.apache.org/r/2758/

Apply SQOOP-354-0.23.0-SNAPSHOT.patch in addition to SQOOP-354.patch if you want to run against 0.23. (Not for commit.)

Here's an update to the latest patch that Tom posted. This makes Sqoop build. If you want to keep up with the latest versions of this patch check it out from Bigtop.

Integrated in Sqoop-jdk-1.6 #62 (See https://builds.apache.org/job/Sqoop-jdk-1.6/62/)
SQOOP-354 SQOOP needs to be made compatible with Hadoop .23 release

blee : http://svn.apache.org/viewvc/?view=rev&rev=1199149
Files :

• /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/lib/TestLargeObjectLoader.java
• /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java
• /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/testutil/MockObjectFactory.java

Closing this since it was committed in 1.4.0. Fixing tests is covered in SQOOP-397.