[SQOOP-1053] Fix sqoop to work with Hadoop 2.0.4 alpha

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.3
• Fix Version/s: 1.4.4
• Component/s: test

Environment

Hadoop 2.0.4

• Assignee: Venkat Ranganathan
• Reporter: Venkat Ranganathan
• Created: 21/May/13 00:51
• Updated: 22/May/13 14:10
• Resolved: 22/May/13 08:13

Description

I updated the build and ivy settings to build Sqoop with Hadoop 2.0.4 alpha.

One of the compression tests (testBzip2TextCompression) fails with the following stack trace

Testcase: testBzip2TextCompression took 1.21 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2Decompressor(Bzip2Factory.java:123)
at org.apache.hadoop.io.compress.BZip2Codec.createDecompressor(BZip2Codec.java:250)
at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:156)
at com.cloudera.sqoop.TestCompression.runTextCompressionTest(TestCompression.java:170)
at com.cloudera.sqoop.TestCompression.testBzip2TextCompression(TestCompression.java:189)

The reason is in 2.0.4, with native BZip2 codec, the configuration is checked and we don't do setConf for BZip2 codec. Fixed the test to set the configuration irrespective of whether it was passed in or we created the codec.

Issue Links

• links to: Review board link