Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1053

Fix sqoop to work with Hadoop 2.0.4 alpha

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.3
    • Fix Version/s: 1.4.4
    • Component/s: test
    • Labels:
      None
    • Environment:

      Hadoop 2.0.4

      Description

      I updated the build and ivy settings to build Sqoop with Hadoop 2.0.4 alpha.

      One of the compression tests (testBzip2TextCompression) fails with the following stack trace

      Testcase: testBzip2TextCompression took 1.21 sec
      Caused an ERROR
      null
      java.lang.NullPointerException
      at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
      at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2Decompressor(Bzip2Factory.java:123)
      at org.apache.hadoop.io.compress.BZip2Codec.createDecompressor(BZip2Codec.java:250)
      at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:156)
      at com.cloudera.sqoop.TestCompression.runTextCompressionTest(TestCompression.java:170)
      at com.cloudera.sqoop.TestCompression.testBzip2TextCompression(TestCompression.java:189)

      The reason is in 2.0.4, with native BZip2 codec, the configuration is checked and we don't do setConf for BZip2 codec. Fixed the test to set the configuration irrespective of whether it was passed in or we created the codec.

      1. SQOOP-1053.diff.2
        1 kB
        Venkat Ranganathan

        Issue Links

          Activity

          Hide
          venkatnrangan Venkat Ranganathan added a comment -

          Thanks Jarek. This is the latest patch.

          Show
          venkatnrangan Venkat Ranganathan added a comment - Thanks Jarek. This is the latest patch.
          Show
          jarcec Jarek Jarcec Cecho added a comment - The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=ead68ad37e1f7ecf03cba4718acbde8282887c9b Thank you Venkat! Jarcec
          Hide
          hudson Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop20 #635 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/635/)
          SQOOP-1053: Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b
          Files :

          • src/test/com/cloudera/sqoop/TestCompression.java
          • build.xml
          Show
          hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #635 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/635/ ) SQOOP-1053 : Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b Files : src/test/com/cloudera/sqoop/TestCompression.java build.xml
          Hide
          hudson Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop23 #843 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/843/)
          SQOOP-1053: Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b
          Files :

          • src/test/com/cloudera/sqoop/TestCompression.java
          • build.xml
          Show
          hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #843 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/843/ ) SQOOP-1053 : Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b Files : src/test/com/cloudera/sqoop/TestCompression.java build.xml
          Hide
          hudson Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop200 #636 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/636/)
          SQOOP-1053: Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b
          Files :

          • build.xml
          • src/test/com/cloudera/sqoop/TestCompression.java
          Show
          hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #636 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/636/ ) SQOOP-1053 : Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b Files : build.xml src/test/com/cloudera/sqoop/TestCompression.java
          Hide
          hudson Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop100 #639 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/639/)
          SQOOP-1053: Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b
          Files :

          • src/test/com/cloudera/sqoop/TestCompression.java
          • build.xml
          Show
          hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #639 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/639/ ) SQOOP-1053 : Fix sqoop to work with Hadoop 2.0.4 alpha (Revision ead68ad37e1f7ecf03cba4718acbde8282887c9b) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ead68ad37e1f7ecf03cba4718acbde8282887c9b Files : src/test/com/cloudera/sqoop/TestCompression.java build.xml

            People

            • Assignee:
              venkatnrangan Venkat Ranganathan
              Reporter:
              venkatnrangan Venkat Ranganathan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development