Sqoop
  1. Sqoop
  2. SQOOP-462

Sqoop hbase Test Compilation failure.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.1-incubating, 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: hbase-integration
    • Labels:
      None

      Description

      Executing the following ant command, hbase test compilation failed.

      ant -d -f build.xml clean tar -Dmaven.test.skip=true -Dhadoopversion=100 -Dhadoop.version=1.0.1 -Dzookeeper.version=3.3.4 -Dhbase.version=0.92.0 -Dresolvers=internal

      Build failure:
      [javac] /root/HDP/hdp_builds/sqoop-branch-1.4.1/src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java:30: cannot find symbol
      [javac] symbol : class HBaseTestingUtility
      [javac] location: package org.apache.hadoop.hbase
      [javac] import org.apache.hadoop.hbase.HBaseTestingUtility;
      [javac] ^
      [javac] /root/HDP/hdp_builds/sqoop-branch-1.4.1/src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java:110: cannot find symbol
      [javac] symbol : class HBaseTestingUtility
      [javac] location: class com.cloudera.sqoop.hbase.HBaseTestCase
      [javac] private HBaseTestingUtility hbaseTestUtil;
      [javac] ^
      [javac] /root/HDP/hdp_builds/sqoop-branch-1.4.1/src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java:116: cannot find symbol
      [javac] symbol : class HBaseTestingUtility
      [javac] location: class com.cloudera.sqoop.hbase.HBaseTestCase
      [javac] hbaseTestUtil = new HBaseTestingUtility(conf);
      [javac] ^
      [javac] /root/HDP/hdp_builds/sqoop-branch-1.4.1/src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java:167: internal error; cannot instantiate Configuration(boolean) at org.apache.hadoop.conf.Configuration to ()
      [javac] HTable table = new HTable(new Configuration(
      [javac] ^
      [javac] Note: Some input files use or override a deprecated API.
      [javac] Note: Recompile with -Xlint:deprecation for details.
      [javac] Note: Some input files use unchecked or unsafe operations.
      [javac] Note: Recompile with -Xlint:unchecked for details.
      [javac] 4 errors

      1. Sqoop-462.patch
        0.6 kB
        Ashish Singh
      2. Sqoop-462.patch
        0.6 kB
        Ashish Singh

        Activity

        Hide
        Ashish Singh added a comment -

        Adding the ivysettings.xml path file.

        Show
        Ashish Singh added a comment - Adding the ivysettings.xml path file.
        Hide
        Arvind Prabhakar added a comment -

        Thanks for the patch Ashish. However, we cannot use this patch because while attaching it you did not check the checkbox next to "Grant license to ASF for inclusion in ASF works" option.

        If you reattach the patch with the license grant, we will be happy to take it forward.

        Show
        Arvind Prabhakar added a comment - Thanks for the patch Ashish. However, we cannot use this patch because while attaching it you did not check the checkbox next to "Grant license to ASF for inclusion in ASF works" option. If you reattach the patch with the license grant, we will be happy to take it forward.
        Hide
        Ashish Singh added a comment -

        Arvind,

        I have re-attached the Sqoop build patch. Also selected the "Grant license to ASF for inclusion in ASF works".

        Show
        Ashish Singh added a comment - Arvind, I have re-attached the Sqoop build patch. Also selected the "Grant license to ASF for inclusion in ASF works".
        Hide
        Arvind Prabhakar added a comment -

        Ashish - In order to test the patch, I wanted to reproduce the problem first. However, even after deleting my ~/.ivy2 and ~/.m2 directories - along with using a fresh checkout of the branch-1.4.1, I am unable to reproduce it.

        Can you confirm if this problem is reproducible, and if so, why is it that in your case the internal resolver chain is not going beyond the fs config?

        Show
        Arvind Prabhakar added a comment - Ashish - In order to test the patch, I wanted to reproduce the problem first. However, even after deleting my ~/.ivy2 and ~/.m2 directories - along with using a fresh checkout of the branch-1.4.1, I am unable to reproduce it. Can you confirm if this problem is reproducible, and if so, why is it that in your case the internal resolver chain is not going beyond the fs config?
        Hide
        Arvind Prabhakar added a comment -

        BTW - I am using ant 1.8.2 on Mac OSX.

        Show
        Arvind Prabhakar added a comment - BTW - I am using ant 1.8.2 on Mac OSX.
        Hide
        Giridharan Kesavan added a comment -

        steps to re-produce.

        mvn install on hbase 0.92.0 branch and update m2 cache.

        make sqoop use the filesystem resolver to resolve hbase.
        ant package -Dresolvers=internal

        sqoop build would fail with the error mentioned above.

        This happens when we do an offline sqoop build. In this case the -Dresolvers=internal is set and hence the fs (filesystem) resolver is used. The maven pattern is not set to consider the artifact classifier and this would just resolve the hbase jar in place of hbase test jar. By applying this patch the maven pattern for filesystem resolver is updated to match against artifacts with classifiers.

        Show
        Giridharan Kesavan added a comment - steps to re-produce. mvn install on hbase 0.92.0 branch and update m2 cache. make sqoop use the filesystem resolver to resolve hbase. ant package -Dresolvers=internal sqoop build would fail with the error mentioned above. This happens when we do an offline sqoop build. In this case the -Dresolvers=internal is set and hence the fs (filesystem) resolver is used. The maven pattern is not set to consider the artifact classifier and this would just resolve the hbase jar in place of hbase test jar. By applying this patch the maven pattern for filesystem resolver is updated to match against artifacts with classifiers.
        Hide
        Ashish Singh added a comment -

        Any idea when the patch will be committed.

        Show
        Ashish Singh added a comment - Any idea when the patch will be committed.
        Hide
        Cheolsoo Park added a comment -

        +1.

        I am sorry for the delay. The patch seems correct to me. I will commit it.

        Show
        Cheolsoo Park added a comment - +1. I am sorry for the delay. The patch seems correct to me. I will commit it.
        Hide
        Cheolsoo Park added a comment -

        The patch committed. Thank you very much, Ashish!

        Btw, I cannot set assignee to him. Can anyone add him to contributors please?

        Show
        Cheolsoo Park added a comment - The patch committed. Thank you very much, Ashish! Btw, I cannot set assignee to him. Can anyone add him to contributors please?
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #230 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/230/)
        SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402)

        Result = SUCCESS
        cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402
        Files :

        • ivy/ivysettings.xml
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #230 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/230/ ) SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402 Files : ivy/ivysettings.xml
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #221 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/221/)
        SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402)

        Result = SUCCESS
        cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402
        Files :

        • ivy/ivysettings.xml
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #221 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/221/ ) SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402 Files : ivy/ivysettings.xml
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #370 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/370/)
        SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402)

        Result = SUCCESS
        cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402
        Files :

        • ivy/ivysettings.xml
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #370 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/370/ ) SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402 Files : ivy/ivysettings.xml
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #220 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/220/)
        SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402)

        Result = SUCCESS
        cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402
        Files :

        • ivy/ivysettings.xml
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #220 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/220/ ) SQOOP-462 Sqoop hbase Test Compilation failure (Revision 97f272a4ba808a8630e4bd37d902a29d8a2b5402) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=97f272a4ba808a8630e4bd37d902a29d8a2b5402 Files : ivy/ivysettings.xml
        Hide
        Ashish Singh added a comment -

        Thanks Cheolsoo, for committing the patch.

        Show
        Ashish Singh added a comment - Thanks Cheolsoo, for committing the patch.

          People

          • Assignee:
            Ashish Singh
            Reporter:
            Ashish Singh
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development