Sqoop
  1. Sqoop
  2. SQOOP-1322

Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.4
    • Fix Version/s: 1.4.5
    • Component/s: None
    • Labels:
      None
    • Environment:

      All

      Description

      SQOOP-931 introduced HCatalog integration with Sqoop. Since then there have been enhancements to HCatalog in Hive. This is to update the HCatalog support in Sqoop

      1. SQOOP-1322-2.patch
        143 kB
        Venkat Ranganathan
      2. SQOOP-1322-3.patch
        146 kB
        Venkat Ranganathan

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Hide
          Venkat Ranganathan added a comment -

          Thank you Jarek Jarcec Cecho for reviewing and committing it.

          Show
          Venkat Ranganathan added a comment - Thank you Jarek Jarcec Cecho for reviewing and committing it.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #860 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/860/)
          SQOOP-1322: Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3)

          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java
          • src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java
          • src/test/org/apache/sqoop/hcat/HCatalogImportTest.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • ivy.xml
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java
          • src/test/com/cloudera/sqoop/TestConnFactory.java
          • build.xml
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java
          • src/test/org/apache/sqoop/hcat/HCatalogExportTest.java
          • src/docs/user/hcatalog.txt
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java
          • src/java/org/apache/sqoop/tool/CodeGenTool.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #860 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/860/ ) SQOOP-1322 : Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3 ) src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java src/test/org/apache/sqoop/hcat/HCatalogImportTest.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java src/java/org/apache/sqoop/manager/ConnManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java ivy.xml src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java src/test/com/cloudera/sqoop/TestConnFactory.java build.xml src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java src/test/org/apache/sqoop/hcat/HCatalogExportTest.java src/docs/user/hcatalog.txt src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/ExportTool.java src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java src/java/org/apache/sqoop/tool/CodeGenTool.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1098 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1098/)
          SQOOP-1322: Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3)

          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java
          • build.xml
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java
          • src/test/com/cloudera/sqoop/TestConnFactory.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java
          • src/java/org/apache/sqoop/tool/CodeGenTool.java
          • src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java
          • src/test/org/apache/sqoop/hcat/HCatalogImportTest.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/docs/user/hcatalog.txt
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/test/org/apache/sqoop/hcat/HCatalogExportTest.java
          • ivy.xml
          • src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java
          Show
          Hudson added a comment - FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1098 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1098/ ) SQOOP-1322 : Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3 ) src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/manager/ConnManager.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java build.xml src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java src/test/com/cloudera/sqoop/TestConnFactory.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java src/java/org/apache/sqoop/tool/CodeGenTool.java src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java src/test/org/apache/sqoop/hcat/HCatalogImportTest.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java src/java/org/apache/sqoop/SqoopOptions.java src/docs/user/hcatalog.txt src/java/org/apache/sqoop/tool/ExportTool.java src/test/org/apache/sqoop/hcat/HCatalogExportTest.java ivy.xml src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #895 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/895/)
          SQOOP-1322: Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3)

          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java
          • src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java
          • build.xml
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java
          • src/test/org/apache/sqoop/hcat/HCatalogExportTest.java
          • ivy.xml
          • src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java
          • src/test/org/apache/sqoop/hcat/HCatalogImportTest.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java
          • src/docs/user/hcatalog.txt
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java
          • src/java/org/apache/sqoop/tool/CodeGenTool.java
          • src/test/com/cloudera/sqoop/TestConnFactory.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java
          Show
          Hudson added a comment - FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #895 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/895/ ) SQOOP-1322 : Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3 ) src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java src/java/org/apache/sqoop/manager/ConnManager.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java build.xml src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java src/java/org/apache/sqoop/manager/SqlManager.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java src/test/org/apache/sqoop/hcat/HCatalogExportTest.java ivy.xml src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java src/test/org/apache/sqoop/hcat/HCatalogImportTest.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java src/docs/user/hcatalog.txt src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java src/java/org/apache/sqoop/tool/CodeGenTool.java src/test/com/cloudera/sqoop/TestConnFactory.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #901 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/901/)
          SQOOP-1322: Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3)

          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/tool/ImportTool.java
          • src/java/org/apache/sqoop/manager/ConnManager.java
          • src/test/org/apache/sqoop/hcat/HCatalogExportTest.java
          • src/test/com/cloudera/sqoop/TestConnFactory.java
          • src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java
          • src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java
          • build.xml
          • src/docs/user/hcatalog.txt
          • src/java/org/apache/sqoop/SqoopOptions.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java
          • src/test/org/apache/sqoop/hcat/HCatalogImportTest.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java
          • src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java
          • src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java
          • src/java/org/apache/sqoop/tool/CodeGenTool.java
          • ivy.xml
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java
          • src/java/org/apache/sqoop/tool/ExportTool.java
          • src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java
          • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #901 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/901/ ) SQOOP-1322 : Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3 ) src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatExportMapper.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/tool/ImportTool.java src/java/org/apache/sqoop/manager/ConnManager.java src/test/org/apache/sqoop/hcat/HCatalogExportTest.java src/test/com/cloudera/sqoop/TestConnFactory.java src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportHelper.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java build.xml src/docs/user/hcatalog.txt src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableHCatImportMapper.java src/test/org/apache/sqoop/hcat/HCatalogImportTest.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatExportManualTest.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java src/test/org/apache/sqoop/manager/netezza/DirectNetezzaHCatImportManualTest.java src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableTextImportMapper.java src/java/org/apache/sqoop/tool/CodeGenTool.java ivy.xml src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java src/java/org/apache/sqoop/tool/ExportTool.java src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportHelper.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java
          Hide
          Jarek Jarcec Cecho added a comment -

          Thank you for the hard work on this patch [~nrv]!

          Show
          Jarek Jarcec Cecho added a comment - Thank you for the hard work on this patch [~nrv] !
          Hide
          ASF subversion and git services added a comment -

          Commit c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3 in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c65b56f ]

          SQOOP-1322: Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions

          (Venkat Ranganathan via Jarek Jarcec Cecho)

          Show
          ASF subversion and git services added a comment - Commit c65b56f6f2a1c1bd75818c5ca2fb4b14eb958be3 in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c65b56f ] SQOOP-1322 : Enhance Sqoop HCatalog Integration to cover features introduced in newer Hive versions (Venkat Ranganathan via Jarek Jarcec Cecho)
          Hide
          Venkat Ranganathan added a comment -

          Cumulative patch for SQOOP-1322 and sub tasks

          Show
          Venkat Ranganathan added a comment - Cumulative patch for SQOOP-1322 and sub tasks

            People

            • Assignee:
              Venkat Ranganathan
              Reporter:
              Venkat Ranganathan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development