[SQOOP-3438] Sqoop Import with create hcatalog table for ORC will not work with Hive3 as the table created would be a ACID table and transactional

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.4.7
• Fix Version/s: 1.5.0
• Component/s: hive-integration
• Assignee: Dénes Bodó
• Reporter: Dénes Bodó
• Created: 08/May/19 13:15
• Updated: 14/Oct/21 14:26
• Resolved: 05/Jun/19 08:41

Description

PROBLEM: Running a sqoop import command with the option --create-hcatalog-table will not work due to the following reasons
When create-hcatalog-table is used it creates the table as a Managed ACID table.
HCatalog does not support transactional or bucketing table

So customer who need to create a ORC based table cannot use sqoop to create a ORC based table which means their existing code where if in case they use sqoop to create these tables would fail.

The current workaround is a two step process
1. Create the ORC table in hive with the keyword external and set transactional to false
2. Then use the sqoop command to load the data into the orc table.

The request is to add in an extra argument in the sqoop command line to specify that the table is external (example: --hcatalog-external-table )so we can use the option --hcatalog-storage-stanza "stored as orc tblproperties (\"transactional\"=\"false\")".

---

Thank you mbalakrishnan for your findings. This ticket is created based on your work.

Issue Links

• links to: GitHub Pull Request #80