[OOZIE-2393] Allow table drop in hcat prepare

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 4.1.0, 4.2.0
• Fix Version/s: 5.0.0b1
• Component/s: core
• Labels: hcatalog prepare
• Assignee: Abhishek Bafna
• Reporter: Muhammad Ehsan ul Haque
• Created: 06/Nov/15 13:16
• Updated: 25/Jan/18 21:00
• Resolved: 22/Mar/17 14:50

Description

The hcat prepare only allows to drop partitions. It would be nice to also allow dropping of table, depending on the URL. The current format of the URL is

hcat://[metastore server]:[port]/[database name]/[table name]/[partkey1]=[value];[partkey2]=[value]

where at least one partition must be provided, otherwise the prepare step fails with the fololwing exception.

Starting the execution of prepare actions
Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and server=thrift://datavault-prod-app2.internal.machines:9083
Prepare execution in the Launcher Mapper has failed
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error trying to drop hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
org.apache.oozie.action.hadoop.LauncherException: Error trying to drop hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to drop hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
at org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
at org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
at org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
at org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
... 8 more
Caused by: java.net.URISyntaxException: URI path is not in expected format: hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
at org.apache.oozie.util.HCatURI.<init>(HCatURI.java:52)
at org.apache.oozie.util.HCatURI.<init>(HCatURI.java:48)
at org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
... 12 more

Oozie Launcher failed, finishing Hadoop job gracefully

Suggestion

In the URL if the partition is not provided then it should delete the entire table.

Issue Links

• links to: Review Board Link