[HIVE-9588] Reimplement HCatClientHMSImpl.dropPartitions() with HMSC.dropPartitions()

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14.0
• Fix Version/s: 1.2.0
• Component/s: HCatalog, Metastore, Thrift API
• Labels: TODOC1.2
• Assignee: Mithun Radhakrishnan
• Reporter: Mithun Radhakrishnan
• Created: 05/Feb/15 06:39
• Updated: 18/May/15 19:52
• Resolved: 09/Mar/15 06:02

Description

HCatClientHMSImpl.dropPartitions() currently has an embarrassingly inefficient implementation. The partial partition-spec is converted into a filter-string. The partitions are fetched from the server, and then dropped one by one.

Here's a reimplementation that uses the ExprNode-based HiveMetaStoreClient.dropPartitions(). It cuts out the excessive back-and-forth between the HMS and the client-side. It also reduces the memory footprint (from loading all the partitions that are to be dropped).

Issue Links

• depends upon: HIVE-9628 HiveMetaStoreClient.dropPartitions(...List<ObjectPair<Integer,byte[]>>...) doesn't take (boolean needResult)
• is depended upon by: HIVE-9629 HCatClient.dropPartitions() needs speeding up.
• is depended upon by: HIVE-9633 Add HCatClient.dropPartitions() overload to skip deletion of partition-directories.